CN114740966A - Multi-modal image display control method and system and computer equipment - Google Patents
Multi-modal image display control method and system and computer equipment Download PDFInfo
- Publication number
- CN114740966A CN114740966A CN202011539861.9A CN202011539861A CN114740966A CN 114740966 A CN114740966 A CN 114740966A CN 202011539861 A CN202011539861 A CN 202011539861A CN 114740966 A CN114740966 A CN 114740966A
- Authority
- CN
- China
- Prior art keywords
- user
- display
- display control
- eye
- equipment
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
- G06F3/013—Eye tracking input arrangements
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
- G06F3/0484—Interaction techniques based on graphical user interfaces [GUI] for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range
Abstract
The multi-mode image display control method comprises the steps of acquiring eye gaze positions of a user in real time, generating an adjusting instruction according to the eye gaze positions of the user, and finally adjusting display layout of a display area according to the adjusting instruction. According to the multi-mode image display control method, the eye gaze position of the user is detected through eye movement identification, the position of the user gaze focus on the display device is further obtained, then the display layout of the display area is adjusted according to the position of the user gaze focus on the display device, the content blocks observed by the gaze focus are intelligently highlighted, the user is enabled to be concentrated on the interested content blocks, display of other content blocks is weakened, and the visual experience of the user is greatly improved.
Description
Technical Field
The application relates to the technical field of image fusion display control of multiple medical devices, in particular to a multi-modal image display control method, a multi-modal image display control system and computer equipment.
Background
With the development of technology, various display devices (such as displays, projectors, LED screens, etc.) are increasingly used in the field of medical diagnosis, and the same display device can simultaneously display a plurality of display contents. However, in practical applications, the conventional method for adjusting the size of the display screen can only manually adjust the size of the display screen, and cannot automatically highlight the content focused by the user according to the focus of the user.
Disclosure of Invention
Based on this, the application provides a multi-modal image display control method, a multi-modal image display control system and a computer device, aiming at the problem that the content concerned by the user cannot be automatically highlighted according to the gazing focus of the user in the traditional technical scheme.
A multi-modal image display control method, comprising:
acquiring the eye gaze position of a user in real time;
generating an adjusting instruction according to the eye gaze position of the user;
and adjusting the display layout of the display area according to the adjusting instruction.
In one embodiment, the step of acquiring the eye fixation position of the user in real time comprises the following steps:
capturing eyeball characteristics in real time through images or scanning and extracting the eyeball characteristics in real time, and obtaining the eye fixation position of the user according to eyeball characteristic analysis.
In one embodiment, the step of generating an adjustment instruction according to the eye fixation position of the user comprises:
and determining a display block watched by the eyes of the user on a display area according to the eye watching position of the user, and generating the adjusting instruction.
In one embodiment, the step of adjusting the display layout of the display area includes:
controlling the display block to highlight
In one embodiment, the highlighting includes one or more of zooming in, rotating, tilting, switching different sections, displaying a layout change, or highlighting a mark.
A multi-modality image display control system, comprising:
the camera shooting device is used for acquiring a face image of a user;
the detection equipment is used for acquiring the eye fixation position of the user in real time according to the face image of the user;
the content display control equipment is connected with the detection equipment and used for generating an adjusting instruction according to the eye gaze position of the user; and
and the display equipment is connected with the content display control equipment and is used for responding to the adjusting instruction so as to adjust the display layout of the display area of the display equipment.
In one embodiment, the detection apparatus comprises:
the camera equipment interface module is used for acquiring the user face image shot by the camera equipment; and
the user fixation position analysis module is connected with the camera equipment interface module and receives the user face image transmitted by the camera equipment interface module and is used for processing the user image so as to obtain the position of the user eye fixation focus on the display equipment.
In one embodiment, the content display control apparatus includes:
and the visual range acquisition module is connected with the user watching position analysis module and used for determining a display block watched by the user eyes on the display area according to the position of the user eye watching focus on the display equipment and generating the adjusting instruction.
In one embodiment, the display device includes:
and the adjusting module is connected with the visual range acquiring module and is used for controlling the display area block to be highlighted.
A computer device comprising a memory storing a computer program and a processor implementing the steps of the multimodal image display control method of any of the above embodiments when the computer program is executed.
According to the multi-mode image display control method, the eye gaze position of the user is obtained in real time, then the adjusting instruction is generated according to the eye gaze position of the user, and finally the display layout of the display area is adjusted according to the adjusting instruction. According to the multi-mode image display control method, the eye gaze position of the user is detected through eye movement identification, the position of the user gaze focus on the display device is further obtained, then the display layout of the display area is adjusted according to the position of the user gaze focus on the display device, the content blocks observed by the gaze focus are intelligently highlighted, the user is enabled to be concentrated on the interested content blocks, display of other content blocks is weakened, and the visual experience of the user is greatly improved.
Drawings
In order to more clearly illustrate the technical solutions in the embodiments or the conventional technologies of the present application, the drawings used in the descriptions of the embodiments or the conventional technologies will be briefly introduced below, it is obvious that the drawings in the following descriptions are only some embodiments of the present application, and for those skilled in the art, other drawings can be obtained according to the drawings without creative efforts.
FIG. 1 is a flow chart of a multi-modal image display control method according to an embodiment of the present application;
FIG. 2 is a basic flowchart of a multi-modal image display control method according to another embodiment of the present application;
FIG. 3 is a flowchart illustrating a method for obtaining a gaze location of an eye of a user according to an embodiment of the present application;
FIG. 4 is a schematic structural diagram of a multi-modal image display control system according to an embodiment of the present application;
fig. 5 is a schematic structural diagram of a multi-modal image display control system according to another embodiment of the present application.
Description of the main element reference numerals
10. An image pickup apparatus; 20. detecting equipment; 21. an image pickup device interface module; 22. a user gaze location analysis module; 30. a content display control device; 31. a visible range acquisition module; 40. a display device; 41. and an adjusting module.
Detailed Description
In order to make the aforementioned objects, features and advantages of the present application more comprehensible, embodiments accompanying the present application are described in detail below with reference to the accompanying drawings. In the following description, numerous specific details are set forth in order to provide a thorough understanding of the present application. This application is capable of embodiments in many different forms than those described herein and those skilled in the art will be able to make similar modifications without departing from the spirit of the application and it is therefore not intended to be limited to the embodiments disclosed below.
It will be understood that, as used herein, the terms "first," "second," and the like may be used herein to describe various elements, but these elements should not be limited by these terms. These terms are only used to distinguish one element from another. For example, a first acquisition module may be referred to as a second acquisition module, and similarly, a second acquisition module may be referred to as a first acquisition module, without departing from the scope of the present application. The first acquisition module and the second acquisition module are both acquisition modules, but are not the same acquisition module.
It will be understood that when an element is referred to as being "disposed on" another element, it can be directly on the other element or intervening elements may also be present. When an element is referred to as being "connected" to another element, it can be directly connected to the other element or intervening elements may also be present.
Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein in the description of the present application is for the purpose of describing particular embodiments only and is not intended to be limiting of the application. As used herein, the term "and/or" includes any and all combinations of one or more of the associated listed items.
The eye control technology is an emerging technology and mainly comprises eyeball identification and eyeball tracking; the electronic product is controlled by eyes mainly by means of an electronic camera, infrared detection or wireless sensing and the like. Eyeball tracking, also known as gaze tracking, is a technique that uses sensors to capture and extract eyeball characteristic information, measure eye movement, and estimate gaze direction or eye gaze location. When the eyes of a person look at different directions, the eyes can slightly change, the changes can generate extractable features, and the computer can extract the features through image capturing or scanning, so that the changes of the eyes can be tracked in real time, the state and the demand of a user can be predicted, the response is carried out, and the purpose of controlling the equipment by the eyes is achieved.
Referring to fig. 1, the present application provides a multi-modal image display control method. The multi-modal image display control method is an intelligent multi-modal image display control method for multi-medical equipment images based on an eye control technology. The multi-modal image display control method includes:
and S10, acquiring the eye fixation position of the user in real time.
And S20, generating an adjusting instruction according to the eye fixation position of the user.
And S30, adjusting the display layout of the display area according to the adjusting instruction.
Specifically, the medical diagnosis apparatus to which the multimodal image display control method of the present application is applied includes, but is not limited to, a digital subtraction angiography apparatus, a CT apparatus, a nuclear magnetic resonance apparatus, or a DR apparatus. The display device 40 in the medical diagnosis device includes, but is not limited to, a display, a projector, an LED screen, and other electronic products. The content displayed by the display device 40 includes, but is not limited to, X-ray images, CT images, ultrasound images, and the like.
The image pickup apparatus 10 generally has a camera. The camera comprises a general camera software and hardware module. The camera can be added with infrared sensors and other sensors for positioning and tracking eyeballs and calculating the attention points of users besides intelligent interactive operations such as video chatting, portrait and gesture recognition, the users can conveniently operate products and the like by using the eyeballs, meanwhile, the background is used for collecting and analyzing big data, and the camera has great commercial and technical value significance. In one implementation, an eye control instrument may be disposed on the display device 40, and an infrared light source, an optical sensor, an image processor, and a viewpoint calculation core are built in the eye control instrument; therefore, a corresponding image is created and projected on human eyes, the image information of the head and the eyes of the user is captured, the characteristics of the captured image are extracted, and the eye fixation position of the user is accurately calculated.
Referring to fig. 2, in one implementation, step S10 includes capturing eyeball features in real time through an image or extracting eyeball features through real-time scanning, and obtaining the eye gaze position of the user according to eyeball feature analysis. Specifically, the eyeball tracking technology is a scientific application technology, firstly, the tracking is carried out according to the characteristic changes of the eyeball and the periphery of the eyeball, secondly, the tracking is carried out according to the angle change of the iris, and thirdly, the characteristics are extracted by actively projecting light beams such as infrared rays and the like to the iris. It is understood that the user gaze location is the focal point of the user's eye gaze.
Optionally, firstly, the image of the face of the user is acquired in real time through the camera equipment interface module 21, and then eyeball features are extracted; eyeball characteristics include the location of the white, iris and pupil; secondly, converting the position information into coordinates in a space coordinate system of the display equipment according to the space coordinate system of the display equipment; and finally, calculating the position of the user watching position on the display device 40 through the coordinates in the display device space coordinate system and the plane coordinates of the display device.
Optionally, referring to fig. 3, in an implementation, the image capturing apparatus 10 is set on the display apparatus 40, the image capturing apparatus is turned on, and image capturing is performed at a certain frequency. And identifying the positions of the white eyes, the irises and the pupils of the left eye of the user according to the collected images, and identifying the positions of the white eyes, the irises and the pupils of the right eye of the user. And determining a first plane perpendicular to the display plane according to the position of the left eye pupil of the user and the position of the right eye pupil of the user. The first plane and the display plane intersect on a straight line. A point on the straight line is selected so that the distance from the point to the left and right pupils is equal. This point is the focus of the user's eye gaze.
The display area of the display device 40 may be divided into a plurality of display sections. When the eye gaze position of the user is located within the display area of the display device 40, a display area where the eye gaze focus of the user falls on the display area is further determined, and the display area identifier and the preset highlighting identifier form an adjustment instruction. The preset highlighting mark includes but is not limited to one or more of highlighting marks such as zooming, rotating, tilting, switching different sections, display layout change and the like. Namely, after the eyes of the user watch that the focus falls on the display block 1 on the display area and meets certain conditions, when specific rules such as the time that the focus stays in the area reaches a preset threshold value, the system automatically magnifies, rotates, tilts, switches different sections, changes the display layout or selects the mark display block 1.
Specifically, the step of generating the adjustment instruction includes:
step 1: judging whether the time that the user gazing position stays in the same display block is less than a preset threshold value or not, if so, not executing any operation, and otherwise, executing the step 2;
step 2: judging the coordinates of the display equipment 40 where the user gazing position is located, and obtaining the display block identification of the display block where the coordinates are located according to the display layout of the current display equipment 40;
and step 3: and (4) forming a primary adjusting instruction according to the display block identification obtained in the step (2) and the preset highlighted identification.
In one embodiment, the step of adjusting the display layout of the display area includes:
and 4, step 4: judging whether the display block identifier in the output adjusting instruction is in a display block identifier list of the display layout of the current fusion picture, if not, not executing any operation, otherwise, executing the step 5;
and 5: and adjusting the layout of the fusion picture according to the display block identifier and the preset highlighting identifier in the output adjusting instruction, so as to highlight the display of the display block.
When the user's eye gaze position does not fall within the display area, the display layout of the display area may be controlled to be unchanged. Alternatively, when the user eye gaze position does not fall within the display area, but falls within the functional block, the adjustment instruction may be a corresponding functional instruction. For example, when the user's eye gaze location falls on the screen power, and for a preset time, the screen power is turned on or a power off inquiry dialog box pops up.
According to the multi-mode image display control method, the eye gaze position of the user is obtained in real time, then the display area where the focus position is located is obtained according to the coordinates of the eye gaze position of the user in the whole display area, the mark data and the highlighting rule data of the display area form an adjusting instruction, and finally the display layout of the display area is adjusted according to the adjusting instruction. According to the multi-mode image display control method, the eye gaze position of the user is detected through eye movement identification, the position of the user gaze focus on the display device 40 is further obtained, then the display layout of the display area is adjusted according to the position of the user gaze focus on the display device 40, the content blocks observed by the gaze focus are intelligently highlighted, the user is enabled to be focused on the interested content blocks, the display of other content blocks is weakened, and the visual experience of the user is greatly improved.
The display device 40 may be a multi-content display device. The display area on the display device 40 may be divided into a plurality of display blocks for displaying the multi-content fusion picture. Specifically, the position coordinates of each display section on the display device 40 may be stored in advance. And after the eye fixation position of the user is obtained, comparing the obtained eye fixation position with the position coordinates of each display block to determine the display block corresponding to the eye fixation position of the user on the display area.
In one implementation, in order to prevent the display device 40 from misjudging or the user from misoperating, the multimodal image display control method based on the eye control technology further includes: when it is detected that the current viewpoint position is located in a specific region of the display device 40 or that the eye motion of the user is a specific eye motion, a cancel or return operation instruction is executed. Specifically, a specific region of the display device 40 is set in advance, or a specific eye motion is set in advance, and the specific region and/or the specific eye motion is/are made to correspond to the cancel/return operation instruction. If the display device 40 is judged by mistake or the user operates by mistake, the user can control the display device 40 to execute a cancel or return operation. That is, when it is detected that the viewpoint position of the user falls within the specific area, the previous operation is cancelled or the previous interface is returned; or when the user is detected to make a specific eye action, canceling the previous operation or returning to the previous interface.
Further, an eye control trigger area may be preset, when it is detected that the current viewpoint position of the user is in the eye control trigger area, a dialog box for determining whether to pop up is popped up, and if a determination instruction of the user is received, the eye control operation of the whole screen of the display device 40 is started. Therefore, when the user needs to use the eye control technology to operate the whole screen, the eye control operation is actively started; when the screen is not needed to be used, the screen is not opened, and misjudgment in the process of watching the screen is prevented. And when the effective fixation position or the effective eye movement made by the user cannot be detected within the preset time period, closing the eye control operation.
In addition, the eye control technology is also applied to medical treatment and special people, such as autism, and helps autism children to correct some perception differences, so that the symptoms are relieved; by measuring the eye jump movement and the pupil reaction, the technology is applied to the actual life to diagnose the symptoms of schizophrenia, Parkinson's disease, hyperactivity, cerebral concussion and the like, and the people who are gradually frozen and paralyzed can control the disabled through eyeballs to help the disabled to equally feel the convenient life brought by intelligence.
Referring to fig. 4, the application provides a multi-modal image display control system. The display control system includes an image pickup apparatus 10, a detection apparatus 20, a content display control apparatus 30, and a display apparatus 40.
The image pickup apparatus 10 is used to acquire a user face image. The detection device 20 is configured to obtain the eye gaze position of the user in real time according to the face image of the user. The content display control device 30 is connected to the detection device 20, and is configured to generate an adjustment instruction according to the eye gaze position of the user. The display device 40 is connected to the content display control device 30, and is configured to respond to the adjustment instruction to adjust the display layout of the display area of the display device 30.
Specifically, the medical diagnosis device to which the multi-modality image display control system of the present application is applied includes, but is not limited to, a digital subtraction angiography device, a CT device, a nuclear magnetic resonance device, or a DR device. The display device 40 in the medical diagnosis device includes, but is not limited to, a display, a projector, an LED screen, and other electronic products. The content displayed by the display device 40 includes, but is not limited to, X-ray images, CT images, and the like.
The image pickup apparatus 10 generally has a camera. The camera comprises a general camera software and hardware module. The camera can be added with infrared sensors and other sensors for positioning and tracking eyeballs and calculating the attention points of users besides intelligent interactive operations such as video chatting, portrait and gesture recognition, the users can conveniently operate products and the like by using the eyeballs, meanwhile, the background is used for collecting and analyzing big data, and the camera has great commercial and technical value significance. In one implementation, an eye control instrument may be disposed on the display device 40, and an infrared light source, an optical sensor, an image processor, and a viewpoint calculation core are built in the eye control instrument; therefore, a corresponding image is created and projected on human eyes, the image information of the head and the eyes of the user is captured, the characteristics of the captured image are extracted, and the eye fixation position of the user is accurately calculated. In this implementation, the above-described eye control apparatus is provided as the image pickup apparatus 10.
It is to be understood that the structure of the content display control device 30 is not particularly limited as long as an adjustment instruction can be generated according to the user eye gaze position. In an alternative embodiment, the content display control device 30 is a single chip or a microprocessor.
Referring to fig. 2, in one implementation, after the eye gaze position of the user is obtained, the content display control device 30 is used to determine whether the eye gaze position of the user is within the display area. When the eye gaze position of the user is in the display area, further determining a display area where the eye gaze focus of the user falls on the display area, and further generating an adjusting instruction. The adjusting instruction is a combination of a display block identifier and a highlight identifier.
It can be understood that the eyeball features can be captured in real time through images or extracted through real-time scanning, and the eye gaze position of the user is obtained through eyeball feature analysis. Specifically, the eyeball tracking technology is a scientific application technology, firstly, tracking is carried out according to characteristic changes of eyeballs and the peripheries of the eyeballs, secondly, tracking is carried out according to angle changes of irises, and thirdly, light beams such as infrared rays are actively projected to the irises to extract the characteristics. It is understood that the user gaze location is the focal point of the user's eye gaze.
Optionally, firstly, the image of the face of the user is acquired in real time through the camera equipment interface module 21, and then eyeball features are extracted; eyeball characteristics include the location of the white, iris and pupil; secondly, the user fixation position analysis module 22 converts the position information into coordinates in a space coordinate system of the display equipment according to the definition of the space coordinate system of the display equipment; and finally, calculating the position of the user watching position on the display device 40 through the coordinates in the display device space coordinate system and the plane coordinates of the display device.
Optionally, referring to fig. 3, in one implementation, the image capturing apparatus 10 is set on the display apparatus 40, the image capturing apparatus 10 is turned on, and image capturing is performed at a certain frequency. The user gaze position analysis module 22 identifies the positions of the white, iris and pupil of the left eye of the user according to the collected image, and identifies the positions of the white, iris and pupil of the right eye of the user. And determining a first plane perpendicular to the display plane according to the position of the left eye pupil of the user and the position of the right eye pupil of the user. The first plane and the display plane intersect on a straight line. One point on the straight line is selected so that the distances from the point to the left and right pupils are equal. This point is the focus of the user's eye gaze.
The display area of the display device 40 may be divided into a plurality of display sections. When the eye gaze position of the user is located in the display area of the display device 40, the visible range obtaining module 31 further determines a display area where the eye gaze focus of the user falls on the display area, and the display area identifier and the preset highlighting identifier form an adjustment instruction. The preset highlighting mark includes but is not limited to one or more of highlighting marks such as zooming, rotating, tilting, switching different sections, displaying layout changes and the like. That is, after the user focuses on the display area 1 with the focus on the display area and meets a certain condition, the adjusting module 41 automatically enlarges, rotates, tilts, switches different sections, changes the display layout or selects the marked display area 1 when the specific rule is that the focus stays in the area for a time reaching a preset threshold.
Specifically, the step of generating the adjustment instruction includes:
step 1: judging whether the time that the user gazing position stays in the same display block is less than a preset threshold value or not, if so, not executing any operation, and otherwise, executing the step 2;
step 2: judging the coordinate of the position where the user pays attention to on the display equipment 40, and obtaining the display block identification of the display block where the coordinate is located according to the display layout of the current display equipment 40;
and step 3: and (3) forming a primary adjusting instruction according to the display block identification obtained in the step (2) and the preset highlighting identification.
In one embodiment, the step of adjusting the display layout of the display area includes:
and 4, step 4: judging whether the display block identifier in the output adjusting instruction is in a display block identifier list of the display layout of the current fusion picture, if not, not executing any operation, otherwise, executing the step 5;
and 5: and adjusting the layout of the fusion picture according to the display block identifier and the preset highlighting identifier in the output adjusting instruction, so as to highlight the display of the display block.
When the user's eye gaze position does not fall within the display area, the display layout of the display area may be controlled to be unchanged. Alternatively, when the user eye gaze position does not fall within the display area, but falls within the functional block, the adjustment instruction may be a corresponding functional instruction. For example, when the user's eye gaze location falls on the screen power, and for a preset time, the screen power is turned on or a power off inquiry dialog box pops up.
The multimodal image display control system described above acquires the face image of the user in real time by the image pickup device 10, the detection device 20 acquires the eye gaze position of the user from the real-time face image, then the content display control device 30 generates an adjustment instruction according to the eye gaze position of the user, and finally the display device 40 adjusts the display layout of the display area according to the adjustment instruction. The display control system detects the eye gaze position of the user through eye movement identification, and then obtains the position of the user gaze focus on the display equipment 40, and then adjusts the display layout of the display area according to the position of the user gaze focus on the display equipment 40, so as to intelligently highlight the content blocks observed by the gaze focus, so that the user is concentrated in the interested content blocks, the display of other content blocks is weakened, and the visual experience of the user is greatly improved.
The technical features of the embodiments described above may be arbitrarily combined, and for the sake of brevity, all possible combinations of the technical features in the embodiments described above are not described, but should be considered as being within the scope of the present specification as long as there is no contradiction between the combinations of the technical features.
The above-mentioned embodiments only express several embodiments of the present application, and the description thereof is more specific and detailed, but not construed as limiting the claims. It should be noted that, for a person skilled in the art, several variations and modifications can be made without departing from the concept of the present application, which falls within the scope of protection of the present application. Therefore, the protection scope of the present patent shall be subject to the appended claims.
Claims (10)
1. A multi-modal image display control method, characterized by comprising:
acquiring the eye gaze position of a user in real time;
generating an adjusting instruction according to the eye gaze position of the user;
and adjusting the display layout of the display according to the adjusting instruction.
2. The multimodal image display control method according to claim 1, wherein the step of acquiring the user's eye gaze position in real time comprises:
and capturing eyeball characteristics in real time through images or scanning and extracting the eyeball characteristics in real time, and analyzing according to the eyeball characteristics to obtain the eye fixation position of the user.
3. The multimodal image display control method of claim 2, wherein the step of generating an adjustment instruction according to the user eye gaze position comprises: and determining a display area watched by the eyes of the user on a display area according to the eye watching position of the user, and generating the adjusting instruction.
4. The multimodal image display control method of claim 3, wherein the step of adjusting the display layout of the display device region comprises:
and controlling the display block to be highlighted.
5. The multi-modal image display control method of claim 4, wherein the manner of highlighting comprises one or more of zooming in, rotating, tilting, switching different sections, display layout changes, or highlighting indicia.
6. A multi-modal image display control system, comprising:
the camera shooting device is used for acquiring a face image of a user;
the detection equipment is used for acquiring the eye fixation position of the user in real time according to the face image of the user; the content display control equipment is connected with the detection equipment and used for generating an adjusting instruction according to the eye gaze position of the user; and
and the display equipment is connected with the content display control equipment and is used for responding to the adjusting instruction so as to adjust the display layout of the display area of the display equipment.
7. The multi-modal image display control system of claim 6, wherein the detection device comprises:
the camera equipment interface module is used for acquiring the user face image shot by the camera equipment; and
the user fixation position analysis module is connected with the camera equipment interface module and receives the user face image transmitted by the camera equipment interface module and is used for processing the user image so as to obtain the position of the user eye fixation focus on the display equipment.
8. The multimodal image display control system of claim 7, wherein the content display control apparatus comprises:
and the visual range acquisition module is connected with the user watching position analysis module and used for determining a display block watched by the user eyes on the display area according to the position of the user eye watching focus on the display equipment and generating the adjusting instruction.
9. The display control system according to claim 8, wherein the display device comprises:
and the adjusting module is connected with the visual range acquiring module and is used for controlling the display area block to be highlighted.
10. A computer device comprising a memory and a processor, the memory storing a computer program, characterized in that the processor implements the steps of the multimodal image display control method of any of claims 1 to 5 when executing the computer program.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011539861.9A CN114740966A (en) | 2020-12-23 | 2020-12-23 | Multi-modal image display control method and system and computer equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011539861.9A CN114740966A (en) | 2020-12-23 | 2020-12-23 | Multi-modal image display control method and system and computer equipment |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN114740966A true CN114740966A (en) | 2022-07-12 |
Family
ID=82273778
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011539861.9A Pending CN114740966A (en) | 2020-12-23 | 2020-12-23 | Multi-modal image display control method and system and computer equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114740966A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116382549A (en) * | 2023-05-22 | 2023-07-04 | 昆山嘉提信息科技有限公司 | Image processing method and device based on visual feedback |
-
2020
- 2020-12-23 CN CN202011539861.9A patent/CN114740966A/en active Pending
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116382549A (en) * | 2023-05-22 | 2023-07-04 | 昆山嘉提信息科技有限公司 | Image processing method and device based on visual feedback |
| CN116382549B (en) * | 2023-05-22 | 2023-09-01 | 昆山嘉提信息科技有限公司 | Image processing method and device based on visual feedback |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108427503B (en) | Human eye tracking method and human eye tracking device | |
| US10182720B2 (en) | System and method for interacting with and analyzing media on a display using eye gaze tracking | |
| EP3453316B1 (en) | Eye tracking using eyeball center position | |
| JP3673834B2 (en) | Gaze input communication method using eye movement | |
| US20120133754A1 (en) | Gaze tracking system and method for controlling internet protocol tv at a distance | |
| KR102092931B1 (en) | Method for eye-tracking and user terminal for executing the same | |
| US10456072B2 (en) | Image interpretation support apparatus and method | |
| US20170255260A1 (en) | Information processing apparatus, information processing method, and program | |
| JP6123694B2 (en) | Information processing apparatus, information processing method, and program | |
| JP2016515242A (en) | Method and apparatus for gazing point estimation without calibration | |
| JP2016512765A (en) | On-axis gaze tracking system and method | |
| JPH11175246A (en) | Sight line detector and method therefor | |
| JP2011059528A (en) | Display device and control method | |
| KR20130107981A (en) | Device and method for tracking sight line | |
| JP2006293786A (en) | Market research apparatus having visual line input unit | |
| Cho et al. | Gaze Detection by Wearable Eye‐Tracking and NIR LED‐Based Head‐Tracking Device Based on SVR | |
| JP2010123019A (en) | Device and method for recognizing motion | |
| Lander et al. | hEYEbrid: A hybrid approach for mobile calibration-free gaze estimation | |
| Rantanen et al. | The effect of clicking by smiling on the accuracy of head-mounted gaze tracking | |
| Lee et al. | Auto-focusing method for remote gaze tracking camera | |
| CN114740966A (en) | Multi-modal image display control method and system and computer equipment | |
| CN111459285B (en) | Display device control method based on eye control technology, display device and storage medium | |
| Heo et al. | Object recognition and selection method by gaze tracking and SURF algorithm | |
| CN112651270A (en) | Gaze information determination method and apparatus, terminal device and display object | |
| EP3922166B1 (en) | Display device, display method and display program |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |