CN102081503A - Electronic reader capable of automatically turning pages based on eye tracking and method thereof - Google Patents
Electronic reader capable of automatically turning pages based on eye tracking and method thereof Download PDFInfo
- Publication number
- CN102081503A CN102081503A CN 201110026978 CN201110026978A CN102081503A CN 102081503 A CN102081503 A CN 102081503A CN 201110026978 CN201110026978 CN 201110026978 CN 201110026978 A CN201110026978 A CN 201110026978A CN 102081503 A CN102081503 A CN 102081503A
- Authority
- CN
- China
- Prior art keywords
- user
- sight line
- display screen
- electronic reader
- image information
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Landscapes
- User Interface Of Digital Computer (AREA)
- Position Input By Displaying (AREA)
Abstract
The invention discloses an electronic reader capable of automatically turning pages based on eye tracking, which belongs to the technical field of electronic information. The electronic reader comprises a display, an image acquisition unit and a microprocessor. The display comprises a display unit and a display screen. The display screen is divided into two areas. The image information acquisition unit is used for acquiring the facial image information of a user. The microprocessor comprises a sight positioning and detecting unit and an information processing unit. The sight positioning and detecting unit is used for analyzing and processing the facial image information of the user and determining the display area where the sight of the user is. The information processing unit is used for comparing the areas where the current and previous sights of the user are. If the current sight area of the user is different from the previous one, the processor module refreshes the displayed content in the display area where the previous sight of the user is. The invention realizes to pre-refresh and turn pages in advance and accurately represent the content for the user, thereby avoiding the manual operation, saving the time and implementing the browsing continuity of the user.
Description
Technical field
The invention belongs to electronic information technical field, relate to a kind of electronic reader, relate in particular to and a kind ofly follow the trail of the electronic reader of automatic page turning and follow the trail of the method for automatic page turning based on sight line based on sight line.
Background technology
The electronic reader of Cun Zaiing mainly is two kinds of button or touch-screen types in the market, promptly carry out page turn over operation by the button that is arranged on the electronic reader, or utilize touch-screen directly on the electronic reader screen, to click, slide page, no matter be the button or the electronic reader of touch-screen type, all need the user to carry out manual operation according to the speed and the progress of reading, when user's both hands are occupied, use above-mentioned electronic reader very inconvenient.
In addition, along with the development of electronic reader, the interactive mode of people and electronic reader is more and more, yet there is certain limitation in traditional human-computer interaction device such as button, touch-screen at aspects such as naturality of using and friendly.Therefore, need a kind ofly to meet the interpersonal communication custom, electronic reader good man-machine interaction, means capable of automatic page turning can be provided.
Summary of the invention
The objective of the invention is to, provide a kind of need not user's manual operation, follow the trail of the electronic reader of automatic page turning according to what moving of user's sight line carried out page turning automatically based on sight line.
Another purpose of the present invention is, provide a kind of easy and simple to handle, save reading time, strengthen and read the successional electronic reader of following the trail of automatic page turning based on sight line.
A further object of the present invention is, what a kind of electronic reader was provided follows the trail of the method for automatic page turning based on sight line, make easy and simple to handle, save reading time, strengthen and read continuity, man-machine interaction is strong.
For achieving the above object, the present invention adopts following technological means:
A kind of electronic reader of following the trail of automatic page turning based on sight line, comprise microprocessor and human-computer interaction module wherein, human-computer interaction module comprises display unit and sight line tracing unit, wherein, described display unit comprises a display screen, and described display screen comprises two viewing areas; Described sight line tracing unit comprises the image information acquisition module, is used to obtain the image information of user's face; Microprocessor comprises sight line detection and localization unit and information process unit, and sight line detection and localization unit is used for the image information of user's face that described image information acquisition module is got access to and carries out analyzing and processing, determines the viewing area at user's sight line place; Information process unit is used for according to the testing result of sight line detection and localization the unit relatively current sight line of user and the viewing area at sight line place last time, last time viewing area, sight line place was different if viewing area, user current sight line place is with the user, then information process unit last time refreshed content displayed in the viewing area, sight line place the user, and showed in described display screen.
Preferably, the user last time shown content of pages in viewing area, sight line place is refreshed following one page content into the shown content of pages in viewing area, user current sight line place, the described page comprises the page of electronic reader viewing area, and described one page content down comprises the content of the next page that the electronic reader viewing area shows.
Preferably, described image information acquisition module is a camera, and it is arranged at the homonymy of display screen.
Preferably, described display screen comprises touch-screen, is used to receive the touch input of user to electronic reader.
Preferably, this electronic reader also comprises peripheral interface module, and peripheral interface module is used to connect peripherals.
Preferably, described peripheral interface module comprises one or more in USB interface, SD interface and the network interface.
A kind of aforesaid electronic reader is based on the method for sight line tracking automatic page turning, and this method may further comprise the steps: gather user's facial image information; Determine the direction of user's sight line according to user's facial image information of gathering, and obtain user's sight line according to the direction of user's sight line and drop on coordinate on the display screen, drop on coordinate on the display screen, determine viewing area, user's sight line place according to user's sight line; User viewing area, current sight line place and user were last time compared viewing area, sight line place,, then refreshed last time viewing area, sight line place content displayed when the current sight line of user region is different from last time sight line region of user.
Preferably, last time viewing area, sight line place is refreshed following one page content into the shown content in viewing area, current sight line place.
Preferably, described image information acquisition module is gathered user's facial image information at interval with preset time.
Preferably, determine the coordinate of user's sight line on display screen according to user's facial image information of gathering, further may further comprise the steps: search subscriber facial zone in user's facial image information, after detecting user's facial zone, microprocessor calculates the distance of human eye to display screen according to the area of user's facial zone; According to the distance location human eye of human eye to display screen, obtain the coordinate of human eye in user's face-image in user's facial zone, wherein the lower left corner of user's face-image is user's face-image origin; Estimate the position in the pupil center of circle according to the coordinate of human eye in user's face-image; The pupil center of circle is accurately located in position by the pupil center of circle of estimating; Determine the iris boundary and the center of circle by the pupil center of circle; Perspective geometry by the iris boundary and the center of circle concerns the normal direction of the oval iris circle of reckoner requisition family direction of visual lines, obtains the angle between user's sight line and the parallel or vertical electronic reader plane; According to the angle between described user's sight line and the parallel or vertical electronic reader plane, obtain user's sight line and drop on coordinate on the display screen.
Preferably, this method adopts the Adaboost people's face detection algorithm search subscriber facial zone based on the class rectangular characteristic.
Preferably, this method adopts the method location human eye based on the face feature.
A kind of electronic reader based on sight line tracking automatic page turning comprises display unit, image acquisition units and microprocessor, and display unit comprises display module and two display screens; The image information acquisition module is used to obtain the image information of user's face; Microprocessor comprises sight line detection and localization unit and information process unit, and sight line detection and localization unit is used for the image information of user's face that described image information acquisition module is got access to and carries out analyzing and processing, determines the display screen at user's sight line place; Information process unit is used for current sight line of comparison user and sight line place display screen last time, if last time sight line place display screen was different with the user for the current sight line of user place display screen, information process unit last time refreshed content displayed in the display screen of sight line place the user.
Preferably, the user last time sight line place display screen be refreshed following one page content into the shown content of the current sight line of user place display screen.
Preferably, described two display location are in the homonymy of electronic reader.
Preferably, described image information acquisition module is a camera, and it is arranged at the homonymy of two display screens.
Preferably, described display screen comprises touch-screen, is used to receive the touch input of user to electronic reader.
Preferably, this electronic reader also comprises peripheral interface module, and peripheral interface module is used to connect peripherals.
Preferably, described peripheral interface module comprises one or more in USB interface, SD interface and the network interface.
A kind of aforesaid electronic reader is followed the trail of the method for automatic page turning based on sight line, this method comprises microprocessor and human-computer interaction module, and wherein, human-computer interaction module comprises display unit and sight line tracing unit, wherein, described display unit comprises two display screens; Described sight line tracing unit comprises the image information acquisition module, is used to obtain the image information of user's face; Microprocessor comprises sight line detection and localization unit and information process unit, and sight line detection and localization unit is used for the image information of user's face that described image information acquisition module is got access to and carries out analyzing and processing, determines the display screen at user's sight line place; Information process unit is used for according to the testing result of sight line detection and localization the unit relatively current sight line of user and the display screen at sight line place last time, if last time sight line place display screen was different with the user for the current sight line of user place display screen, then information process unit last time refreshed content displayed in the display screen of sight line place the user.
Preferably, the user last time sight line place display screen be refreshed following one page content into the shown content of the current sight line of user place display screen.
Preferably, the image information acquisition module is gathered user's facial image information at interval with preset time.
Preferably, determine the coordinate of user's sight line on display screen according to user's facial image information of gathering, further may further comprise the steps: search subscriber facial zone in user's facial image information, after detecting user's facial zone, microprocessor calculates the distance of human eye to display screen according to the area of user's facial zone; According to the distance location human eye of human eye to display screen, obtain the coordinate of human eye in user's face-image in user's facial zone, wherein the lower left corner of user's face-image is user's face-image origin; Estimate the position in the pupil center of circle according to the coordinate of human eye in user's face-image; The pupil center of circle is accurately located in position by the pupil center of circle of estimating; Determine the iris boundary and the center of circle by the pupil center of circle; Perspective geometry by the iris boundary and the center of circle concerns the normal direction of the oval iris circle of reckoner requisition family direction of visual lines, obtains the angle between user's sight line and the parallel or vertical electronic reader plane; According to the angle between described user's sight line and the parallel or vertical electronic reader plane, obtain user's sight line and drop on coordinate on the display screen, wherein, two display screens adopt the same coordinate systems.
Preferably, this method adopts the Adaboost people's face detection algorithm search subscriber facial zone based on the class rectangular characteristic.
Preferably, this method adopts the method location human eye based on the face feature.
Beneficial effect of the present invention is: the present invention is divided into two parts with display screen, e-book is shown, variation according to user's sight line, thereby control display screen two parts page turning respectively, refresh in advance page turning in advance, for the user presents the content that will see, have substantivity, naturality and terseness exactly.The present invention is based on the electronic reader with automatic page turning function that sight line is followed the trail of, both removed the operation of manual button or touch screen page turning from, simple operation, man-machine interaction are strong, have saved reading time simultaneously, have strengthened the continuity that the user browses.
Description of drawings
Fig. 1 is the structural representation of electronic reader of the present invention.
The synoptic diagram that the electronic reader display screen split screen that Fig. 2 provides for first embodiment of the invention shows.
Fig. 3 is the concrete processing procedure of the sight line location of electronic reader of the present invention.
Fig. 3 a is the sight line location schematic top plan view of electronic reader of the present invention.
Fig. 3 b is the sight line location schematic side view of electronic reader of the present invention.
Fig. 4 is the flow chart of steps that the present invention is based on the electronic reader automatic page turning method of sight line tracking.
Fig. 5 is the workflow diagram of the electronic reader automatic page turning method of first embodiment of the invention.
The synoptic diagram that the electronic reader display screen split screen that Fig. 6 provides for second embodiment of the invention shows.
Embodiment
Below in conjunction with drawings and Examples summary of the invention of the present invention is further described.
Fig. 1 illustrates the structural representation of electronic reader of the present invention.Electronic reader based on sight line tracking automatic page turning of the present invention mainly comprises: microprocessor 1, peripheral interface module 2, human-computer interaction module 3, memory module 4 and power management module 5.Wherein, human-computer interaction module 3 comprises: display unit 31, touch screen unit 32 and sight line tracing unit 33.Sight line tracing unit 33 comprises image information acquisition module 331 and image information receiver module 332.Microprocessor 1 comprises sight line detection and localization unit 11 and information process unit 12.
Particularly, microprocessor 1 is the core of electronic reader, to order with data are analyzed, judgement, computing and control each module co-ordination of electronic reader.Microprocessor 1 is used for that steering order sends, the scheduling of data stream and peripheral equipment management etc.Wherein, steering order mainly comprises memory reference instruction, data processing instructions, coprocessor instruction, status register transfer instruction, exception handling instruction etc.; The scheduling of data stream mainly be to data message reception, write and read scheduling; Peripheral hardware comprises equipment such as USB interface, SD interface, touch-screen, display screen, camera, loudspeaker.Sight line detection and localization unit 11 is used for the image information analyzing and processing to described user's face, determines the viewing area at user's sight line place; Information process unit 12 is used for current sight line of comparison user and sight line region last time, if last time the sight line region was different with the user in the current sight line of user region, information process unit 12 control display units 31 are in user's following one page content of the current sight line of viewing area, sight line place explicit user region displaying contents last time.
Human-computer interaction module 3 is used to realize that user's input operation processing and output show, comprises display unit 31, touch screen unit 32 and sight line tracing unit 33.Touch screen unit 32 be used to realize the user by touch-screen to the electronic reader setting, open operations such as closing application program; Display unit 31 is used to realize that display screen divides two parts output to show the content of e-book, browses for the user; Sight line tracing unit 33 is by gathering the sight line change in information of human eye, offer microprocessor 1, thereby microprocessor 1 control display unit 31 realizes that two parts of display screen refresh displaying contents respectively, that is, the user last time the shown content of pages in viewing area, sight line place be refreshed following one page content into viewing area, user current sight line place displayed content.Refresh displaying contents for example to the displaying contents page turning, be meant that one of display screen zone shows the content of the next page of another regional current page, or display screen shows the content of the next page of another display screen current page size.
The page of the present invention is meant the page of electronic reader viewing area, and following one page content is meant the content of the next page that the electronic reader viewing area shows.
Sight line tracing unit 33 in the human-computer interaction module 3 comprises image information acquisition module 331 and image information receiver module 332.Image information acquisition module 331, for example be camera 6, be positioned over the homonymy of electronic reader display screen, for example the display screen top, the user's who is used to obtain facial image information, and user's facial image information of obtaining sent to image information receiver module 332; Image information receiver module 332 is used to receive the captured user's of camera facial image information and store.
Employing is based on Adaboost people's face detection algorithm search subscriber facial zone in face-image of class rectangular characteristic; Employing is located human eye based on the method for face feature; In eye image, estimate the position in the pupil center of circle; Accurately locate the pupil center of circle by the pupil center of circle of estimating; Determine the iris boundary and the center of circle by the pupil center of circle; Calculate the normal direction of oval iris circle and be considered as direction of visual lines by basic perspective geometry relation.
The present invention is based on the electronic reader that sight line is followed the trail of with automatic page turning function, can be by display screen being divided into the embodiment that two parts show or left and right sides two parts show up and down, option perhaps is set in electronic reader, selects display mode voluntarily according to reading habit by the user.
With reference to shown in Figure 4, for the present invention is based on the flow chart of steps of the electronic reader automatic page turning method that sight line follows the trail of.
Step S101: user's facial image information is gathered.
The camera 6 that is positioned over the homonymy of electronic reader display screen obtains user's facial image information, and user's facial image information of obtaining is sent to microprocessor 1.
Step S102: the direction of determining user's sight line according to user's facial information of gathering, and drop on coordinate on the display screen according to direction calculating user's sight line of user's sight line, drop on coordinate on the display screen according to user's sight line, determine the zone of user's sight line place display screen.
User's facial image information that 1 pair of microprocessor receives is carried out analyzing and processing, and the appearance profile information of scanning people face positions human eye, identifies the attitude of eyeball, realizes the location to user's sight line.
Concrete, step S102 further comprises the steps, shown in Fig. 3, Fig. 3 a and Fig. 3 b:
With the camera is that true origin is set up a three-dimensional coordinate system.
S1021: in image, search for human face region.Employing is based on Adaboost people's face detection algorithm search subscriber facial zone in the image of image information receiver module storage of class rectangular characteristic, after detecting user's facial zone, microprocessor 1 is according to the area of user's facial zone, calculate human eye to display screen apart from d.
S1022: location human eye.Adopt in user's facial zone and locate human eye based on the method for face feature, (Xi, Yi), wherein the lower left corner of user's face-image is the initial point of image coordinate can to obtain the coordinate of human eye in user's face-image.
S1023: the position of in eye image, estimating the pupil center of circle.
S1024: accurately locate the pupil center of circle by the pupil center of circle of estimating.
S1025: determine the iris boundary and the center of circle by the pupil center of circle.
S1026: the normal direction of calculating oval iris circle.Calculate the normal direction of oval iris circle and be considered as direction of visual lines by basic perspective geometry relation, obtain the angle between user's sight line and the parallel or vertical electronic reader plane.
α be direction of visual lines with the angle that with the camera is the vertical plane in the three-dimensional coordinate system of initial point, β be direction of visual lines with the angle that with the camera is the surface level in the three-dimensional coordinate system of initial point.
S1027:, obtain user's sight line and be projected in coordinate on the display screen according to the angle between user's sight line and the parallel or vertical electronic reader plane;
By with down conversion, can with the coordinate of human eye in user's face-image (Xi, Yi) be converted to the camera be in the three-dimensional coordinate system of initial point coordinate (X, Y).
X=g（d）(W/2-Xi)
Y=g（d）(L/2-Yi)
Wherein, be the coefficient relevant g(d) with distance, W is the width of user's face-image, L is the height of user's face-image.Further obtain people's an eye line and be projected in coordinate (X+d*tan(α) on the display screen, Y+d*tan(β)).
Step S103: user viewing area, current sight line place and user were last time compared viewing area, sight line place, when the current sight line of user region is different from last time sight line region of user, show following one page content of viewing area, current sight line place displayed content in viewing area, sight line place last time.
Repeat above-mentioned steps S101-S103, the realization electronic reader is followed the trail of based on sight line and is carried out automatic page turning.
Image information acquisition module 331 is gathered user's facial image information at interval with preset time, electronic reader can set in advance the time interval of sight line location, or the user manually is provided with reading time at interval according to oneself reading habit, the frequency of control user's facial image acquisition and sight line location.
By the following examples the electronic reader that the present invention is based on sight line tracking automatic page turning is further described.
Embodiment one
With reference to shown in Figure 2, the synoptic diagram that the electronic reader display screen split screen that provides for first embodiment of the invention shows.The display screen top that the present invention is based on the electronic reader of sight line tracking automatic page turning is provided with camera 6, camera 6 sends to 332 storages of image information receiver module after obtaining user's facial image information, sends to microprocessor 1 by image information receiver module 332 again.Camera 6 belows are display screen, and in the present embodiment, being divided into display screen up and down, two zones show respectively.
In conjunction with shown in Figure 5, be the workflow diagram of first embodiment of the invention electronic reader simultaneously.When the user used electronic reader of the present invention to browse e-book, the image information of user's face that image information acquisition module 331 obtains sent to image information receiver module 332; Image information receiver module 332 reception cameras 6 captured image informations are stored and are transferred to microprocessor 1 and carry out Image Information Processing.Image information acquisition module 331 also can directly be transferred to microprocessor 1 with the image information of user's face of obtaining.User's face-image of the 11 pairs of receptions in sight line detection and localization unit of microprocessor 1 carries out analyzing and processing according to step and principle shown in Fig. 3, Fig. 3 a and Fig. 3 b, orient the current direction of visual lines of user, obtain the position coordinates of sight line on display screen, and then to judge the active user be in the first half content of browse displays screen or in the Lower Half content of browse displays screen.
When the first half content of finishing watching display screen as the user begins to see the Lower Half content of display screen, after the sight line that 11 of the sight line detection and localization unit of microprocessor 1 detect human eye moves down into the Lower Half of display screen by the first half of display screen, information process unit 12 control display units 31 show following one page of half area in the upper half area of display screen content promptly realizes page turning automatically.
When the user finishes watching the Lower Half content of display screen, when its sight line turned to the upper half area of display screen, at this moment the upper half area of display screen had been finished and has been refreshed page turning, the content of the first half that the user can the direct reading display screen; Simultaneously, 11 of the sight line detection and localization unit of microprocessor 1 detect the upper half area that moves on to display screen on the half area of sight line by display screen of human eye, information process unit 12 control display units 31 show following one page of upper half area in the half area of display screen content promptly realizes page turning automatically.
When the user finishes watching the first half content of display screen, when its sight line turned to the half area of display screen once more, the half area of display screen has also refreshed page turning to be finished, the content of the Lower Half that the user can the direct reading display screen; The first half of display screen begins to carry out page turn over operation simultaneously.So repeatedly, the realization electronic reader is followed the trail of based on sight line, carries out automatic page turning.
Judge user's sight line is positioned at which zone of display screen, the coordinate figure that in fact exactly user's sight line is fallen within the point on the display screen is compared with the coordinate figure of viewing area.For example with the bottom, the leftmost side of display screen as true origin, in the present embodiment, display screen is divided into two zones up and down, the display screen half area is the first area, the display screen upper half area is a second area, when user's sight line in the Y-axis coordinate figure of display screen Y-axis maximal value greater than the display screen first area, and during less than the Y-axis maximal value of display screen second area, expression user sight line is transferred to second area from the first area of display screen.In like manner, when user's sight line at the Y-axis coordinate figure of display screen during less than the Y-axis minimum value of display screen second area, expression user sight line is transferred to the first area from the second area of display screen.
The electronic reader display screen that the present invention is based on sight line tracking automatic page turning also can be set to two, two display screen upper and lower settings, top is first display screen, the bottom is second display screen, two display screens adopt the same coordinate system, with the bottom, the leftmost side of first display screen as true origin, relatively user's sight line falls within the coordinate figure of the point on the display screen and the coordinate figure of display screen, when user's sight line in the Y-axis coordinate figure of display screen Y-axis maximal value greater than first display screen, and during less than the Y-axis maximal value of second display screen, expression user sight line is transferred to second display screen from first display screen.In like manner, when user's sight line at the Y-axis coordinate figure of display screen during less than the second display screen Y-axis minimum value, expression user sight line is transferred to first display screen from second display screen.
Embodiment two
The synoptic diagram that the electronic reader display screen split screen that Fig. 6 provides for second embodiment of the invention shows.
In the present embodiment, camera 6 still is positioned over the homonymy of electronic reader display screen, is used to obtain user's image information.Two zones about display screen is divided into, about two zones can show reading content respectively, and carry out automatic page turning according to sight line mobile.The difference of present embodiment and embodiment one only is that display screen is a left and right sides subregion, and the functional structure and the principle of work of electronic reader are identical, no longer describes in detail.
In the present embodiment, judge user's sight line is positioned at which zone of display screen, also be bottom, the leftmost side with display screen as true origin, the coordinate figure that user's sight line is fallen within the point on the display screen is compared with the coordinate figure of display screen area.In the present embodiment, two zones about display screen is divided into, zone, display screen left side is the first area, display screen right-hand part territory is a second area, when user's sight line in the X-axis coordinate figure of display screen X-axis maximal value greater than the display screen first area, and during less than the X-axis maximal value of display screen second area, expression user sight line is transferred to second area from the first area of display screen.In like manner, when user's sight line at the X-axis coordinate figure of display screen during less than the X-axis minimum value of display screen second area, expression user sight line is transferred to the first area from the second area of display screen.
The electronic reader display screen that the present invention is based on sight line tracking automatic page turning also can be set to two, be provided with about two display screens, the left side is first display screen, right-hand part is second display screen, two display screens adopt the same coordinate system, with the bottom, the leftmost side of first display screen as true origin, relatively user's sight line falls within the coordinate figure of the point on the display screen and the coordinate figure of display screen, when user's sight line in the X-axis coordinate figure of display screen X-axis maximal value greater than first display screen, and during less than the X-axis maximal value of second display screen, expression user sight line is transferred to second display screen from first display screen.In like manner, when user's sight line at the X-axis coordinate figure of display screen during less than the second display screen X-axis minimum value, expression user sight line is transferred to first display screen from second display screen.
The present invention is based on the electronic reader with automatic page turning function of sight line tracking and the automatic page turning method that electronic reader is followed the trail of based on sight line, realized refreshing in advance in advance page turning, present the content that to see for the user exactly, both removed from and started the operation of button or touch screen page turning, save the time again, realized the continuity that the user browses.
Claims (10)
1. follow the trail of the electronic reader of automatic page turning based on sight line for one kind, it is characterized in that: comprise microprocessor and human-computer interaction module, wherein,
Human-computer interaction module comprises display unit and sight line tracing unit, and wherein, described display unit comprises a display screen, and described display screen comprises two viewing areas; Described sight line tracing unit comprises the image information acquisition module, is used to obtain the image information of user's face;
Microprocessor comprises sight line detection and localization unit and information process unit, and sight line detection and localization unit is used for the image information of user's face that described image information acquisition module is got access to and carries out analyzing and processing, determines the viewing area at user's sight line place; Information process unit is used for according to the testing result of sight line detection and localization the unit relatively current sight line of user and the viewing area at sight line place last time, last time viewing area, sight line place was different if viewing area, user current sight line place is with the user, then information process unit last time refreshed content displayed in the viewing area, sight line place the user, and was last time showing in the viewing area, sight line place.
2. the electronic reader based on sight line tracking automatic page turning according to claim 1, it is characterized in that: the image information acquisition module comprises camera, described camera is arranged at the homonymy of display screen.
3. the electronic reader of following the trail of automatic page turning based on sight line according to claim 1, it is characterized in that: the user last time shown content of pages in viewing area, sight line place is refreshed following one page content into the shown content of pages in viewing area, user current sight line place, the described page comprises the page of electronic reader viewing area, and described one page content down comprises the content of the next page that the electronic reader viewing area shows.
4. an electronic reader as claimed in claim 1 is followed the trail of the method for automatic page turning based on sight line, it is characterized in that this method may further comprise the steps:
Gather user's facial image information;
Determine the direction of user's sight line according to user's facial information of gathering, and obtain user's sight line according to the direction of user's sight line and drop on coordinate on the display screen, drop on coordinate on the display screen, determine viewing area, user's sight line place according to user's sight line;
User viewing area, current sight line place and user were last time compared viewing area, sight line place,, then refreshed last time viewing area, sight line place content displayed when the current sight line of user region is different from last time sight line region of user.
5. electronic reader according to claim 4 is characterized in that based on the method for sight line tracking automatic page turning: determine the coordinate of user's sight line on display screen according to user's facial image information of gathering, further may further comprise the steps:
Search subscriber facial zone in user's facial image information, after detecting user's facial zone, microprocessor calculates the distance of human eye to display screen according to the area of user's facial zone;
According to the distance location human eye of human eye to display screen, calculate the coordinate of human eye in user's face-image in user's facial zone, wherein the lower left corner of user's face-image is user's face-image origin;
Estimate the position in the pupil center of circle according to the coordinate of human eye in user's face-image;
The pupil center of circle is accurately located in position by the pupil center of circle of estimating;
Determine the iris boundary and the center of circle by the pupil center of circle;
Perspective geometry by the iris boundary and the center of circle concerns the normal direction of the oval iris circle of reckoner requisition family direction of visual lines, obtains the angle between user's sight line and the parallel or vertical electronic reader plane;
According to the angle between described user's sight line and the parallel or vertical electronic reader plane, obtain user's sight line and drop on coordinate on the display screen.
6. the electronic reader based on sight line tracking automatic page turning comprises microprocessor and human-computer interaction module, wherein,
Human-computer interaction module comprises display unit and sight line tracing unit, and wherein, described display unit comprises two display screens; Described sight line tracing unit comprises the image information acquisition module, is used to obtain the image information of user's face;
Microprocessor comprises sight line detection and localization unit and information process unit, and sight line detection and localization unit is used for the image information of user's face that described image information acquisition module is got access to and carries out analyzing and processing, determines the display screen at user's sight line place; Information process unit is used for according to the testing result of sight line detection and localization the unit relatively current sight line of user and the display screen at sight line place last time, if last time sight line place display screen was different with the user for the current sight line of user place display screen, then information process unit last time refreshed content displayed in the display screen of sight line place the user.
7. the electronic reader based on sight line tracking automatic page turning according to claim 6, it is characterized in that: described two display location are in the homonymy of electronic reader.
8. the electronic reader based on sight line tracking automatic page turning according to claim 7, it is characterized in that: the image information acquisition module is a camera, is arranged at the homonymy of two display screens.
9. an electronic reader as claimed in claim 6 is followed the trail of the method for automatic page turning based on sight line, it is characterized in that this method may further comprise the steps:
Gather user's facial image information;
Determine the direction of user's sight line according to user's facial image information of gathering, and obtain user's sight line according to the direction of user's sight line and drop on coordinate on the display screen, drop on coordinate on the display screen, determine the display screen at user's sight line place according to user's sight line;
With the current sight line of user place display screen and user last time sight line place display screen compare, when the current sight line of user place display screen is different from the user last time during the display screen of sight line place, refresh content displayed at sight line place display screen last time.
10. electronic reader according to claim 9 is characterized in that based on the method for sight line tracking automatic page turning: determine the coordinate of user's sight line on display screen according to user's facial image information of gathering, further may further comprise the steps:
Search subscriber facial zone in user's facial image information, after detecting user's facial zone, microprocessor calculates the distance of human eye to display screen according to the area of user's facial zone;
According to the distance location human eye of human eye to display screen, obtain the coordinate of human eye in user's face-image in user's facial zone, wherein the lower left corner of user's face-image is user's face-image origin;
Estimate the position in the pupil center of circle according to the coordinate of human eye in user's face-image;
The pupil center of circle is accurately located in position by the pupil center of circle of estimating;
Determine the iris boundary and the center of circle by the pupil center of circle;
Perspective geometry by the iris boundary and the center of circle concerns the normal direction of the oval iris circle of reckoner requisition family direction of visual lines, obtains the angle between user's sight line and the parallel or vertical electronic reader plane;
According to the angle between described user's sight line and the parallel or vertical electronic reader plane, obtain user's sight line and drop on coordinate on the display screen, wherein, two display screens adopt the same coordinate systems.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN 201110026978 CN102081503B (en) | 2011-01-25 | 2011-01-25 | Electronic reader capable of automatically turning pages based on eye tracking and method thereof |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN 201110026978 CN102081503B (en) | 2011-01-25 | 2011-01-25 | Electronic reader capable of automatically turning pages based on eye tracking and method thereof |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN102081503A true CN102081503A (en) | 2011-06-01 |
| CN102081503B CN102081503B (en) | 2013-06-26 |
Family
ID=44087487
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN 201110026978 Active CN102081503B (en) | 2011-01-25 | 2011-01-25 | Electronic reader capable of automatically turning pages based on eye tracking and method thereof |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN102081503B (en) |
Cited By (36)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102551385A (en) * | 2011-12-29 | 2012-07-11 | 广东工业大学 | Automatically-paging reading chair based on sight tracking and control device and control method of automatically-paging reading chair |
| CN102880290A (en) * | 2012-08-24 | 2013-01-16 | 中兴通讯股份有限公司 | Display control method, device and terminal |
| CN102917120A (en) * | 2012-09-20 | 2013-02-06 | 北京百纳威尔科技有限公司 | Mobile terminal and method for refreshing information displayed by mobile terminal |
| CN102999164A (en) * | 2012-11-30 | 2013-03-27 | 广东欧珀移动通信有限公司 | E-book page turning control method and intelligent terminal |
| CN103064520A (en) * | 2013-01-31 | 2013-04-24 | 东莞宇龙通信科技有限公司 | Mobile terminal and method for controlling page to roll by same |
| CN103197755A (en) * | 2012-01-04 | 2013-07-10 | 中国移动通信集团公司 | Page turning method, device and terminal |
| CN103257712A (en) * | 2013-05-30 | 2013-08-21 | 苏州福丰科技有限公司 | Reading marking terminal based on facial recognition |
| CN103699308A (en) * | 2013-12-24 | 2014-04-02 | 惠州Tcl移动通信有限公司 | Automatic paging method and device |
| CN103809737A (en) * | 2012-11-13 | 2014-05-21 | 华为技术有限公司 | Method and device for human-computer interaction |
| CN103841250A (en) * | 2012-11-22 | 2014-06-04 | 纳讯（青岛）通信有限公司 | Mobile phone |
| CN103853419A (en) * | 2012-12-04 | 2014-06-11 | 联想(北京)有限公司 | Method and electronic equipment for information processing |
| CN103870164A (en) * | 2012-12-17 | 2014-06-18 | 联想(北京)有限公司 | Processing method and electronic device |
| CN104090657A (en) * | 2014-06-27 | 2014-10-08 | 小米科技有限责任公司 | Method and device for controlling page turning |
| CN104089606A (en) * | 2014-06-30 | 2014-10-08 | 天津大学 | Free space eye tracking measurement method |
| CN104360787A (en) * | 2014-10-17 | 2015-02-18 | 联想(北京)有限公司 | Display method and electronic device |
| CN104464769A (en) * | 2013-09-18 | 2015-03-25 | 布克查克控股有限公司 | Playback system for synchronised soundtracks for electronic media content |
| CN104484044A (en) * | 2014-12-23 | 2015-04-01 | 上海斐讯数据通信技术有限公司 | Advertisement pushing method and advertisement pushing system |
| CN104536687A (en) * | 2014-12-22 | 2015-04-22 | 深圳市金立通信设备有限公司 | Terminal |
| CN104536663A (en) * | 2014-12-22 | 2015-04-22 | 深圳市金立通信设备有限公司 | Page turning method |
| CN104571508A (en) * | 2014-12-29 | 2015-04-29 | 北京元心科技有限公司 | Method for operating data displayed by mobile terminal |
| CN104765442A (en) * | 2014-01-08 | 2015-07-08 | 腾讯科技（深圳）有限公司 | Automatic browsing method and device |
| CN105046283A (en) * | 2015-08-31 | 2015-11-11 | 宇龙计算机通信科技(深圳)有限公司 | Terminal operation method and terminal operation device |
| CN105045395A (en) * | 2015-08-28 | 2015-11-11 | 京东方科技集团股份有限公司 | Display device and image display method |
| CN105320255A (en) * | 2014-07-30 | 2016-02-10 | 腾讯科技（深圳）有限公司 | Data loading method and device |
| CN105631280A (en) * | 2015-07-28 | 2016-06-01 | 宇龙计算机通信科技(深圳)有限公司 | Icon control method, icon control device and terminal |
| EP2976692A4 (en) * | 2013-03-21 | 2016-12-07 | Lg Electronics Inc | Display device detecting gaze location and method for controlling thereof |
| CN106325510A (en) * | 2016-08-19 | 2017-01-11 | 联想(北京)有限公司 | Information processing method and electronic equipment |
| CN106527693A (en) * | 2016-10-31 | 2017-03-22 | 维沃移动通信有限公司 | Application control method and mobile terminal |
| CN106598257A (en) * | 2016-12-23 | 2017-04-26 | 北京奇虎科技有限公司 | Mobile terminal-based reading control method and apparatus |
| US9672421B2 (en) | 2013-10-04 | 2017-06-06 | Utechzone Co., Ltd. | Method and apparatus for recording reading behavior |
| CN108510701A (en) * | 2018-03-20 | 2018-09-07 | 深圳市泰和安科技有限公司 | A kind of intelligent smog alarm method, apparatus and system |
| CN110673724A (en) * | 2019-09-16 | 2020-01-10 | Tcl移动通信科技（宁波）有限公司 | Interface switching method and device, storage medium and terminal |
| CN111077994A (en) * | 2019-06-09 | 2020-04-28 | 广东小天才科技有限公司 | Control method of display screen in click-to-read scene and click-to-read machine equipment |
| CN111124223A (en) * | 2019-12-11 | 2020-05-08 | 维沃移动通信有限公司 | Application interface switching method and electronic equipment |
| CN111190527A (en) * | 2019-12-18 | 2020-05-22 | 华为技术有限公司 | Screen display method and terminal equipment |
| CN114615394A (en) * | 2022-03-07 | 2022-06-10 | 云知声智能科技股份有限公司 | Word extraction method and device, electronic equipment and storage medium |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101291364A (en) * | 2008-05-30 | 2008-10-22 | 深圳华为通信技术有限公司 | Interaction method and device of mobile communication terminal, and mobile communication terminal thereof |
| CN101311882A (en) * | 2007-05-23 | 2008-11-26 | 华为技术有限公司 | Eye tracking human-machine interaction method and apparatus |
| CN101344816A (en) * | 2008-08-15 | 2009-01-14 | 华南理工大学 | Human-machine interaction method and device based on sight tracing and gesture discriminating |
| CN101377738A (en) * | 2007-08-27 | 2009-03-04 | 汉王科技股份有限公司 | Method for implementing page turning of electric document |
-
2011
- 2011-01-25 CN CN 201110026978 patent/CN102081503B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101311882A (en) * | 2007-05-23 | 2008-11-26 | 华为技术有限公司 | Eye tracking human-machine interaction method and apparatus |
| CN101377738A (en) * | 2007-08-27 | 2009-03-04 | 汉王科技股份有限公司 | Method for implementing page turning of electric document |
| CN101291364A (en) * | 2008-05-30 | 2008-10-22 | 深圳华为通信技术有限公司 | Interaction method and device of mobile communication terminal, and mobile communication terminal thereof |
| CN101344816A (en) * | 2008-08-15 | 2009-01-14 | 华南理工大学 | Human-machine interaction method and device based on sight tracing and gesture discriminating |
Cited By (49)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102551385A (en) * | 2011-12-29 | 2012-07-11 | 广东工业大学 | Automatically-paging reading chair based on sight tracking and control device and control method of automatically-paging reading chair |
| CN103197755A (en) * | 2012-01-04 | 2013-07-10 | 中国移动通信集团公司 | Page turning method, device and terminal |
| CN102880290A (en) * | 2012-08-24 | 2013-01-16 | 中兴通讯股份有限公司 | Display control method, device and terminal |
| US20150192990A1 (en) * | 2012-08-24 | 2015-07-09 | Zte Corporation | Display control method, apparatus, and terminal |
| WO2014029229A1 (en) * | 2012-08-24 | 2014-02-27 | 中兴通讯股份有限公司 | Display control method, apparatus, and terminal |
| CN102880290B (en) * | 2012-08-24 | 2016-06-22 | 中兴通讯股份有限公司 | A kind of display control method, device and terminal |
| CN102917120A (en) * | 2012-09-20 | 2013-02-06 | 北京百纳威尔科技有限公司 | Mobile terminal and method for refreshing information displayed by mobile terminal |
| CN103809737A (en) * | 2012-11-13 | 2014-05-21 | 华为技术有限公司 | Method and device for human-computer interaction |
| US9740281B2 (en) | 2012-11-13 | 2017-08-22 | Huawei Technologies Co., Ltd. | Human-machine interaction method and apparatus |
| CN103841250A (en) * | 2012-11-22 | 2014-06-04 | 纳讯（青岛）通信有限公司 | Mobile phone |
| CN102999164B (en) * | 2012-11-30 | 2016-08-03 | 广东欧珀移动通信有限公司 | A kind of e-book flipping-over control method and intelligent terminal |
| CN102999164A (en) * | 2012-11-30 | 2013-03-27 | 广东欧珀移动通信有限公司 | E-book page turning control method and intelligent terminal |
| CN103853419A (en) * | 2012-12-04 | 2014-06-11 | 联想(北京)有限公司 | Method and electronic equipment for information processing |
| CN103870164A (en) * | 2012-12-17 | 2014-06-18 | 联想(北京)有限公司 | Processing method and electronic device |
| CN103064520B (en) * | 2013-01-31 | 2016-03-09 | 东莞宇龙通信科技有限公司 | The method of mobile terminal and control page scroll thereof |
| CN103064520A (en) * | 2013-01-31 | 2013-04-24 | 东莞宇龙通信科技有限公司 | Mobile terminal and method for controlling page to roll by same |
| EP2976692A4 (en) * | 2013-03-21 | 2016-12-07 | Lg Electronics Inc | Display device detecting gaze location and method for controlling thereof |
| CN103257712A (en) * | 2013-05-30 | 2013-08-21 | 苏州福丰科技有限公司 | Reading marking terminal based on facial recognition |
| CN104464769A (en) * | 2013-09-18 | 2015-03-25 | 布克查克控股有限公司 | Playback system for synchronised soundtracks for electronic media content |
| US9672421B2 (en) | 2013-10-04 | 2017-06-06 | Utechzone Co., Ltd. | Method and apparatus for recording reading behavior |
| CN103699308A (en) * | 2013-12-24 | 2014-04-02 | 惠州Tcl移动通信有限公司 | Automatic paging method and device |
| CN104765442A (en) * | 2014-01-08 | 2015-07-08 | 腾讯科技（深圳）有限公司 | Automatic browsing method and device |
| CN104090657A (en) * | 2014-06-27 | 2014-10-08 | 小米科技有限责任公司 | Method and device for controlling page turning |
| CN104089606A (en) * | 2014-06-30 | 2014-10-08 | 天津大学 | Free space eye tracking measurement method |
| CN105320255A (en) * | 2014-07-30 | 2016-02-10 | 腾讯科技（深圳）有限公司 | Data loading method and device |
| CN105320255B (en) * | 2014-07-30 | 2019-11-19 | 腾讯科技（深圳）有限公司 | Data load method and device |
| CN104360787A (en) * | 2014-10-17 | 2015-02-18 | 联想(北京)有限公司 | Display method and electronic device |
| CN104536687B (en) * | 2014-12-22 | 2019-04-23 | 深圳市金立通信设备有限公司 | A kind of page turning terminal |
| CN104536663A (en) * | 2014-12-22 | 2015-04-22 | 深圳市金立通信设备有限公司 | Page turning method |
| CN104536687A (en) * | 2014-12-22 | 2015-04-22 | 深圳市金立通信设备有限公司 | Terminal |
| CN104484044B (en) * | 2014-12-23 | 2018-07-31 | 上海斐讯数据通信技术有限公司 | A kind of advertisement sending method and system |
| CN104484044A (en) * | 2014-12-23 | 2015-04-01 | 上海斐讯数据通信技术有限公司 | Advertisement pushing method and advertisement pushing system |
| CN104571508A (en) * | 2014-12-29 | 2015-04-29 | 北京元心科技有限公司 | Method for operating data displayed by mobile terminal |
| CN105631280B (en) * | 2015-07-28 | 2018-08-24 | 宇龙计算机通信科技(深圳)有限公司 | Icon control method, icon control device and terminal |
| CN105631280A (en) * | 2015-07-28 | 2016-06-01 | 宇龙计算机通信科技(深圳)有限公司 | Icon control method, icon control device and terminal |
| CN105045395A (en) * | 2015-08-28 | 2015-11-11 | 京东方科技集团股份有限公司 | Display device and image display method |
| CN105046283A (en) * | 2015-08-31 | 2015-11-11 | 宇龙计算机通信科技(深圳)有限公司 | Terminal operation method and terminal operation device |
| CN106325510A (en) * | 2016-08-19 | 2017-01-11 | 联想(北京)有限公司 | Information processing method and electronic equipment |
| CN106325510B (en) * | 2016-08-19 | 2019-09-24 | 联想(北京)有限公司 | Information processing method and electronic equipment |
| CN106527693A (en) * | 2016-10-31 | 2017-03-22 | 维沃移动通信有限公司 | Application control method and mobile terminal |
| CN106598257A (en) * | 2016-12-23 | 2017-04-26 | 北京奇虎科技有限公司 | Mobile terminal-based reading control method and apparatus |
| CN108510701B (en) * | 2018-03-20 | 2019-11-22 | 深圳市泰和安科技有限公司 | A kind of intelligent smog alarm method, apparatus and system |
| CN108510701A (en) * | 2018-03-20 | 2018-09-07 | 深圳市泰和安科技有限公司 | A kind of intelligent smog alarm method, apparatus and system |
| CN111077994A (en) * | 2019-06-09 | 2020-04-28 | 广东小天才科技有限公司 | Control method of display screen in click-to-read scene and click-to-read machine equipment |
| CN110673724A (en) * | 2019-09-16 | 2020-01-10 | Tcl移动通信科技（宁波）有限公司 | Interface switching method and device, storage medium and terminal |
| CN111124223A (en) * | 2019-12-11 | 2020-05-08 | 维沃移动通信有限公司 | Application interface switching method and electronic equipment |
| CN111190527A (en) * | 2019-12-18 | 2020-05-22 | 华为技术有限公司 | Screen display method and terminal equipment |
| WO2021121103A1 (en) * | 2019-12-18 | 2021-06-24 | 华为技术有限公司 | Screen display method, and terminal apparatus |
| CN114615394A (en) * | 2022-03-07 | 2022-06-10 | 云知声智能科技股份有限公司 | Word extraction method and device, electronic equipment and storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN102081503B (en) | 2013-06-26 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN102081503B (en) | Electronic reader capable of automatically turning pages based on eye tracking and method thereof | |
| US20210349543A1 (en) | Data and user interaction based on device proximity | |
| CN103576857B (en) | Stare the fast wake-up of tracking system | |
| CN109343700B (en) | Eye movement control calibration data acquisition method and device | |
| EP2879020B1 (en) | Display control method, apparatus, and terminal | |
| US10082863B2 (en) | Gaze-based automatic scrolling | |
| CN107077197B (en) | 3D visualization map | |
| US10665022B2 (en) | Augmented reality display system for overlaying apparel and fitness information | |
| CN110555358B (en) | Method and apparatus for detecting and identifying features in an AR/VR scene | |
| CN109375765B (en) | Eyeball tracking interaction method and device | |
| US20150338914A1 (en) | Gaze-assisted touchscreen inputs | |
| US9317746B2 (en) | Techniques for occlusion accomodation | |
| US11783499B2 (en) | Enabling automatic measurements | |
| CN105373785A (en) | Method and device of hand gesture recognition and detection on the basis of deep neural network | |
| US10664215B2 (en) | Controlling multiple shared displays in an electronic collaboration setting | |
| CN103838365A (en) | Penetrating head-wearing display system and interactive operation method | |
| CN103139463A (en) | Method, system and mobile device for augmenting reality | |
| CN102830797A (en) | Man-machine interaction method and system based on sight judgment | |
| US20180107440A1 (en) | Dynamic User Interface for Multiple Shared Displays in an Electronic Collaboration Setting | |
| CN109409319A (en) | A kind of pet image beautification method, device and its storage medium | |
| US20180107341A1 (en) | Volumetric Tracking for Orthogonal Displays in an Electronic Collaboration Setting | |
| CN102830800B (en) | Method and system for controlling digital signage by utilizing gesture recognition | |
| Pires et al. | Unwrapping the eye for visible-spectrum gaze tracking on wearable devices | |
| CN102799271A (en) | Method and system for identifying interactive commands based on human hand gestures | |
| CN103399629A (en) | Method and device for capturing gesture displaying coordinates |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C14 | Grant of patent or utility model | ||
| GR01 | Patent grant |