CN102081503B - Electronic reader capable of automatically turning pages based on eye tracking and method thereof - Google Patents
Electronic reader capable of automatically turning pages based on eye tracking and method thereof Download PDFInfo
- Publication number
- CN102081503B CN102081503B CN 201110026978 CN201110026978A CN102081503B CN 102081503 B CN102081503 B CN 102081503B CN 201110026978 CN201110026978 CN 201110026978 CN 201110026978 A CN201110026978 A CN 201110026978A CN 102081503 B CN102081503 B CN 102081503B
- Authority
- CN
- China
- Prior art keywords
- user
- sight line
- display screen
- electronic reader
- image information
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Abstract
The invention discloses an electronic reader capable of automatically turning pages based on eye tracking, which belongs to the technical field of electronic information. The electronic reader comprises a display, an image acquisition unit and a microprocessor. The display comprises a display unit and a display screen. The display screen is divided into two areas. The image information acquisition unit is used for acquiring the facial image information of a user. The microprocessor comprises a sight positioning and detecting unit and an information processing unit. The sight positioning and detecting unit is used for analyzing and processing the facial image information of the user and determining the display area where the sight of the user is. The information processing unit is used for comparing the areas where the current and previous sights of the user are. If the current sight area of the user is different from the previous one, the processor module refreshes the displayed content in the display area where the previous sight of the user is. The invention realizes to pre-refresh and turn pages in advance and accurately represent the content for the user, thereby avoiding the manual operation, saving the time and implementing the browsing continuity of the user.
Description
Technical field
The invention belongs to electronic information technical field, relate to a kind of electronic reader, relate in particular to and a kind ofly follow the trail of the electronic reader of automatic page turning and follow the trail of the method for automatic page turning based on sight line based on sight line.
Background technology
The electronic reader that exists in the market is mainly two kinds of button or touch-screen types, namely carry out page turn over operation by the button that is arranged on electronic reader, or utilize touch-screen directly to click on the electronic reader screen, slide page, no matter be button or the electronic reader of touch-screen type, all need the user to carry out manual operation according to speed and the progress of reading, when user's both hands are occupied, use above-mentioned electronic reader very inconvenient.
In addition, along with the development of electronic reader, the interactive mode of people and electronic reader is more and more, yet there is certain limitation in traditional human-computer interaction device such as button, touch-screen at aspects such as the naturality of using and friendly.Therefore, need a kind ofly to meet the interpersonal communication custom, good man-machine interaction, electronic reader means capable of automatic page turning can be provided.
Summary of the invention
The object of the invention is to, provide a kind of need not user's manual operation, follow the trail of the electronic reader of automatic page turning according to what page turning was carried out in the movement of user's sight line automatically based on sight line.
Another purpose of the present invention is, provide a kind of easy and simple to handle, save reading time, strengthen and read the successional electronic reader of following the trail of automatic page turning based on sight line.
A further object of the present invention is, what a kind of electronic reader was provided follows the trail of the method for automatic page turning based on sight line, make easy and simple to handle, save reading time, strengthen and read continuity, man-machine interaction is strong.
For achieving the above object, the present invention adopts following technological means:
A kind of electronic reader of following the trail of automatic page turning based on sight line, comprise microprocessor and human-computer interaction module wherein, human-computer interaction module comprises display unit and sight line tracing unit, wherein, described display unit comprises a display screen, and described display screen comprises two viewing areas; Described sight line tracing unit comprises the image information collecting module, is used for obtaining the image information of user's face; Microprocessor comprises sight line detection and localization unit and information process unit, and sight line detection and localization unit is used for the image information of user's face that described image information collecting module is got and carries out analyzing and processing, determines the viewing area at user's sight line place; Information process unit is used for according to the testing result of sight line detection and localization the unit relatively current sight line of user and the viewing area at sight line place last time, last time viewing area, sight line place was different if viewing area, user current sight line place is from the user, information process unit last time refreshed the content of demonstration in viewing area, sight line place the user, and showed in described display screen.
Preferably, the user last time shown content of pages in viewing area, sight line place is refreshed lower one page content into the shown content of pages in viewing area, user current sight line place, the described page comprises the page of electronic reader viewing area, and described lower one page content comprises the content of the next page of electronic reader viewing area demonstration.
Preferably, described image information collecting module is camera, and it is arranged at the homonymy of display screen.
Preferably, described display screen comprises touch-screen, is used for receiving the user to the touch input of electronic reader.
Preferably, this electronic reader also comprises peripheral interface module, and peripheral interface module is used for connecting peripherals.
Preferably, described peripheral interface module comprises one or more in USB interface, SD interface and network interface.
A kind of electronic reader as above is followed the trail of the method for automatic page turning based on sight line, the method comprises the following steps: gather user's facial image information; Determine the direction of user's sight line according to the user's facial image information that gathers, and obtain user's sight line according to the direction of user's sight line and drop on coordinate on display screen, drop on coordinate on display screen according to user's sight line, determine viewing area, user's sight line place; User viewing area, current sight line place and user were last time compared viewing area, sight line place, when the current sight line of user region is different from last time sight line region of user, refreshed the content that viewing area, sight line place last time shows.
Preferably, last time viewing area, sight line place is refreshed lower one page content into the shown content in viewing area, current sight line place.
Preferably, described image information collecting module is with predetermined time interval collection user facial image information.
Preferably, determine the coordinate of user's sight line on display screen according to the user's facial image information that gathers, further comprising the steps: search subscriber facial zone in user's facial image information, after user's facial zone being detected, microprocessor calculates human eye to the distance of display screen according to the area of user's facial zone; Locate human eye according to human eye to the distance of display screen in user's facial zone, obtain the coordinate of human eye in user's face-image, wherein the lower left corner of user's face-image is user's face-image origin; Coordinate according to human eye in user's face-image is estimated the position in the pupil center of circle; Accurately locate the pupil center of circle by the position in the pupil center of circle of estimating; By the pupil center of circle determining iris boundary and the center of circle; The normal direction that concerns the oval iris circle of computational representation user direction of visual lines by the perspective geometry in iris boundary and the center of circle obtains the angle between user's sight line and parallel or vertical electronic reader plane; According to the angle between described user's sight line and parallel or vertical electronic reader plane, obtain user's sight line and drop on coordinate on display screen.
Preferably, this method adopts the Adaboost people's face detection algorithm search subscriber facial zone based on the class rectangular characteristic.
Preferably, this method adopts based on the method for face feature and locates human eye.
A kind of electronic reader of following the trail of automatic page turning based on sight line comprises display unit, image acquisition units and microprocessor, and display unit comprises display module and two display screens; The image information collecting module is used for obtaining the image information of user's face; Microprocessor comprises sight line detection and localization unit and information process unit, and sight line detection and localization unit is used for the image information of user's face that described image information collecting module is got and carries out analyzing and processing, determines the display screen at user's sight line place; Information process unit is used for the relatively current sight line of user and last time sight line place display screen, if last time sight line place display screen was different from the user for the current sight line of user place display screen, information process unit last time refreshed the content of demonstration in the display screen of sight line place the user.
Preferably, the user last time sight line place display screen be refreshed lower one page content into the shown content of the current sight line of user place display screen.
Preferably, described two display location are in the homonymy of electronic reader.
Preferably, described image information collecting module is camera, and it is arranged at the homonymy of two display screens.
Preferably, described display screen comprises touch-screen, is used for receiving the user to the touch input of electronic reader.
Preferably, this electronic reader also comprises peripheral interface module, and peripheral interface module is used for connecting peripherals.
Preferably, described peripheral interface module comprises one or more in USB interface, SD interface and network interface.
A kind of electronic reader as above is followed the trail of the method for automatic page turning based on sight line, the method comprises microprocessor and human-computer interaction module, and wherein, human-computer interaction module comprises display unit and sight line tracing unit, wherein, described display unit comprises two display screens; Described sight line tracing unit comprises the image information collecting module, is used for obtaining the image information of user's face; Microprocessor comprises sight line detection and localization unit and information process unit, and sight line detection and localization unit is used for the image information of user's face that described image information collecting module is got and carries out analyzing and processing, determines the display screen at user's sight line place; Information process unit is used for according to the testing result of sight line detection and localization the unit relatively current sight line of user and the display screen at sight line place last time, if last time sight line place display screen was different from the user for the current sight line of user place display screen, information process unit last time refreshed the content of demonstration in the display screen of sight line place the user.
Preferably, the user last time sight line place display screen be refreshed lower one page content into the shown content of the current sight line of user place display screen.
Preferably, the image information collecting module is with predetermined time interval collection user facial image information.
Preferably, determine the coordinate of user's sight line on display screen according to the user's facial image information that gathers, further comprising the steps: search subscriber facial zone in user's facial image information, after user's facial zone being detected, microprocessor calculates human eye to the distance of display screen according to the area of user's facial zone; Locate human eye according to human eye to the distance of display screen in user's facial zone, obtain the coordinate of human eye in user's face-image, wherein the lower left corner of user's face-image is user's face-image origin; Coordinate according to human eye in user's face-image is estimated the position in the pupil center of circle; Accurately locate the pupil center of circle by the position in the pupil center of circle of estimating; By the pupil center of circle determining iris boundary and the center of circle; The normal direction that concerns the oval iris circle of computational representation user direction of visual lines by the perspective geometry in iris boundary and the center of circle obtains the angle between user's sight line and parallel or vertical electronic reader plane; According to the angle between described user's sight line and parallel or vertical electronic reader plane, obtain user's sight line and drop on coordinate on display screen, wherein, two display screens adopt the same coordinate systems.
Preferably, this method adopts the Adaboost people's face detection algorithm search subscriber facial zone based on the class rectangular characteristic.
Preferably, this method adopts based on the method for face feature and locates human eye.
Beneficial effect of the present invention is: the present invention is divided into two parts with display screen, e-book is shown, variation according to user's sight line, thereby control display screen two parts page turning respectively, refresh in advance in advance page turning, for the user presents the content that will see, have substantivity, naturality and terseness exactly.The present invention is based on the electronic reader with automatic page turning function that sight line is followed the trail of, both removed the operation of manual button or touch screen page turning from, simple operation, man-machine interaction are strong, have saved reading time simultaneously, have strengthened the continuity that the user browses.
Description of drawings
Fig. 1 is the structural representation of electronic reader of the present invention.
The schematic diagram that the electronic reader display screen split screen that Fig. 2 provides for first embodiment of the invention shows.
Fig. 3 is the concrete processing procedure that the sight line of electronic reader of the present invention is located.
Fig. 3 a is that the sight line of electronic reader of the present invention is located schematic top plan view.
Fig. 3 b is that the sight line of electronic reader of the present invention is located schematic side view.
Fig. 4 is the flow chart of steps that the present invention is based on the electronic reader automatic page turning method of sight line tracking.
Fig. 5 is the workflow diagram of the electronic reader automatic page turning method of first embodiment of the invention.
The schematic diagram that the electronic reader display screen split screen that Fig. 6 provides for second embodiment of the invention shows.
Embodiment
Below in conjunction with drawings and Examples, summary of the invention of the present invention is further described.
Fig. 1 illustrates the structural representation of electronic reader of the present invention.Electronic reader based on sight line tracking automatic page turning of the present invention mainly comprises: microprocessor 1, peripheral interface module 2, human-computer interaction module 3, memory module 4 and power management module 5.Wherein, human-computer interaction module 3 comprises: display unit 31, touch screen unit 32 and sight line tracing unit 33.Sight line tracing unit 33 comprises image information collecting module 331 and image information receiver module 332.Microprocessor 1 comprises sight line detection and localization unit 11 and information process unit 12.
Particularly, microprocessor 1 is the core of electronic reader, to order and data analysis, judgement, computing and control each module co-ordination of electronic reader.Microprocessor 1 is used for that steering order sends, the scheduling of data stream and peripheral equipment management etc.Wherein, steering order mainly comprises memory reference instruction, data processing instructions, coprocessor instruction, status register transfer instruction, exception handling instruction etc.; The scheduling of data stream be mainly to data message reception, write and read scheduling; Peripheral hardware comprises the equipment such as USB interface, SD interface, touch-screen, display screen, camera, loudspeaker.The viewing area at user's sight line place is determined for the image information analyzing and processing to described user's face in sight line detection and localization unit 11; Information process unit 12 is used for the relatively current sight line of user and last time sight line region, if last time the sight line region was different from the user in the current sight line of user region, information process unit 12 control display unit 31 last time showed lower one page content of the current sight line of user region displaying contents in viewing area, sight line place the user.
Human-computer interaction module 3 is used for realizing user's input operation processing and output display, comprises display unit 31, touch screen unit 32 and sight line tracing unit 33.Touch screen unit 32 be used for realizing the user by touch-screen to the electronic reader setting, open operations such as closing application program; Display unit 31 is used for realizing that display screen divides the content of two parts output display e-book, browses for the user; Sight line tracing unit 33 is by the variation of the sight line information of collection human eye, offer microprocessor 1, thereby microprocessor 1 control display unit 31 realizes that two parts of display screen refresh respectively displaying contents, that is, the user last time the shown content of pages in viewing area, sight line place be refreshed lower one page content into viewing area, user current sight line place displayed content.Refresh displaying contents for example to the displaying contents page turning, refer to that one of display screen zone shows the content of the next page of another regional current page, or display screen shows the content of the next page of another display screen current page size.
The page of the present invention refers to the page of electronic reader viewing area, and lower one page content refers to the content of the next page of electronic reader viewing area demonstration.
Sight line tracing unit 33 in human-computer interaction module 3 comprises image information collecting module 331 and image information receiver module 332.Image information collecting module 331, be for example camera 6, be positioned over the homonymy of electronic reader display screen, for example the display screen top, for the user's who obtains facial image information, and user's facial image information of obtaining is sent to image information receiver module 332; Image information receiver module 332 is used for receiving the captured user's of camera facial image information and storing.
Employing is based on Adaboost people's face detection algorithm search subscriber facial zone in face-image of class rectangular characteristic; Employing is located human eye based on the method for face feature; Estimate the position in the pupil center of circle in eye image; Accurately locate the pupil center of circle by the pupil center of circle of estimating; By the pupil center of circle determining iris boundary and the center of circle; Calculate the normal direction of oval iris circle and be considered as direction of visual lines by basic perspective geometry relation.
The present invention is based on the electronic reader with automatic page turning function that sight line is followed the trail of, can be by display screen being divided into the embodiment that up and down two parts show or left and right two parts show, option perhaps is set in electronic reader, selects voluntarily display mode by the user according to reading habit.
With reference to shown in Figure 4, for the present invention is based on the flow chart of steps of the electronic reader automatic page turning method that sight line follows the trail of.
Step S101: user's facial image information gathers.
The camera 6 that is positioned over the homonymy of electronic reader display screen obtains user's facial image information, and user's facial image information of obtaining is sent to microprocessor 1.
Step S102: the direction of determining user's sight line according to the user's facial information that gathers, and drop on coordinate on display screen according to direction calculating user's sight line of user's sight line, drop on coordinate on display screen according to user's sight line, determine the zone of user's sight line place display screen.
User's facial image information that 1 pair of microprocessor receives is carried out analyzing and processing, and the appearance profile information of scanning people face positions human eye, identifies the attitude of eyeball, realizes the location to user's sight line.
Concrete, step S102 further comprises the steps, as shown in Fig. 3, Fig. 3 a and Fig. 3 b:
Set up a three-dimensional coordinate system take camera as true origin.
S1021: search for human face region in image.Employing is based on Adaboost people's face detection algorithm search subscriber facial zone in the image of image information receiver module storage of class rectangular characteristic, after user's facial zone being detected, microprocessor 1 is according to the area of user's facial zone, calculate human eye to display screen apart from d.
S1022: location human eye.Adopt in user's facial zone and locate human eye based on the method for face feature, can obtain the coordinate (Xi, Yi) of human eye in user's face-image, wherein the lower left corner of user's face-image is the initial point of image coordinate.
S1023: the position of estimating the pupil center of circle in eye image.
S1024: accurately locate the pupil center of circle by the pupil center of circle of estimating.
S1025: by the pupil center of circle determining iris boundary and the center of circle.
S1026: the normal direction of calculating oval iris circle.Calculate the normal direction of oval iris circle and be considered as direction of visual lines by basic perspective geometry relation, obtaining the angle between user's sight line and parallel or vertical electronic reader plane.
α is the angle of the vertical plane in direction of visual lines and three-dimensional coordinate system take camera as initial point, and β is the angle of the surface level in direction of visual lines and three-dimensional coordinate system take camera as initial point.
S1027: according to the angle between user's sight line and parallel or vertical electronic reader plane, obtain user's sight line and be projected in coordinate on display screen;
By with down conversion, can be with human eye the coordinate (Xi, Yi) in user's face-image be converted to coordinate (X, Y) in three-dimensional coordinate system take camera as initial point.
X=g（d）(W/2-Xi)
Y=g（d）(L/2-Yi)
Wherein, be and the coefficient of Range-based g(d), W is the width of user's face-image, and L is the height of user's face-image.Further obtain people's an eye line and be projected in coordinate (X+d*tan(α) on display screen, Y+d*tan(β)).
Step S103: user viewing area, current sight line place and user were last time compared viewing area, sight line place, when the current sight line of user region is different from last time sight line region of user, show lower one page content of viewing area, current sight line place displayed content in viewing area, sight line place last time.
Repeat above-mentioned steps S101-S103, realizing that electronic reader is followed the trail of based on sight line carries out automatic page turning.
Image information collecting module 331 is with predetermined time interval collection user facial image information, electronic reader can set in advance the time interval of sight line location, or the user manually arranges interval reading time according to oneself reading habit, controls the frequency of user's facial image acquisition and sight line location.
The electronic reader of following the trail of by the following examples automatic page turning to the present invention is based on sight line is further described.
Embodiment one
With reference to shown in Figure 2, the schematic diagram that the electronic reader display screen split screen that provides for first embodiment of the invention shows.The present invention is based on sight line follows the trail of the display screen top of the electronic reader of automatic page turning and is provided with camera 6, camera 6 sends to 332 storages of image information receiver module after obtaining user's facial image information, then sends to microprocessor 1 by image information receiver module 332.Camera 6 belows are display screen, in the present embodiment, display screen is divided into two zones, up and down shows respectively.
In conjunction with shown in Figure 5, be the workflow diagram of first embodiment of the invention electronic reader simultaneously.When the user used electronic reader of the present invention to browse e-book, the image information of user's face that image information collecting module 331 is obtained sent to image information receiver module 332; The image information receiver module 332 captured image informations of reception camera 6 are stored and are transferred to microprocessor 1 and carry out Image Information Processing.Image information collecting module 331 also can directly be transferred to microprocessor 1 with the image information of user's face of obtaining.User's face-image of the 11 pairs of receptions in sight line detection and localization unit of microprocessor 1 carries out analyzing and processing according to step and principle shown in Fig. 3, Fig. 3 a and Fig. 3 b, orient the current direction of visual lines of user, obtain the position coordinates of sight line on display screen, and then to judge the active user be in the first half content of browse displays screen or in the Lower Half content of browse displays screen.
When the first half content of finishing watching display screen as the user begins to see the Lower Half content of display screen, after the sight line that 11 of the sight line detection and localization unit of microprocessor 1 detect human eye moves down into the Lower Half of display screen by the first half of display screen, information process unit 12 control display unit 31 show the content of lower one page of half area in the upper half area of display screen, namely automatically realize page turning.
Finish watching the Lower Half content of display screen as the user, when its sight line turned to the upper half area of display screen, at this moment the upper half area of display screen had been completed and has been refreshed page turning, the content of the first half that the user can the direct reading display screen; Simultaneously, 11 of the sight line detection and localization unit of microprocessor 1 detect the upper half area that moves on to display screen on the half area of sight line by display screen of human eye, information process unit 12 control display unit 31 show the content of lower one page of upper half area in the half area of display screen, namely automatically realize page turning.
Finish watching the first half content of display screen as the user, when its sight line turned to the half area of display screen again, the half area of display screen has also refreshed page turning to be completed, the content of the Lower Half that the user can the direct reading display screen; The first half of display screen begins to carry out page turn over operation simultaneously.So repeatedly, realize that electronic reader follows the trail of based on sight line, carry out automatic page turning.
Judgement user sight line is positioned at which zone of display screen, and the coordinate figure that in fact exactly user's sight line is fallen within the point on display screen is compared with the coordinate figure of viewing area.For example with the bottom, the leftmost side of display screen as true origin, in the present embodiment, display screen is divided into two zones, up and down, the display screen half area is the first area, the display screen upper half area is second area, in the Y-axis coordinate figure of the display screen Y-axis maximal value greater than the display screen first area, and during less than the Y-axis maximal value of display screen second area, expression user sight line is transferred to second area from the first area of display screen when user's sight line.In like manner, at the Y-axis coordinate figure of display screen during less than the Y-axis minimum value of display screen second area, expression user sight line is transferred to the first area from the second area of display screen when user's sight line.
the electronic reader display screen that the present invention is based on sight line tracking automatic page turning also can be set to two, two display screens are setting up and down, top is the first display screen, the bottom is second display screen, two display screens adopt the same coordinate system, with the bottom, the leftmost side of the first display screen as true origin, relatively user's sight line falls within the coordinate figure of the point on display screen and the coordinate figure of display screen, when user's sight line in the Y-axis coordinate figure of the display screen Y-axis maximal value greater than the first display screen, and during less than the Y-axis maximal value of second display screen, expression user sight line is transferred to second display screen from the first display screen.In like manner, at the Y-axis coordinate figure of display screen during less than second display screen Y-axis minimum value, expression user sight line is transferred to the first display screen from second display screen when user's sight line.
Embodiment two
The schematic diagram that the electronic reader display screen split screen that Fig. 6 provides for second embodiment of the invention shows.
In the present embodiment, camera 6 still is positioned over the homonymy of electronic reader display screen, is used for obtaining user's image information.Display screen is divided into two zones, left and right, and two zones, left and right can show respectively reading content, and carry out automatic page turning according to the movement of sight line.The difference of the present embodiment and embodiment one only is that display screen is the left and right subregion, and functional structure and the principle of work of electronic reader are identical, no longer describes in detail.
In the present embodiment, judgement user sight line is positioned at which zone of display screen, be also bottom, the leftmost side with display screen as true origin, the coordinate figure that user's sight line is fallen within the point on display screen is compared with the coordinate figure of display screen area.In the present embodiment, display screen is divided into two zones, left and right, zone, display screen left side is the first area, display screen right-hand part territory is second area, when user's sight line in the X-axis coordinate figure of the display screen X-axis maximal value greater than the display screen first area, and during less than the X-axis maximal value of display screen second area, expression user sight line is transferred to second area from the first area of display screen.In like manner, at the X-axis coordinate figure of display screen during less than the X-axis minimum value of display screen second area, expression user sight line is transferred to the first area from the second area of display screen when user's sight line.
the electronic reader display screen that the present invention is based on sight line tracking automatic page turning also can be set to two, two display screen left and right settings, the left side is the first display screen, right-hand part is second display screen, two display screens adopt the same coordinate system, with the bottom, the leftmost side of the first display screen as true origin, relatively user's sight line falls within the coordinate figure of the point on display screen and the coordinate figure of display screen, when user's sight line in the X-axis coordinate figure of the display screen X-axis maximal value greater than the first display screen, and during less than the X-axis maximal value of second display screen, expression user sight line is transferred to second display screen from the first display screen.In like manner, at the X-axis coordinate figure of display screen during less than second display screen X-axis minimum value, expression user sight line is transferred to the first display screen from second display screen when user's sight line.
The present invention is based on the electronic reader with automatic page turning function of sight line tracking and the automatic page turning method that electronic reader is followed the trail of based on sight line, realized refreshing in advance in advance page turning, present for the user content that will see exactly, both removed from and started the operation of button or touch screen page turning, save the time again, realized the continuity that the user browses.
Claims (9)
1. electronic reader of following the trail of automatic page turning based on sight line is characterized in that: comprise microprocessor and human-computer interaction module, wherein,
Human-computer interaction module comprises display unit and sight line tracing unit, and wherein, described display unit comprises a display screen, and described display screen comprises two viewing areas; Described sight line tracing unit comprises the image information collecting module, is used for obtaining the image information of user's face;
microprocessor comprises sight line detection and localization unit and information process unit, and sight line detection and localization unit is used for the image information of user's face that described image information collecting module is got and carries out analyzing and processing, determines the viewing area at user's sight line place, information process unit is used for according to the testing result of sight line detection and localization the unit relatively current sight line of user and the viewing area at sight line place last time, last time viewing area, sight line place was different if viewing area, user current sight line place is from the user, information process unit last time refreshed the content of demonstration in viewing area, sight line place the user, and show lower one page content of the content of pages that viewing area, user current sight line place is shown in viewing area, sight line place last time, the described page comprises the page of electronic reader viewing area, described lower one page content comprises the content of the next page of electronic reader viewing area demonstration.
2. the electronic reader of following the trail of automatic page turning based on sight line according to claim 1, it is characterized in that: the image information collecting module comprises camera, described camera is arranged at the homonymy of display screen.
3. an electronic reader as claimed in claim 1 is followed the trail of the method for automatic page turning based on sight line, it is characterized in that, the method comprises the following steps:
Gather user's facial image information;
Determine the direction of user's sight line according to the user's facial information that gathers, and obtain user's sight line according to the direction of user's sight line and drop on coordinate on display screen, drop on coordinate on display screen according to user's sight line, determine viewing area, user's sight line place;
User viewing area, current sight line place and user were last time compared viewing area, sight line place, when the current sight line of user region is different from last time sight line region of user, refresh the content of last time viewing area, sight line place demonstration and be lower one page content of the shown content of pages in viewing area, user current sight line place, the described page comprises the page of electronic reader viewing area, and described lower one page content comprises the content of the next page of electronic reader viewing area demonstration.
4. electronic reader according to claim 3 is followed the trail of the method for automatic page turning based on sight line, it is characterized in that: determine the coordinate of user's sight line on display screen according to the user's facial image information that gathers, further comprising the steps:
Search subscriber facial zone in user's facial image information, after user's facial zone being detected, microprocessor calculates human eye to the distance of display screen according to the area of user's facial zone;
Locate human eye according to human eye to the distance of display screen in user's facial zone, calculate the coordinate of human eye in user's face-image, wherein the lower left corner of user's face-image is user's face-image origin;
Coordinate according to human eye in user's face-image is estimated the position in the pupil center of circle;
Accurately locate the pupil center of circle by the position in the pupil center of circle of estimating;
By the pupil center of circle determining iris boundary and the center of circle;
The normal direction that concerns the oval iris circle of computational representation user direction of visual lines by the perspective geometry in iris boundary and the center of circle obtains the angle between user's sight line and parallel or vertical electronic reader plane;
According to the angle between described user's sight line and parallel or vertical electronic reader plane, obtain user's sight line and drop on coordinate on display screen.
5. an electronic reader of following the trail of automatic page turning based on sight line, comprise microprocessor and human-computer interaction module, wherein,
Human-computer interaction module comprises display unit and sight line tracing unit, and wherein, described display unit comprises two display screens; Described sight line tracing unit comprises the image information collecting module, is used for obtaining the image information of user's face;
Microprocessor comprises sight line detection and localization unit and information process unit, and sight line detection and localization unit is used for the image information of user's face that described image information collecting module is got and carries out analyzing and processing, determines the display screen at user's sight line place; Information process unit is used for according to the testing result of sight line detection and localization the unit relatively current sight line of user and the display screen at sight line place last time, if last time sight line place display screen was different from the user for the current sight line of user place display screen, the information process unit content that last time refreshed demonstration in the display screen of sight line place the user is lower one page content of the shown content of pages of the current sight line of user place display screen, the described page comprises the page of electronic reader display screen, and described lower one page content comprises the content of the next page that the electronic reader display screen shows.
6. the electronic reader of following the trail of automatic page turning based on sight line according to claim 5, it is characterized in that: described two display location are in the homonymy of electronic reader.
7. the electronic reader of following the trail of automatic page turning based on sight line according to claim 6, it is characterized in that: the image information collecting module is camera, is arranged at the homonymy of two display screens.
8. an electronic reader as claimed in claim 5 is followed the trail of the method for automatic page turning based on sight line, it is characterized in that, the method comprises the following steps:
Gather user's facial image information;
Determine the direction of user's sight line according to the user's facial image information that gathers, and obtain user's sight line according to the direction of user's sight line and drop on coordinate on display screen, drop on coordinate on display screen according to user's sight line, determine the display screen at user's sight line place;
With the current sight line of user place display screen and user last time sight line place display screen compare, when the current sight line of user place display screen is different from the user last time during the display screen of sight line place, the content that refreshes demonstration at sight line place display screen last time is lower one page content of the shown content of pages of the current sight line of user place display screen, the described page comprises the page of electronic reader display screen, and described lower one page content comprises the content of the next page that the electronic reader display screen shows.
9. electronic reader according to claim 8 is followed the trail of the method for automatic page turning based on sight line, it is characterized in that: determine the coordinate of user's sight line on display screen according to the user's facial image information that gathers, further comprising the steps:
Search subscriber facial zone in user's facial image information, after user's facial zone being detected, microprocessor calculates human eye to the distance of display screen according to the area of user's facial zone;
Locate human eye according to human eye to the distance of display screen in user's facial zone, obtain the coordinate of human eye in user's face-image, wherein the lower left corner of user's face-image is user's face-image origin;
Coordinate according to human eye in user's face-image is estimated the position in the pupil center of circle;
Accurately locate the pupil center of circle by the position in the pupil center of circle of estimating;
By the pupil center of circle determining iris boundary and the center of circle;
The normal direction that concerns the oval iris circle of computational representation user direction of visual lines by the perspective geometry in iris boundary and the center of circle obtains the angle between user's sight line and parallel or vertical electronic reader plane;
According to the angle between described user's sight line and parallel or vertical electronic reader plane, obtain user's sight line and drop on coordinate on display screen, wherein, two display screens adopt the same coordinate systems.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN 201110026978 CN102081503B (en) | 2011-01-25 | 2011-01-25 | Electronic reader capable of automatically turning pages based on eye tracking and method thereof |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN 201110026978 CN102081503B (en) | 2011-01-25 | 2011-01-25 | Electronic reader capable of automatically turning pages based on eye tracking and method thereof |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN102081503A CN102081503A (en) | 2011-06-01 |
| CN102081503B true CN102081503B (en) | 2013-06-26 |
Family
ID=44087487
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN 201110026978 Active CN102081503B (en) | 2011-01-25 | 2011-01-25 | Electronic reader capable of automatically turning pages based on eye tracking and method thereof |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN102081503B (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| TWI550438B (en) * | 2013-10-04 | 2016-09-21 | 由田新技股份有限公司 | Method and apparatus for recording reading behavior |
Families Citing this family (35)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102551385B (en) * | 2011-12-29 | 2014-08-20 | 广东工业大学 | Automatically-paging reading chair based on sight tracking and control device and control method of automatically-paging reading chair |
| CN103197755A (en) * | 2012-01-04 | 2013-07-10 | 中国移动通信集团公司 | Page turning method, device and terminal |
| CN102880290B (en) * | 2012-08-24 | 2016-06-22 | 中兴通讯股份有限公司 | A kind of display control method, device and terminal |
| CN102917120A (en) * | 2012-09-20 | 2013-02-06 | 北京百纳威尔科技有限公司 | Mobile terminal and method for refreshing information displayed by mobile terminal |
| CN103809737A (en) * | 2012-11-13 | 2014-05-21 | 华为技术有限公司 | Method and device for human-computer interaction |
| CN103841250A (en) * | 2012-11-22 | 2014-06-04 | 纳讯（青岛）通信有限公司 | Mobile phone |
| CN102999164B (en) * | 2012-11-30 | 2016-08-03 | 广东欧珀移动通信有限公司 | A kind of e-book flipping-over control method and intelligent terminal |
| CN103853419B (en) * | 2012-12-04 | 2017-08-29 | 联想(北京)有限公司 | The method and electronic equipment of information processing |
| CN103870164A (en) * | 2012-12-17 | 2014-06-18 | 联想(北京)有限公司 | Processing method and electronic device |
| CN103064520B (en) * | 2013-01-31 | 2016-03-09 | 东莞宇龙通信科技有限公司 | The method of mobile terminal and control page scroll thereof |
| KR102081930B1 (en) * | 2013-03-21 | 2020-02-26 | 엘지전자 주식회사 | Display device detecting gaze location and method for controlling thereof |
| CN103257712A (en) * | 2013-05-30 | 2013-08-21 | 苏州福丰科技有限公司 | Reading marking terminal based on facial recognition |
| IN2014DE02666A (en) * | 2013-09-18 | 2015-06-26 | Booktrack Holdings Ltd | |
| CN103699308A (en) * | 2013-12-24 | 2014-04-02 | 惠州Tcl移动通信有限公司 | Automatic paging method and device |
| CN104765442B (en) * | 2014-01-08 | 2018-04-20 | 腾讯科技（深圳）有限公司 | Auto-browsing method and auto-browsing device |
| CN104090657A (en) * | 2014-06-27 | 2014-10-08 | 小米科技有限责任公司 | Method and device for controlling page turning |
| CN104089606B (en) * | 2014-06-30 | 2016-08-17 | 天津大学 | A kind of free space eye tracking measuring method |
| CN105320255B (en) * | 2014-07-30 | 2019-11-19 | 腾讯科技（深圳）有限公司 | Data load method and device |
| CN104360787A (en) * | 2014-10-17 | 2015-02-18 | 联想(北京)有限公司 | Display method and electronic device |
| CN104536663B (en) * | 2014-12-22 | 2019-05-17 | 深圳市金立通信设备有限公司 | A kind of page turning method |
| CN104536687B (en) * | 2014-12-22 | 2019-04-23 | 深圳市金立通信设备有限公司 | A kind of page turning terminal |
| CN104484044B (en) * | 2014-12-23 | 2018-07-31 | 上海斐讯数据通信技术有限公司 | A kind of advertisement sending method and system |
| CN104571508A (en) * | 2014-12-29 | 2015-04-29 | 北京元心科技有限公司 | Method for operating data displayed by mobile terminal |
| CN105631280B (en) * | 2015-07-28 | 2018-08-24 | 宇龙计算机通信科技(深圳)有限公司 | Icon control method, icon control device and terminal |
| CN105045395A (en) * | 2015-08-28 | 2015-11-11 | 京东方科技集团股份有限公司 | Display device and image display method |
| CN105046283A (en) * | 2015-08-31 | 2015-11-11 | 宇龙计算机通信科技(深圳)有限公司 | Terminal operation method and terminal operation device |
| CN106325510B (en) * | 2016-08-19 | 2019-09-24 | 联想(北京)有限公司 | Information processing method and electronic equipment |
| CN106527693A (en) * | 2016-10-31 | 2017-03-22 | 维沃移动通信有限公司 | Application control method and mobile terminal |
| CN106598257A (en) * | 2016-12-23 | 2017-04-26 | 北京奇虎科技有限公司 | Mobile terminal-based reading control method and apparatus |
| CN108510701B (en) * | 2018-03-20 | 2019-11-22 | 深圳市泰和安科技有限公司 | A kind of intelligent smog alarm method, apparatus and system |
| CN111077994A (en) * | 2019-06-09 | 2020-04-28 | 广东小天才科技有限公司 | Control method of display screen in click-to-read scene and click-to-read machine equipment |
| CN110673724A (en) * | 2019-09-16 | 2020-01-10 | Tcl移动通信科技（宁波）有限公司 | Interface switching method and device, storage medium and terminal |
| CN111124223A (en) * | 2019-12-11 | 2020-05-08 | 维沃移动通信有限公司 | Application interface switching method and electronic equipment |
| CN111190527A (en) * | 2019-12-18 | 2020-05-22 | 华为技术有限公司 | Screen display method and terminal equipment |
| CN114615394A (en) * | 2022-03-07 | 2022-06-10 | 云知声智能科技股份有限公司 | Word extraction method and device, electronic equipment and storage medium |
Family Cites Families (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101311882A (en) * | 2007-05-23 | 2008-11-26 | 华为技术有限公司 | Eye tracking human-machine interaction method and apparatus |
| CN101377738B (en) * | 2007-08-27 | 2011-06-22 | 汉王科技股份有限公司 | Method for implementing page turning of electric document |
| CN101291364B (en) * | 2008-05-30 | 2011-04-27 | 华为终端有限公司 | Interaction method and device of mobile communication terminal, and mobile communication terminal thereof |
| CN101344816B (en) * | 2008-08-15 | 2010-08-11 | 华南理工大学 | Human-machine interaction method and device based on sight tracing and gesture discriminating |
-
2011
- 2011-01-25 CN CN 201110026978 patent/CN102081503B/en active Active
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| TWI550438B (en) * | 2013-10-04 | 2016-09-21 | 由田新技股份有限公司 | Method and apparatus for recording reading behavior |
Also Published As
| Publication number | Publication date |
|---|---|
| CN102081503A (en) | 2011-06-01 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN102081503B (en) | Electronic reader capable of automatically turning pages based on eye tracking and method thereof | |
| CN109343700B (en) | Eye movement control calibration data acquisition method and device | |
| CN102799277B (en) | Wink action-based man-machine interaction method and system | |
| CN102830797B (en) | A kind of man-machine interaction method based on sight line judgement and system | |
| CN109375765B (en) | Eyeball tracking interaction method and device | |
| US10665022B2 (en) | Augmented reality display system for overlaying apparel and fitness information | |
| US9575559B2 (en) | Gaze-assisted touchscreen inputs | |
| EP2879020B1 (en) | Display control method, apparatus, and terminal | |
| CN102547123B (en) | Self-adapting sightline tracking system and method based on face recognition technology | |
| CN103164022B (en) | Many fingers touch method and device, portable terminal | |
| CN103970264B (en) | Gesture recognition and control method and device | |
| CN103139463A (en) | Method, system and mobile device for augmenting reality | |
| US9317746B2 (en) | Techniques for occlusion accomodation | |
| US11783499B2 (en) | Enabling automatic measurements | |
| CN104217350A (en) | Virtual try-on realization method and device | |
| US10366281B2 (en) | Gesture identification with natural images | |
| CN110727496B (en) | Layout method and device of graphical user interface, electronic equipment and storage medium | |
| CN109409319A (en) | A kind of pet image beautification method, device and its storage medium | |
| WO2012142869A1 (en) | Method and apparatus for automatically adjusting terminal interface display | |
| CN103902222A (en) | Touch screen slide adjusting method, touch screen slide adjusting device and touch device | |
| CN102830800B (en) | Method and system for controlling digital signage by utilizing gesture recognition | |
| Pires et al. | Unwrapping the eye for visible-spectrum gaze tracking on wearable devices | |
| CN102799271A (en) | Method and system for identifying interactive commands based on human hand gestures | |
| CN102508551B (en) | Sight capturing method and man-machine interaction method adopting sight capturing | |
| CN106570747A (en) | Glasses online adaption method and system combining hand gesture recognition |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C14 | Grant of patent or utility model | ||
| GR01 | Patent grant |