WO2015044830A1 - Methods and system for improving the readability of text displayed on an electronic device screen - Google Patents

Methods and system for improving the readability of text displayed on an electronic device screen Download PDF

Info

Publication number
WO2015044830A1
WO2015044830A1 PCT/IB2014/064521 IB2014064521W WO2015044830A1 WO 2015044830 A1 WO2015044830 A1 WO 2015044830A1 IB 2014064521 W IB2014064521 W IB 2014064521W WO 2015044830 A1 WO2015044830 A1 WO 2015044830A1
Authority
WO
WIPO (PCT)
Prior art keywords
text
user
device screen
screen
enhancing
Prior art date
Application number
PCT/IB2014/064521
Other languages
French (fr)
Inventor
Iris ELKABETZ
Tzur Di-Cori
Original Assignee
Visuality Imaging Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Visuality Imaging Ltd filed Critical Visuality Imaging Ltd
Publication of WO2015044830A1 publication Critical patent/WO2015044830A1/en

Links

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/048Interaction techniques based on graphical user interfaces [GUI]
    • G06F3/0487Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser
    • G06F3/0489Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser using dedicated keyboard keys or combinations thereof
    • G06F3/04897Special input arrangements or commands for improving display capability
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/048Interaction techniques based on graphical user interfaces [GUI]
    • G06F3/0481Interaction techniques based on graphical user interfaces [GUI] based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/048Interaction techniques based on graphical user interfaces [GUI]
    • G06F3/0484Interaction techniques based on graphical user interfaces [GUI] for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range
    • G06F3/0485Scrolling or panning
    • GPHYSICS
    • G09EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09BEDUCATIONAL OR DEMONSTRATION APPLIANCES; APPLIANCES FOR TEACHING, OR COMMUNICATING WITH, THE BLIND, DEAF OR MUTE; MODELS; PLANETARIA; GLOBES; MAPS; DIAGRAMS
    • G09B21/00Teaching, or communicating with, the blind, deaf or mute
    • G09B21/001Teaching or communicating with blind persons
    • G09B21/008Teaching or communicating with blind persons using visual presentation of the information for the partially sighted
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F2203/00Indexing scheme relating to G06F3/00 - G06F3/048
    • G06F2203/048Indexing scheme relating to G06F3/048
    • G06F2203/04805Virtual magnifying lens, i.e. window or frame movable on top of displayed information to enlarge it for better reading or selection
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F2203/00Indexing scheme relating to G06F3/00 - G06F3/048
    • G06F2203/048Indexing scheme relating to G06F3/048
    • G06F2203/04806Zoom, i.e. interaction techniques or interactors for controlling the zooming operation
    • GPHYSICS
    • G09EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09GARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION
    • G09G5/00Control arrangements or circuits for visual indicators common to cathode-ray tube indicators and other visual indicators
    • G09G5/22Control arrangements or circuits for visual indicators common to cathode-ray tube indicators and other visual indicators characterised by the display of characters or indicia using display control signals derived from coded signals representing the characters or indicia, e.g. with a character-code memory
    • G09G5/30Control of display attribute

Definitions

  • Embodiments disclosed herein relate in general to methods and system that enable visually impaired persons to read text displayed on smart-phones, tablets, computers, televisions and other screen-based devices and equipment.
  • Persons suffering from visual impairments such as Presbyopia (age-related loss of accommodation), Hyperopia (farsightedness), and Astigmatism (cylindrical deformation) require in general reading glasses.
  • a visually impaired person may not wear glasses for purposes other than reading, but is very likely to wear glasses each time he/she needs to read a text. This situation is encountered with increased frequency in a world dominated by mobile devices such as smart-phones, where texting, tweeting and other communications are expressed as text on the smart-phone screen.
  • the repeated actions of wearing and removing reading glasses are uncomfortable and bothersome. Reading glasses may be forgotten or misplaced, causing additional discomfort.
  • ASD Age-related Macular Degeneration
  • AMD Age-related Macular Degeneration
  • the Fovea positioned at the center of the Macula.
  • the Fovea is non-functional and the center of the visual field is lost.
  • One way to improve visual capability is to display images on the Para- Foveal regions of the Retina, which in many patients may still be functional (depending on the geographical involvement of the diseased Macula).
  • AMD affected persons have an extreme difficulty reading text displayed on electronic device screens, in particular on small screens such as in smart-phones.
  • methods for improving the readability of text displayed on an electronic device screen to a user comprising the steps of performing, for the user, a customization procedure that results in user-specific text enhancement parameters, and enhancing text displayed on the device screen according to the user-specific text enhancement parameters.
  • the text enhancement includes visual adjustment of the text and text scrolling in a region of interest (ROI).
  • the visual adjustment includes at least text enlargement to a size determined during customization and/or adjusted during use.
  • the text scrolling includes shifting an enhanced text section horizontally along a text line from left-to- right or from right-to-left.
  • the text enhancement may be triggered automatically or manually.
  • the ROI may be determined automatically or chosen manually.
  • the visual adjustment of the text may further include color contrast improvement, re-coloring, contouring, blinking, or a combination thereof.
  • the customization procedure is performed while the user does not wear reading glasses and the text displayed on the device screen is enhanced for reading without reading glasses.
  • the customization procedure may be performed while the user wears non-reader glasses, wherein the text displayed on the device screen is enhanced for reading with the non-reader glasses.
  • FIG. 1A shows schematically in a flow chart an embodiment of a method disclosed herein
  • FIG. IB shows schematically in a flow chart details of the method of FIG. 1A;
  • FIG. 2 shows an example of a webpage, used in the description of the following figures
  • FIG. 3 shows an example of a non-preserving enlarged text line in the Web page of FIG.
  • FIG. 4 shows an example of a preserving enlarged text line in the Web page of FIG. 2: (a) non-enlarged text and (b) enlarged text;
  • FIG. 5 shows an example of color contrast improvement and re-coloring performed on text in the Web page of FIG. 2: (a) original non-color contrast enhanced text and (b) enhanced color contrast enhanced text;
  • FIG. 6 shows an example of contouring performed on enlarged text in the Web page of FIG. 2: (a) non-contour-enhanced text and (b) contour-enhanced text;
  • FIG. 7 shows an example of non-preserving "full line” enlarged text scrolling in the Web page of FIG. 2: (a) beginning of text line scroll and (b) end of text line scroll;
  • FIG. 8 shows an example of preserving "full line” enlarged text scrolling in the Web page of FIG. 2: (a) beginning of text line scroll and (b) end of text line scroll;
  • FIG. 9 shows an example of scrolling behavior of enlarged text in the Web page of FIG. 2 with a "bubble" configuration: (a) beginning of text line scroll (bubble on the left) and (b) end of text line scroll (bubble on the right);
  • FIG. 10 shows an example of line breaking in the Web page of FIG. 2: (a) original page and (b) static enlargement of ROI paragraph;
  • FIG. 11 shows an illustration of an initial configuration screen for picking a suitable text size for reading without glasses
  • FIG. 12 shows an illustration of an advanced configuration screen
  • FIG. 13 shows schematically in a block diagram a system implementing a method disclosed herein.
  • Such devices include (but are not limited to) mobile devices such as cell- phones, tablet computers (“tablets”) or laptop computers, digital watches and e-book readers (e.g. Kindle), as well as non-mobile devices such as desktop computers and television sets. All such devices have electronic controls and inputs, either integrated or remote.
  • FIG. 1A shows schematically in a flow chart an embodiment of a method disclosed herein.
  • a particular visually impaired person referred to hereinafter as "user" wishes to read text displayed on a device (exemplarily a smart-phone) screen.
  • the text may be a short message service (SMS) text, Whatsapp text, Facebook text, email text, news text, or any other text displayable on a screen.
  • SMS short message service
  • a customization procedure (or simply "customization") is run in a training phase using a dedicated application (referred to sometimes just as “app"), step 102.
  • the app may be integrated into the operating system (OS) of the device, thereby becoming an integral part of the OS.
  • OS operating system
  • the customization is performed normally only once but may be repeated if the user wishes to do so, for example to adjust text enhancement or to add/change a user profile.
  • the customization may include providing personal details such as age, gender, Diopters correction (far vision spherical correction data for the left and for the right eye, reading spherical correction data for the left eye and for the right eye), astigmatism and preferred viewing distanced (from eyes to screen).
  • the customization may further include holding the smart- phone at a convenient distance without glasses and determining and setting (in a device memory) user - specific text enhancement parameters. These parameters may include, for each user and for each user's state (with/without glasses) a text enlargement factor and a text scrolling speed.
  • the customization results in a basic configuration of the app (and of the device).
  • the user may perform an advanced configuration.
  • the advanced configuration allows access to all of the basic configuration options, in addition to options such as text foreground color ("fore-color”), text background color (“back-color”), text enlargement type ("full line”, “bubble” or “line breaks”, see below), text blinking pattern, illumination, astigmatism correction, a "preferred trigger method” option and a "continue automatically to next paragraph” option after completion of enhancement of a text paragraph.
  • all options listed above have default initial values for users who do not wish to perform the optional advanced configuration.
  • Astigmatism correction may exemplarily be performed as follows: a representative character or figure like a circle is presented to the user. This object is then rotated and distorted in a circular fashion by elongating one region while narrowing its perpendicular counterpart. Such rotation may be done either in steps of 10 degrees from 0 degrees to 180 degrees, or by a dynamic progression of the distorted axis from 0 to 180 degrees.
  • the user may choose the preferred presented character, and this choice will be applied to all fonts in future texts presented to the user, i.e. will be one of the user - specific text enhancement parameters for that user).
  • text enhancement includes visual adjustment of the text and text scrolling.
  • text scrolling refers to the action of shifting an enhanced text section horizontally in left-to-right or right-to-left directions along a text line.
  • Visual adjustment of the text includes at least text enlargement.
  • it may include one or more added actions based on configuration options listed above.
  • the optional added text enhancement actions may include color contrast improvement, re- coloring, contouring, blinking, or a combination thereof.
  • Visual adjustment may be done using ordinary image processing, with the source of the image being either the device's screen image (prior to the adjustment) or an external source such as a picture from the device camera.
  • the text enhancement may be triggered automatically or manually, see details below. The text enhancement results in a significantly improved reading experience for the user.
  • FIG. IB shows schematically, in a flow chart, details of the method of FIG. 1A.
  • Customization and (optionally) advanced configuration are performed for each user on a particular device in a training phase step 110. It is assumed that the device is ON and that its screen is active. An active screen may exhibit text, images, or a combination of both.
  • An "operation" or “use” phase starts with step 112, which checks whether text enhancement needs to be triggered.
  • the triggering mechanism may be manual, e.g. by tapping, or automatic, e.g. triggering by opening an app, by an air gesture or by an eye gesture. Details of each triggering mechanism are given below If No, then the display remains unchanged, step 114.
  • a No answer in step 112 may result from a configuration of the app to not automatically enhance text unless triggered (by touch for example) or when no text is found on the screen. If Yes in step 112 and if applicable (triggered by touch), a "region of interest" (ROI) is determined by the location of the touch in step 116. Otherwise, the user may use a user input to control the ROI and adjust it in step 120, as elaborated on in the "User input” section below.
  • the ROI represents a portion of text on the screen in which text enhancement is triggered. Various mechanisms to determine the ROI are listed in detail below.
  • Text enhancement is then performed in step 120. The enhancement includes changing the way ROI text is exhibited such that is readable to users without glasses or to users with glasses suited for long distance vision.
  • Text Enhancement Triggering Text enhancement may be triggered automatically by recognizing common device apps such as SMS, Whatsapp, Email, Facebook or News when opening the respective device app.
  • the automatic triggering may be manually aborted by a touch on the screen, or by waving over and across the screen.
  • the automatic triggering may also be automatically overridden by recognition, by a front camera of the device, of glasses on the user's face (if such recognition has been configured into the device).
  • text enhancement may be triggered by manual tapping.
  • a manual tapping action may exemplarily include a two- finger static touch for a one second period on any portion of text visible on the screen, or a quick double-tap by two adjacent fingers (not to be mistaken for a standard double tap). Other manual tapping actions are of course possible.
  • text enhancement may be triggered by a floating gesture, such as placing the palm of the hand in front of the device for 1 second, the gesture captured by the front camera.
  • text enhancement may be triggered by recognizing an eye gesture, such as two quick eyelid closings or a 1 -second long eyelid closing.
  • determination of the ROI may be achieved by tapping.
  • the ROI may be determined by tapping on a particular position on the screen.
  • the user touches (taps) a location on the screen, in the general area of a chosen text paragraph he/she wishes to enhance.
  • the tap creates a text "box" around the point of touch.
  • the text box may include part of the paragraph or the entire paragraph, depending on the configuration of the dedicated app.
  • the ROI is then extended to the end of the chosen paragraph.
  • optional ROI adjustments may then be performed. These may include extension of the ROI (plus text enhancement) to the start of the chosen paragraph, starting the enhancement at the point of touch, or extending the ROI beyond the chosen paragraph to the end of the entire text.
  • the ROI may be determined automatically when triggered by an app.
  • the triggering may be provided exemplarily by a new SMS/Whatsapp/Facebook message, or by the latest headlines in a news app.
  • the ROI may be determined (guessed) by a "most likely text" criterion, triggered for example by an air gesture or an eye gesture.
  • the most likely text may include the first text line (from the top) or the largest text block on the screen. Since the choice of the most likely text is only a guess, an optional user input may be used to improve the ROI determination. Once some text is enhanced in the guessed ROI, the user may shift the ROI manually using an online user input, described below.
  • move (flow) to a next ROI may be performed automatically. This may involve choosing (by the dedicated app) the next most likely text block immediately below the previous ROI. If no text exists, the enhancement ends.
  • This "flow” is illustrated with an example in FIG. 2. Assume the first ROI determined by any way described above includes the "Corrective Lens - Wikipedia" paragraph. After enhancement of the text in that paragraph, the next, "Category: Corrective Lens” paragraph will be automatically chosen as the new ROI.
  • the text enlargement is achieved by mapping pixel coordinates from the target region ("full-line” or “bubble”, see text scrolling section below) back to the original rendering of the screen (with no enlargement).
  • the mapping is a degenerate affine transformation with only translation and axis-homogeneous scaling components.
  • the constant scaling value is drawn from the per-user configuration, and the translation component is computed and updated so as to simulate a scrolling behavior, from the beginning of the text-line to its end.
  • Naturally occurring fractional pixel coordinates may be interpolated by standard methods such as bi- linear or bi-cubic interpolations.
  • the text enlargement is preferably done to a font size best suiting the user.
  • the size may depend upon user distance from the device and whether the user wears glasses or not.
  • a text enlargement factor can be either constant (as preconfigured) or changed dynamically.
  • the dynamic change may include continuously measuring the distance of the device from the user using the device front-camera and any standard distance estimation method (for example measuring distance between the eyes, and other face feature-based estimations, and using the data to estimate distance of user from camera).
  • a single line in the ROI is enlarged at one time.
  • An exception may be in a "line-break" mode, explained below.
  • the enlarged line may be either "non-preserving", i.e. covering the adjacent lines above and below it (and therefore not altering rest of the screen layout), or "preserving” i.e. pushing the adjacent lines above and below it (and therefore changing the screen layout, but not hiding any other elements within it).
  • the enlarged line in both cases may not always fit in the screen. In cases where it does not fit, text scrolling (see below) is used.
  • FIG. 3 shows an example in which text in the ROI is not preserved, i.e. enlarged text overlaps (covers) surrounding text.
  • FIG. 3(a) shows a non-enhanced screen
  • FIG. 3(b) shows a screen with enhanced (enlarged) text.
  • the ROI is not shown (but may be the whole search result).
  • the text paragraph starting with "A corrective lens is a lens" is enhanced in (b), where the enlarged text includes the words "to treat myopia, hyperopia”. Note that the enlarged text covers some of the other text in the paragraph.
  • FIG. 4 shows an example in which text in the ROI is preserved, i.e. surrounding text is pushed and the screen layout is changed.
  • FIG. 4(a) shows a non-enhanced screen
  • FIG. 4(b) shows a screen with enhanced (enlarged) text.
  • the text paragraph chosen is the same as in FIG. 3(a)
  • the enlarged "to treat myopia, hyperopia" text in FIG. 4(b) is the same as in FIG. 3(b).
  • the enlarged text does not cover other text of the paragraph. Instead, the other text is pushed above and below the enlarged text.
  • the visual enhancement may include maximizing color contrast. Maximal color contrast may be achieved using any automatic thresholding algorithm, see e.g. N. Otsu, IEEE Trans, on Systems, Man and Cybernetics, Vol. 9 (1), pp. 62-66, 1979. Reading can be made more comfortable by changing fore-color and back-color to be other than black and white. Specifically, blue over yellow is known to be especially easy to read. The colors are configurable.
  • FIG. 5 show an example of color contrast improvement and re-coloring performed on
  • ROI text in a Web page is black on a white background.
  • the visual enhancement may include contouring (or a contour effect), i.e. transforming the letters such that only the outline remains and the interior of the letters is colored with the background color.
  • contouring or a contour effect
  • FIG. 6 An example of a contour effect is shown in FIG. 6.
  • the contour-enhanced text, shown in (b) is contoured, with a blue contour on yellow background.
  • the visual enhancement may include blinking the text at high rate. Such blinking may improve the reading experience of some users by drawing focus to the blinking text. The blinking may also make the text appear to move at a slower pace (stroboscopic effect).
  • Text scrolling For example, instead of enlarging an entire line in the ROI at one time (as above), only part of the selected line is enlarged in a full line mode.
  • "Full line” means that the part's width is selected such that its width after enlargement matches exactly the width of the screen.
  • the chosen part of the selected line is smaller if the text enlargement factor it bigger, and larger if the enlargement factor is smaller.
  • the chosen part is then shifted along the text line at a rate configurable by the user, until the line is finished.
  • parts of the screen that are not enlarged can by either non-preserved, i.e. shifted to allow space above and below the enlarged line (thus preserving the entire text but distorting its layout), see FIG. 7, or preserved (with some of the screen hidden by the enlarged text), see FIG. 8.
  • the enlarged "opia, astigmatism,” text in (b) is shifted left-to-right relative to the enlarged "mainly used to trea” text in (a) by scrolling. Some of the line and paragraph text is hidden, and reappears only when the scrolling of the line is finished and the next line begins scrolling.
  • the enlarged "opia, astigmatism,” text in (b) is also shifted left-to-right relative to the "mainly used to trea” text in (a) by scrolling.
  • the surrounding text is pushed down and the screen layout is changed.
  • the text scrolling may include slowdown or complete pause at commas and colons, to allow the user to fully comprehend the sentence. This effect may be provided by a configuration switch in an advanced configuration screen, see below
  • the final enlarged text width may be set to less than the full screen width, (for example 70% of screen width), with the enlargement factor staying the same, and the part of the text line enlarged being smaller.
  • the bubble text is positioned initially at the left or right extremes of the screen (LTR / RTL texts, respectively).
  • the bubble text is then moved along to the other extreme as the text line is scanned, until it reaches the end of the line, see FIG. 9.
  • the figure shows an example of scrolling behavior of enlarged text in the Web page of FIG. 2 with a "bubble" configuration: (a) beginning of text line scroll (bubble on the left) and (b) end of text line scroll (bubble on the right).
  • the enlarged text on the left reads "mainly use", while the enlarged text on the right reads "hyperopia,”. This allows a natural feeling of reading, with the eyes moving along with the text line (as in regular reading) while sacrificing some screen width.
  • FIG. 10 shows in (a) the original unaltered screen, and in (b) the result of the extra line-break scheme, where the entire paragraphs starting with "A corrective lens... " is statically enlarged, with extra line-breaks automatically inserted in the text.
  • This is a static (without animation) scheme that allows users who are having difficulty reading moving text to enjoy the enhancement scheme. In this scheme, manual interaction with the device is necessary, since no animation is employed.
  • a user reading enhanced text may control the ROI and the text enhancement in several ways. While reading enhanced text, the user may use an explicit touch action to change the ROI by swiping in the required direction (up or down). He/she can backtrack or move forward in a line by swiping left or right. Optionally, the user may use the up/down swipe for both types of changes: by swiping down first, the movement of the line is advanced until its completion, then the next line is enhanced. Swiping upwards does the reverse: first backs up the current line until its beginning, and then selects the previous line, at its end. Optionally yet, the user may terminate the enhancement mode by double-clicking or by other types of finger gestures, as proposed above
  • inertial devices located in the electronic device may be used to allow control of the enhanced region.
  • the pace of scrolling of an enhanced line text may be increased by tilting (rotating) the device forward or clockwise (towards one end of the text line) and decreased by tilting the device backward or counterclockwise (toward the other end of the text line).
  • a high backward tilt level may pause the text, and a high forward tilt level may retrace it.
  • head-tracking methods may be used for progressing through the ROI (same as tilting and swiping previously mentioned).
  • An effect similar to device tilting may be achieved by rotating the user's face to one direction or another. Advancements from line to line may be achieved by lowering or raising the face.
  • the face movements may be captured by the front camera, if the application is properly configured (i.e. if the user explicitly indicates in the configuration screen that he/she is interested in head- tracking).
  • control of the currently enhanced portion of the ROI may be achieved by moving the user's hand to one direction or the other.
  • the enhancement may be canceled by quickly moving the hand in front of the device.
  • the app can recognize the presence of glasses worn during text enhancement, and can stop the enhancement or change the enhancement parameters that were set previously by the user for the enhancement while wearing glasses. Similarly, the app can recognize presence of sunglasses. In an embodiment, the presence of sunglasses will trigger no change in text enhancement, based on a default configuration.
  • FIG. 11 shows an illustration of an initial configuration screen for picking a suitable text size for reading without glasses.
  • the initial configuration is performed per user.
  • the user may or may not wear glasses, and worn glasses may be reading glasses or non-reading glasses.
  • the customization may be manual or partly automatic.
  • a screen as in FIG. 11 may prompt/question the user to provide a username, eyeglasses, Diopters and astigmatism and reading speed.
  • the screen may also include switches like "activate by touch", "activate by eye-blinking", etc.
  • another screen as in FIG. 12 may expose all switches representing the parameters used in customization, i.e. coloring, contouring, blinking, preserving, etc. All switches have default values.
  • FIG. 13 shows schematically in a block diagram a system 1300 that can be used to implement a method disclosed herein.
  • System 1300 includes a dedicated server 1302 in communication with a central user database 1304 and with at least one electronic device 1306 that includes a screen 1308 and a dedicated app 1310 for performing the method as described above.
  • app 1310 may be integrated in the operating system of device 1306, thereby ceasing to be an independent entity.
  • Database 1304 stores statistical data on the configuration of various users, provided by the dedicated app in each device used by a user. The data may be processed by the server and used to provide a good first guess for text enhancement for new users.
  • App 1310 may periodically send current configuration + usage statistics to database 1304, to allow a good first guess for additional users and to reduce the customization process duration. This will shorten the first customization step for a new user.
  • a particular device may be used by several users.
  • each user is customized on the device with his/her own set of parameters, and the different users of the same device may be identified using face recognition. Switching between the different users may be performed automatically using the front device camera, or manually by accessing the application' s configuration screen.
  • a specific user may have different parameters for usage with and without glasses. In this case, the differentiation (for usage with or without glasses) can be accomplished using the front camera of the device or manually.
  • the methods and system disclosed herein can provide text enhancement on a display without removal of glasses.
  • patients with AMD may benefit from enlarged text and moving sentences with maximal resolution.
  • Enlarged fonts may be projected onto the Para-Foveal region and enhance reading by the dynamic movement and display of words and sentences to stimulate the functioning Para-Foveal cells in the Retina.
  • font size As the extent of damage and visual loss varies among different AMD patients, they will be able to choose the most comfortable parameters to fit their demands: font size, text speed, contrast, and different color displays.
  • the methods and system disclosed herein may also be useful for Myopic persons who prefer to remain with their glasses on while reading.
  • Myopic persons are enabled to choose their preferred way of reading, either with or without their glasses (depending on the extent of their Myopia), by choosing the apparatus parameters which fit them best.
  • the methods and system disclosed herein may also be used on texts originated by other sources, for example photographed documents, scanned documents, PDFs, and real-time video feeds, and used as a "magnifying glass”.
  • Computer executable instructions implementing an embodiment of a method disclosed herein can be distributed to users on a non-transitory computer-readable medium and are often copied onto a hard disk or other storage medium.
  • a program of instructions When such a program of instructions is to be executed, it is usually loaded into a random access memory of the electronic device thereby configuring the electronic device to act in accordance with a method disclosed herein. All these operations are well known to those skilled in the art and thus are not further described herein.
  • the term "computer-readable medium” encompasses distribution media, intermediate storage media, execution memory of a computer or a programmable electronic device, and any other medium or device capable of storing for later reading a computer program implementing embodiments of a method disclosed herein.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Physics & Mathematics (AREA)
  • Human Computer Interaction (AREA)
  • Business, Economics & Management (AREA)
  • Educational Technology (AREA)
  • Educational Administration (AREA)
  • Health & Medical Sciences (AREA)
  • General Health & Medical Sciences (AREA)
  • Audiology, Speech & Language Pathology (AREA)
  • User Interface Of Digital Computer (AREA)
  • Controls And Circuits For Display Device (AREA)

Abstract

Method and system for improving the readability of text displayed to a user on the screen of an electronic device such as a smart-phone. The method includes customizing an application included in the device with user-specific text enhancement parameters and enhancing text displayed on the device screen according to the user- specific text enhancement parameters. The text enhancement includes at least text enlargement and text scrolling and may be triggered automatically or manually. The method enables users who normally wear reading glasses to read text on the device screen without their reading glasses.

Description

METHODS AND SYSTEM FOR IMPROVING THE READABILITY OF TEXT DISPLAYED ON AN ELECTRONIC DEVICE SCREEN
CROSS REFERENCE TO EXISTING APPLICATIONS
This application is related to and claims priority from US Provisional Patent Application No. 61/883448 having the same title and filed September 27, 2013, which is incorporated herein by reference in its entirety. FIELD
Embodiments disclosed herein relate in general to methods and system that enable visually impaired persons to read text displayed on smart-phones, tablets, computers, televisions and other screen-based devices and equipment.
BACKGROUND
Persons suffering from visual impairments such as Presbyopia (age-related loss of accommodation), Hyperopia (farsightedness), and Astigmatism (cylindrical deformation) require in general reading glasses. A visually impaired person may not wear glasses for purposes other than reading, but is very likely to wear glasses each time he/she needs to read a text. This situation is encountered with increased frequency in a world dominated by mobile devices such as smart-phones, where texting, tweeting and other communications are expressed as text on the smart-phone screen. The repeated actions of wearing and removing reading glasses are uncomfortable and bothersome. Reading glasses may be forgotten or misplaced, causing additional discomfort.
Low vision due to Macular diseases is a major problem world-wide, especially in the elderly population. Age-related Macular Degeneration (AMD) accounts for 55-60% of blindness or low vision in the aging population. As the center of the visual field is affected by the disease process, these patients suffer from poor vision, with a special difficulty in reading. The normal precise vision is carried out by the Fovea, positioned at the center of the Macula. In AMD and other diseases affecting the macula, the Fovea is non-functional and the center of the visual field is lost. One way to improve visual capability is to display images on the Para- Foveal regions of the Retina, which in many patients may still be functional (depending on the geographical involvement of the diseased Macula). AMD affected persons have an extreme difficulty reading text displayed on electronic device screens, in particular on small screens such as in smart-phones.
. Therefore there is a need for, and it would be advantageous to have methods and apparatus that will allow visually impaired persons to read text displayed on electronic device screens without having to wear reading glasses, or improve the reading capability of patients with Macular diseases, especially AMD patients.
SUMMARY
In various embodiments, there are provided methods for improving the readability of text displayed on an electronic device screen to a user, comprising the steps of performing, for the user, a customization procedure that results in user-specific text enhancement parameters, and enhancing text displayed on the device screen according to the user-specific text enhancement parameters. The text enhancement includes visual adjustment of the text and text scrolling in a region of interest (ROI). The visual adjustment includes at least text enlargement to a size determined during customization and/or adjusted during use. The text scrolling includes shifting an enhanced text section horizontally along a text line from left-to- right or from right-to-left. The text enhancement may be triggered automatically or manually. The ROI may be determined automatically or chosen manually. In some embodiments, the visual adjustment of the text may further include color contrast improvement, re-coloring, contouring, blinking, or a combination thereof. In some embodiments, the customization procedure is performed while the user does not wear reading glasses and the text displayed on the device screen is enhanced for reading without reading glasses. In some embodiments, the customization procedure may be performed while the user wears non-reader glasses, wherein the text displayed on the device screen is enhanced for reading with the non-reader glasses.
BRIEF DESCRIPTION OF THE DRAWINGS Non-limiting examples of embodiments disclosed herein are described below with reference to figures attached hereto that are listed following this paragraph. The drawings and descriptions are meant to illuminate and clarify embodiments disclosed herein, and should not be considered limiting in any way.
FIG. 1A shows schematically in a flow chart an embodiment of a method disclosed herein;
FIG. IB shows schematically in a flow chart details of the method of FIG. 1A;
FIG. 2 shows an example of a webpage, used in the description of the following figures
3-10;
FIG. 3 shows an example of a non-preserving enlarged text line in the Web page of FIG.
2: (a) non-enlarged text and (b) enlarged text;
FIG. 4 shows an example of a preserving enlarged text line in the Web page of FIG. 2: (a) non-enlarged text and (b) enlarged text;
FIG. 5 shows an example of color contrast improvement and re-coloring performed on text in the Web page of FIG. 2: (a) original non-color contrast enhanced text and (b) enhanced color contrast enhanced text;
FIG. 6 shows an example of contouring performed on enlarged text in the Web page of FIG. 2: (a) non-contour-enhanced text and (b) contour-enhanced text;
FIG. 7 shows an example of non-preserving "full line" enlarged text scrolling in the Web page of FIG. 2: (a) beginning of text line scroll and (b) end of text line scroll;
FIG. 8 shows an example of preserving "full line" enlarged text scrolling in the Web page of FIG. 2: (a) beginning of text line scroll and (b) end of text line scroll;
FIG. 9 shows an example of scrolling behavior of enlarged text in the Web page of FIG. 2 with a "bubble" configuration: (a) beginning of text line scroll (bubble on the left) and (b) end of text line scroll (bubble on the right);
FIG. 10 shows an example of line breaking in the Web page of FIG. 2: (a) original page and (b) static enlargement of ROI paragraph;
FIG. 11 shows an illustration of an initial configuration screen for picking a suitable text size for reading without glasses;
FIG. 12 shows an illustration of an advanced configuration screen;
FIG. 13 shows schematically in a block diagram a system implementing a method disclosed herein.
DETAILED DESCRIPTION
The following description refers to implementation of method embodiments disclosed herein exemplarily in smart-phones, with the understanding that such embodiments can be implemented in other type of devices having screens (also referred to herein as "displays") that display text. Such devices include (but are not limited to) mobile devices such as cell- phones, tablet computers ("tablets") or laptop computers, digital watches and e-book readers (e.g. Kindle), as well as non-mobile devices such as desktop computers and television sets. All such devices have electronic controls and inputs, either integrated or remote.
FIG. 1A shows schematically in a flow chart an embodiment of a method disclosed herein. Assume that a particular visually impaired person (referred to hereinafter as "user") wishes to read text displayed on a device (exemplarily a smart-phone) screen. The text may be a short message service (SMS) text, Whatsapp text, Facebook text, email text, news text, or any other text displayable on a screen. A customization procedure (or simply "customization") is run in a training phase using a dedicated application (referred to sometimes just as "app"), step 102. In some devices such as smart-phones, tablets, computers, etc., the app may be integrated into the operating system (OS) of the device, thereby becoming an integral part of the OS. The customization is performed normally only once but may be repeated if the user wishes to do so, for example to adjust text enhancement or to add/change a user profile. The customization may include providing personal details such as age, gender, Diopters correction (far vision spherical correction data for the left and for the right eye, reading spherical correction data for the left eye and for the right eye), astigmatism and preferred viewing distanced (from eyes to screen). The customization may further include holding the smart- phone at a convenient distance without glasses and determining and setting (in a device memory) user - specific text enhancement parameters. These parameters may include, for each user and for each user's state (with/without glasses) a text enlargement factor and a text scrolling speed. The customization results in a basic configuration of the app (and of the device). Optionally, after the basic configuration or at any other time, the user may perform an advanced configuration. The advanced configuration allows access to all of the basic configuration options, in addition to options such as text foreground color ("fore-color"), text background color ("back-color"), text enlargement type ("full line", "bubble" or "line breaks", see below), text blinking pattern, illumination, astigmatism correction, a "preferred trigger method" option and a "continue automatically to next paragraph" option after completion of enhancement of a text paragraph. Note that all options listed above have default initial values for users who do not wish to perform the optional advanced configuration.
Astigmatism correction may exemplarily be performed as follows: a representative character or figure like a circle is presented to the user. This object is then rotated and distorted in a circular fashion by elongating one region while narrowing its perpendicular counterpart. Such rotation may be done either in steps of 10 degrees from 0 degrees to 180 degrees, or by a dynamic progression of the distorted axis from 0 to 180 degrees. The user may choose the preferred presented character, and this choice will be applied to all fonts in future texts presented to the user, i.e. will be one of the user - specific text enhancement parameters for that user).
Once the device is configured by customization, text enhancement for the particular user is performed on the device screen in step 104. As used herein, the term "text enhancement" includes visual adjustment of the text and text scrolling. Furthermore, as used herein, "text scrolling" refers to the action of shifting an enhanced text section horizontally in left-to-right or right-to-left directions along a text line.
Visual adjustment of the text includes at least text enlargement. Optionally, it may include one or more added actions based on configuration options listed above. In particular, the optional added text enhancement actions may include color contrast improvement, re- coloring, contouring, blinking, or a combination thereof. Visual adjustment may be done using ordinary image processing, with the source of the image being either the device's screen image (prior to the adjustment) or an external source such as a picture from the device camera. The text enhancement may be triggered automatically or manually, see details below. The text enhancement results in a significantly improved reading experience for the user.
FIG. IB shows schematically, in a flow chart, details of the method of FIG. 1A. Customization and (optionally) advanced configuration are performed for each user on a particular device in a training phase step 110. It is assumed that the device is ON and that its screen is active. An active screen may exhibit text, images, or a combination of both. An "operation" or "use" phase starts with step 112, which checks whether text enhancement needs to be triggered. The triggering mechanism may be manual, e.g. by tapping, or automatic, e.g. triggering by opening an app, by an air gesture or by an eye gesture. Details of each triggering mechanism are given below If No, then the display remains unchanged, step 114. A No answer in step 112 may result from a configuration of the app to not automatically enhance text unless triggered (by touch for example) or when no text is found on the screen. If Yes in step 112 and if applicable (triggered by touch), a "region of interest" (ROI) is determined by the location of the touch in step 116. Otherwise, the user may use a user input to control the ROI and adjust it in step 120, as elaborated on in the "User input" section below. The ROI represents a portion of text on the screen in which text enhancement is triggered. Various mechanisms to determine the ROI are listed in detail below. Text enhancement is then performed in step 120. The enhancement includes changing the way ROI text is exhibited such that is readable to users without glasses or to users with glasses suited for long distance vision.
Text Enhancement Triggering Text enhancement may be triggered automatically by recognizing common device apps such as SMS, Whatsapp, Email, Facebook or News when opening the respective device app. The automatic triggering may be manually aborted by a touch on the screen, or by waving over and across the screen. The automatic triggering may also be automatically overridden by recognition, by a front camera of the device, of glasses on the user's face (if such recognition has been configured into the device). Alternatively, text enhancement may be triggered by manual tapping. A manual tapping action may exemplarily include a two- finger static touch for a one second period on any portion of text visible on the screen, or a quick double-tap by two adjacent fingers (not to be mistaken for a standard double tap). Other manual tapping actions are of course possible. Alternatively yet, text enhancement may be triggered by a floating gesture, such as placing the palm of the hand in front of the device for 1 second, the gesture captured by the front camera. Alternatively yet, text enhancement may be triggered by recognizing an eye gesture, such as two quick eyelid closings or a 1 -second long eyelid closing. ROI determination
In some embodiments, determination of the ROI may be achieved by tapping. The ROI may be determined by tapping on a particular position on the screen. The user touches (taps) a location on the screen, in the general area of a chosen text paragraph he/she wishes to enhance. The tap creates a text "box" around the point of touch. The text box may include part of the paragraph or the entire paragraph, depending on the configuration of the dedicated app. The ROI is then extended to the end of the chosen paragraph. Depending on the configuration, optional ROI adjustments may then be performed. These may include extension of the ROI (plus text enhancement) to the start of the chosen paragraph, starting the enhancement at the point of touch, or extending the ROI beyond the chosen paragraph to the end of the entire text.
In other embodiments, the ROI may be determined automatically when triggered by an app. The triggering may be provided exemplarily by a new SMS/Whatsapp/Facebook message, or by the latest headlines in a news app. In yet other embodiments, the ROI may be determined (guessed) by a "most likely text" criterion, triggered for example by an air gesture or an eye gesture. The most likely text may include the first text line (from the top) or the largest text block on the screen. Since the choice of the most likely text is only a guess, an optional user input may be used to improve the ROI determination. Once some text is enhanced in the guessed ROI, the user may shift the ROI manually using an online user input, described below.
Once enhancement of text in an ROI determined by any of the abovementioned ways is complete, move (flow) to a next ROI may be performed automatically. This may involve choosing (by the dedicated app) the next most likely text block immediately below the previous ROI. If no text exists, the enhancement ends. This "flow" is illustrated with an example in FIG. 2. Assume the first ROI determined by any way described above includes the "Corrective Lens - Wikipedia" paragraph. After enhancement of the text in that paragraph, the next, "Category: Corrective Lens" paragraph will be automatically chosen as the new ROI.
Text enlargement
The text enlargement is achieved by mapping pixel coordinates from the target region ("full-line" or "bubble", see text scrolling section below) back to the original rendering of the screen (with no enlargement). The mapping is a degenerate affine transformation with only translation and axis-homogeneous scaling components. The constant scaling value is drawn from the per-user configuration, and the translation component is computed and updated so as to simulate a scrolling behavior, from the beginning of the text-line to its end. Naturally occurring fractional pixel coordinates may be interpolated by standard methods such as bi- linear or bi-cubic interpolations.
The text enlargement is preferably done to a font size best suiting the user. The size may depend upon user distance from the device and whether the user wears glasses or not. By adjusting the app configuration (set originally based on the customization), a text enlargement factor can be either constant (as preconfigured) or changed dynamically. The dynamic change may include continuously measuring the distance of the device from the user using the device front-camera and any standard distance estimation method (for example measuring distance between the eyes, and other face feature-based estimations, and using the data to estimate distance of user from camera).
In general, only a single line in the ROI is enlarged at one time. An exception may be in a "line-break" mode, explained below. The enlarged line may be either "non-preserving", i.e. covering the adjacent lines above and below it (and therefore not altering rest of the screen layout), or "preserving" i.e. pushing the adjacent lines above and below it (and therefore changing the screen layout, but not hiding any other elements within it). The enlarged line in both cases may not always fit in the screen. In cases where it does not fit, text scrolling (see below) is used.
FIG. 3 shows an example in which text in the ROI is not preserved, i.e. enlarged text overlaps (covers) surrounding text. FIG. 3(a) shows a non-enhanced screen, while FIG. 3(b) shows a screen with enhanced (enlarged) text. The ROI is not shown (but may be the whole search result). The text paragraph starting with "A corrective lens is a lens..." is enhanced in (b), where the enlarged text includes the words "to treat myopia, hyperopia". Note that the enlarged text covers some of the other text in the paragraph.
FIG. 4 shows an example in which text in the ROI is preserved, i.e. surrounding text is pushed and the screen layout is changed. FIG. 4(a) shows a non-enhanced screen, while FIG. 4(b) shows a screen with enhanced (enlarged) text. The text paragraph chosen is the same as in FIG. 3(a), and the enlarged "to treat myopia, hyperopia" text in FIG. 4(b) is the same as in FIG. 3(b). However, the enlarged text does not cover other text of the paragraph. Instead, the other text is pushed above and below the enlarged text.
Additionally to text enlargement, the visual enhancement may include maximizing color contrast. Maximal color contrast may be achieved using any automatic thresholding algorithm, see e.g. N. Otsu, IEEE Trans, on Systems, Man and Cybernetics, Vol. 9 (1), pp. 62-66, 1979. Reading can be made more comfortable by changing fore-color and back-color to be other than black and white. Specifically, blue over yellow is known to be especially easy to read. The colors are configurable.
FIG. 5 show an example of color contrast improvement and re-coloring performed on
ROI text in a Web page. The original non-color contrast enhanced "to treat myopia, hyperopia" text, shown in (a), is black on a white background. The enhanced color contrast enhanced "to treat myopia, hyperopia" text, shown in (b), is blue on a yellow background.
Additionally yet to text enlargement, the visual enhancement may include contouring (or a contour effect), i.e. transforming the letters such that only the outline remains and the interior of the letters is colored with the background color. An example of a contour effect is shown in FIG. 6. The original non-contour-enhanced text, shown in (a) and reading "to treat myopia, hyperopia," is full blue on a yellow background. The contour-enhanced text, shown in (b), is contoured, with a blue contour on yellow background. Additionally yet to text enlargement, the visual enhancement may include blinking the text at high rate. Such blinking may improve the reading experience of some users by drawing focus to the blinking text. The blinking may also make the text appear to move at a slower pace (stroboscopic effect).
Text scrolling
It is likely that text enlarged as above will not fit inside the screen. This problem may be dealt with in several ways, termed "text scrolling". For example, instead of enlarging an entire line in the ROI at one time (as above), only part of the selected line is enlarged in a full line mode. "Full line" means that the part's width is selected such that its width after enlargement matches exactly the width of the screen. Thus, the chosen part of the selected line is smaller if the text enlargement factor it bigger, and larger if the enlargement factor is smaller. The chosen part is then shifted along the text line at a rate configurable by the user, until the line is finished.
To allow both right-to-left and left-to-right lines to be correctly displayed, it is important to recognize the type of letters in the text lines, i.e. if the letters represent a language written left-to-right or right-to left. This can be achieved by querying the device for the text in the ROI or by using optical character recognition (OCR). The user may also switch the text direction manually through a user input.
As mentioned, parts of the screen that are not enlarged can by either non-preserved, i.e. shifted to allow space above and below the enlarged line (thus preserving the entire text but distorting its layout), see FIG. 7, or preserved (with some of the screen hidden by the enlarged text), see FIG. 8. In FIG. 7, the enlarged "opia, astigmatism," text in (b) is shifted left-to-right relative to the enlarged "mainly used to trea" text in (a) by scrolling. Some of the line and paragraph text is hidden, and reappears only when the scrolling of the line is finished and the next line begins scrolling. In FIG. 8, the enlarged "opia, astigmatism," text in (b) is also shifted left-to-right relative to the "mainly used to trea" text in (a) by scrolling. Here however, the surrounding text is pushed down and the screen layout is changed.
Alternatively or additionally, the text scrolling may include slowdown or complete pause at commas and colons, to allow the user to fully comprehend the sentence. This effect may be provided by a configuration switch in an advanced configuration screen, see below
Alternatively, the final enlarged text width may be set to less than the full screen width, (for example 70% of screen width), with the enlargement factor staying the same, and the part of the text line enlarged being smaller. This results in a text "bubble". The bubble text is positioned initially at the left or right extremes of the screen (LTR / RTL texts, respectively). The bubble text is then moved along to the other extreme as the text line is scanned, until it reaches the end of the line, see FIG. 9. The figure shows an example of scrolling behavior of enlarged text in the Web page of FIG. 2 with a "bubble" configuration: (a) beginning of text line scroll (bubble on the left) and (b) end of text line scroll (bubble on the right). The enlarged text on the left reads "mainly use", while the enlarged text on the right reads "hyperopia,". This allows a natural feeling of reading, with the eyes moving along with the text line (as in regular reading) while sacrificing some screen width.
Alternatively yet, the entire screen can be used to accommodate the ROI, by adding extra line-breaks between words and possibly within words, and by eliminating unneeded line breaks. This "line-break" effect is illustrated in FIG. 10. FIG. 10 shows in (a) the original unaltered screen, and in (b) the result of the extra line-break scheme, where the entire paragraphs starting with "A corrective lens... " is statically enlarged, with extra line-breaks automatically inserted in the text. This is a static (without animation) scheme that allows users who are having difficulty reading moving text to enjoy the enhancement scheme. In this scheme, manual interaction with the device is necessary, since no animation is employed.
User input
A user reading enhanced text may control the ROI and the text enhancement in several ways. While reading enhanced text, the user may use an explicit touch action to change the ROI by swiping in the required direction (up or down). He/she can backtrack or move forward in a line by swiping left or right. Optionally, the user may use the up/down swipe for both types of changes: by swiping down first, the movement of the line is advanced until its completion, then the next line is enhanced. Swiping upwards does the reverse: first backs up the current line until its beginning, and then selects the previous line, at its end. Optionally yet, the user may terminate the enhancement mode by double-clicking or by other types of finger gestures, as proposed above
Additionally, inertial devices (e.g. accelerometers or gyroscopes) located in the electronic device may be used to allow control of the enhanced region. For example, the pace of scrolling of an enhanced line text may be increased by tilting (rotating) the device forward or clockwise (towards one end of the text line) and decreased by tilting the device backward or counterclockwise (toward the other end of the text line). A high backward tilt level may pause the text, and a high forward tilt level may retrace it.
Additionally or alternatively, head-tracking methods may be used for progressing through the ROI (same as tilting and swiping previously mentioned). An effect similar to device tilting may be achieved by rotating the user's face to one direction or another. Advancements from line to line may be achieved by lowering or raising the face. The face movements may be captured by the front camera, if the application is properly configured (i.e. if the user explicitly indicates in the configuration screen that he/she is interested in head- tracking).
Additionally or alternatively yet, control of the currently enhanced portion of the ROI may be achieved by moving the user's hand to one direction or the other. The enhancement may be canceled by quickly moving the hand in front of the device.
Users with glasses The app can recognize the presence of glasses worn during text enhancement, and can stop the enhancement or change the enhancement parameters that were set previously by the user for the enhancement while wearing glasses. Similarly, the app can recognize presence of sunglasses. In an embodiment, the presence of sunglasses will trigger no change in text enhancement, based on a default configuration.
FIG. 11 shows an illustration of an initial configuration screen for picking a suitable text size for reading without glasses. The initial configuration is performed per user. The user may or may not wear glasses, and worn glasses may be reading glasses or non-reading glasses. The customization may be manual or partly automatic. Exemplarily, a screen as in FIG. 11 may prompt/question the user to provide a username, eyeglasses, Diopters and astigmatism and reading speed. The screen may also include switches like "activate by touch", "activate by eye-blinking", etc. Exemplarily, another screen as in FIG. 12 may expose all switches representing the parameters used in customization, i.e. coloring, contouring, blinking, preserving, etc. All switches have default values. System
FIG. 13 shows schematically in a block diagram a system 1300 that can be used to implement a method disclosed herein. System 1300 includes a dedicated server 1302 in communication with a central user database 1304 and with at least one electronic device 1306 that includes a screen 1308 and a dedicated app 1310 for performing the method as described above. As mentioned, app 1310 may be integrated in the operating system of device 1306, thereby ceasing to be an independent entity. Database 1304 stores statistical data on the configuration of various users, provided by the dedicated app in each device used by a user. The data may be processed by the server and used to provide a good first guess for text enhancement for new users. App 1310 may periodically send current configuration + usage statistics to database 1304, to allow a good first guess for additional users and to reduce the customization process duration. This will shorten the first customization step for a new user.
A particular device may be used by several users. In such a case, each user is customized on the device with his/her own set of parameters, and the different users of the same device may be identified using face recognition. Switching between the different users may be performed automatically using the front device camera, or manually by accessing the application' s configuration screen. Furthermore, a specific user may have different parameters for usage with and without glasses. In this case, the differentiation (for usage with or without glasses) can be accomplished using the front camera of the device or manually.
Conclusion
In conclusion, the methods and system disclosed herein can provide text enhancement on a display without removal of glasses. For example, patients with AMD may benefit from enlarged text and moving sentences with maximal resolution. Enlarged fonts may be projected onto the Para-Foveal region and enhance reading by the dynamic movement and display of words and sentences to stimulate the functioning Para-Foveal cells in the Retina. As the extent of damage and visual loss varies among different AMD patients, they will be able to choose the most comfortable parameters to fit their demands: font size, text speed, contrast, and different color displays.
The methods and system disclosed herein may also be useful for Myopic persons who prefer to remain with their glasses on while reading. Myopic persons are enabled to choose their preferred way of reading, either with or without their glasses (depending on the extent of their Myopia), by choosing the apparatus parameters which fit them best.
The methods and system disclosed herein may also be used on texts originated by other sources, for example photographed documents, scanned documents, PDFs, and real-time video feeds, and used as a "magnifying glass".
The various features and steps discussed above, as well as other known equivalents for each such feature or step, can be mixed and matched by one of ordinary skill in this art to perform methods in accordance with principles described herein. Although the disclosure has been provided in the context of certain embodiments and examples, it will be understood by those skilled in the art that the disclosure extends beyond the specifically described embodiments to other alternative embodiments and/or uses and obvious modifications and equivalents thereof. Accordingly, the disclosure is not intended to be limited by the specific disclosures of embodiments herein. Any electronic device having a display can be configured or otherwise programmed to implement a method disclosed herein, and to the extent that a particular electronic device is configured to implement such a method, it is within the scope and spirit of the disclosure. Once an electronic device is programmed to perform particular functions pursuant to computer-executable instructions from program software that implements a method disclosed herein, it in effect becomes a special purpose device particular to an embodiment of the method disclosed herein. The techniques necessary to achieve this are well known to those skilled in the art and thus are not further described herein.
Computer executable instructions implementing an embodiment of a method disclosed herein can be distributed to users on a non-transitory computer-readable medium and are often copied onto a hard disk or other storage medium. When such a program of instructions is to be executed, it is usually loaded into a random access memory of the electronic device thereby configuring the electronic device to act in accordance with a method disclosed herein. All these operations are well known to those skilled in the art and thus are not further described herein. The term "computer-readable medium" encompasses distribution media, intermediate storage media, execution memory of a computer or a programmable electronic device, and any other medium or device capable of storing for later reading a computer program implementing embodiments of a method disclosed herein.

Claims

WHAT IS CLAIMED IS
1. A method for improving the readability of text displayed on an electronic device screen to a user, comprising the steps of: per user:
a) performing a customization procedure that results in user-specific text enhancement parameters; and
b) enhancing text displayed on the device screen according to the user-specific text enhancement parameters.
2. The method of claim 1, wherein the step of performing a customization procedure includes configuring a dedicated app in the device with the user-specific text enhancement parameters.
3. The method of claim 2, wherein the step of enhancing text displayed on the device screen according to the user-specific text enhancement parameters includes visual adjustment of the text and text scrolling in a region of interest.
4. The method of claim 3, wherein the visual adjustment of the text and the text scrolling are triggered automatically.
5. The method of claim 3, wherein the visual adjustment of the text and the text scrolling are triggered manually.
6. The method of claim 3, wherein the region of interest is determined automatically.
7. The method of claim 3, wherein the region of interest is chosen manually.
8. The method of claim 3, wherein the visual adjustment of the text includes text enlargement.
9. The method of claim 8, wherein the visual adjustment of the text further includes an action selected from the group consisting of color contrast improvement, re-coloring, contouring, blinking, and a combination thereof.
10. The method of claim 3, wherein the text scrolling includes shifting an enhanced text section horizontally along a text line.
11. The method of claim 10, wherein the shifting is in left-to-right or right-to-left directions.
12. The method of claim 1, wherein the step of performing a customization procedure includes performing the customization procedure while the user does not wear reading glasses, and wherein the step enhancing includes enhancing the text displayed on the device screen for reading without reading glasses.
13. The method of claim 1, wherein the step of performing a customization procedure includes performing the customization procedure while the user wears non- reader glasses, and wherein the step enhancing includes enhancing the text displayed on the device screen for reading with the non-reader glasses.
14. A system for improving the readability of text displayed on an electronic device screen to a user, comprising:
a) a dedicated app included in the device for performing, for the user, a customization procedure that results in user-specific text enhancement parameters, and for enhancing text displayed on the device screen according to the user-specific text enhancement parameters;
b) a database for storing at least some of the user-specific text enhancement parameters and user statistical data; and
c) a server for processing data stored in the database and for providing an initial guess of text enhancement parameters for a new user.
PCT/IB2014/064521 2013-09-27 2014-09-15 Methods and system for improving the readability of text displayed on an electronic device screen WO2015044830A1 (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
US201361883448P 2013-09-27 2013-09-27
US61/883,448 2013-09-27

Publications (1)

Publication Number Publication Date
WO2015044830A1 true WO2015044830A1 (en) 2015-04-02

Family

ID=52742167

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/IB2014/064521 WO2015044830A1 (en) 2013-09-27 2014-09-15 Methods and system for improving the readability of text displayed on an electronic device screen

Country Status (2)

Country Link
TW (1) TW201531931A (en)
WO (1) WO2015044830A1 (en)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP3304528A1 (en) * 2015-05-29 2018-04-11 Deutsche Telekom AG Augmented reality systems and methods to access optical information
EP3565284A4 (en) * 2017-01-12 2020-02-19 Samsung Electronics Co., Ltd. Apparatus and method for providing adaptive user interface

Families Citing this family (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR20180122609A (en) * 2016-01-18 2018-11-13 웨이브시프트 엘엘씨 Evaluation and reduction of near-term effect of electronic display
TWI776159B (en) * 2020-05-26 2022-09-01 林璟辰 Dispaly method and electronic device using the same

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20110119620A1 (en) * 2009-11-18 2011-05-19 Schoen Johan Snapping to text columns while scrolling
US20120268366A1 (en) * 2009-12-23 2012-10-25 Motorola Mobility Llc Method and Device for Visual Compensation
US20130150086A1 (en) * 2011-12-07 2013-06-13 Ebay Inc. Systems and methods for generating location-based group recommendations

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20110119620A1 (en) * 2009-11-18 2011-05-19 Schoen Johan Snapping to text columns while scrolling
US20120268366A1 (en) * 2009-12-23 2012-10-25 Motorola Mobility Llc Method and Device for Visual Compensation
US20130150086A1 (en) * 2011-12-07 2013-06-13 Ebay Inc. Systems and methods for generating location-based group recommendations

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
"ITJungles.", SAMSUNG GALAXY S3: HOW TO ENABLE THE VOLUME KEY TO INCREASE FONT SIZE IN MESSAGING APP., 15 April 2013 (2013-04-15), Retrieved from the Internet <URL:https://www.youtube.com/watch?v=0_SHgq3O7F1> [retrieved on 20150113] *
JUANBAGNELL.: "YTR ProTip: Nokia Lumia 920 High Contrast Mode.", 2012, Retrieved from the Internet <URL:https://www.youtube.com/watch?v=OQ16m_zMjK8> [retrieved on 20150113] *

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP3304528A1 (en) * 2015-05-29 2018-04-11 Deutsche Telekom AG Augmented reality systems and methods to access optical information
EP3565284A4 (en) * 2017-01-12 2020-02-19 Samsung Electronics Co., Ltd. Apparatus and method for providing adaptive user interface
US10852904B2 (en) 2017-01-12 2020-12-01 Samsung Electronics Co., Ltd. Apparatus and method for providing adaptive user interface

Also Published As

Publication number Publication date
TW201531931A (en) 2015-08-16

Similar Documents

Publication Publication Date Title
US11727695B2 (en) Language element vision augmentation methods and devices
US11714487B2 (en) Gaze and smooth pursuit based continuous foveal adjustment
Deemer et al. Low vision enhancement with head-mounted video display systems: are we there yet?
CN105164576B (en) The method that the wear-type electro-optic device for adapting to wearer is controlled
KR20170104463A (en) System and method for improved display
US11385468B2 (en) Hybrid see through augmented reality systems and methods for low vision users
US20170053443A1 (en) Gesture-based reorientation and navigation of a virtual reality (vr) interface
KR102306768B1 (en) Method for assisting visual exploration for an individual suffering from a retinal condition resulting in a scotoma
WO2015044830A1 (en) Methods and system for improving the readability of text displayed on an electronic device screen
US20210303107A1 (en) Devices, methods, and graphical user interfaces for gaze-based navigation
Aguilar et al. Evaluation of a gaze-controlled vision enhancement system for reading in visually impaired people
CN110708405A (en) Folding screen peeping prevention method and folding screen electronic equipment with peeping prevention function
CN110998490A (en) Terminal use mode information collection management method and system
US20240103682A1 (en) Devices, Methods, and Graphical User Interfaces for Interacting with Window Controls in Three-Dimensional Environments
JP2015206816A (en) Display program, display device and display method

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 14847197

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 14847197

Country of ref document: EP

Kind code of ref document: A1