AU2007203103B2 - Reading Product Fabrication Methodology - Google Patents

Description

AUSTRALIA

Patents Act 1990 COMPLETE SPECIFICATION Standard Patent Applicant(s): Walker Reading Technologies, Inc., Invention Title: Reading Product Fabrication Methodology The following statement is a full description of this invention, including the best method for performing it known to me/us: READING PRODUCT FABRICATION METHODOLOGY n ~Reference to Prior Application M This application is a divisional application of Australian patent application No. 2004200447 which, in turn, is a divisional application of Australian patent application No. 24920/99 (767280) and relates to subject M matter disclosed but not claimed in these applications.

c-I Some description of the invention of 2004200447 and 24920/99 is included herein to facilitate the understanding of the present invention. If necessary reference may be had to the disclosure of applications 2004200447 and 24920/99 to understand the present invention, and the disclosure of these applications is incorporated herein by reference.

Field of the Invention The present invention relates to computer assisted text presentation. More specifically, the invention relates to enhanced text presentation using content specific attributes to improve the reading experience.

BACKGROUND OF THE INVENTION The purpose of reading is to make an impact on the mind of the reader. This is true whether the text being read comprises a novel or a heads-up aircraft display.

Material presented in a non-textual medium conveys information well suited to human absorption beyond the corresponding, one-dimensional text. Still pictures present object attributes such as colors, relative sizes, relative locations, patterns, groupings, and hierarchies.

An object is viewed in a context. For example, a picture of a small object located with others in a corner conveys different information than the same object, enlarged, by itself, and in the center. Beginning sentences and paragraphs add contextual information by their location alone. Moving pictures have movement and temporal change Sas added attributes. Much information comes into the brain visually and through pattern recognition. Other information comes as audio information, conveying ee information through tone, changing inflection, and changing volume level.

Computer presentation of text for human reading has failed Cc to utilize much of the human brain. As a result, only, -Iba part or the possible bandwidth is utilized, and computer O text presentation is often thought of as unfriendly, constant, mechanical, and generally lacking in some way.

c-s When given the choice, most people prefer to read a book in 0 5 print rather than a book on a computer screen. Current computer text presentation is poor relative to a book, and even poorer relative to its potential.

cSome work has been done in computerized text presentation. Huanng Patent No. 4,880,385) discloses an opto-mechanical speed reading device that allows viewing c text printed on paper through a viewing window one line at a time, automatically advancing lines.

Advances have been made in methods for the computerized parsing of natural language into parts of speech. Schabes et al. Patent No. 5,475,588), disclose an improved parsing system for creating parse trees. Black, Jr. et al.

Patent No. 5,331, 556) disclose a method for storing parts of speech information in a file along with the original text for improved subsequent searching of the text. Okamoto et al. Patent No. 4,661,924), and Church Patent No. 5,146,405) disclose methods for disambiguating multipleparts-of-speech. Zamora et al. Patent No. 4,887,212), van Vliembergen Patent No. 5,068,789), Hemphill et al.

Patent No. 5,083,268) disclose methods for parsing natural language text. All of the above cited patents are herein incorporated by reference.

Reading is a complex process and there are many factors that determine differences in reading performance between readers and even for the same reader on different occasions.

These include innate neurophysiological conditions such as dyslexia, but also age; behavioral and motivational factors; -2- ±evei or education and prior reading practice; and perceptual O constraints. A reader may also have different reading objectives that will affect how he/she approaches the reading material. Despite all of the above, text is presented by computers, at worst, as one dimensional "beads on a string", at best, similar to the two dimensional text presented by books. Indeed, whether text is presented in print or on an electronic display matters little because the text CI presentation is essentially identical.

Examples of the constant presentation of text are numerous. Topic sentences are presented the same as other sentences in a paragraph. Technical words are presented no different than non-technical words. Adjectives appear the same as nouns. Educationally difficult terms are presented the same as simple terms. The last paragraph in a chapter is presented the same as the first paragraph. Text is presented in left justified, broken lines of constant width requiring a jerking eye movement from far right to far left at regular intervals. Text is broken in mid-sentence, midphrase and mid-thought, applying ancient typesetting rules.

Such text forces the eye to travel back and forth over large distances, mimicking a typewriter carriage. Text is advanced manually, broken into chunks determined by how many lines can fit on the screen.

There are several good reasons for not placing only one enhanced sentence per page in a book. These reasons may have been inappropriately carried over into the computer display of text. The possibility of modifying text presentation to enhance the reader's ability to read text by harnessing the power of digital manipulation appears to have been overlooked by those most knowledgeable in computing.

What would be desirable is a method for displaying the location of text with in relation to the text hierarchies within which the text resides. A method for displaying text on opposed pages which allows reading text without waiting for page changes would also be desirable.

A text display system making use of the natural human ability and preference to move forward while viewing objects would be useful. A method for displaying text CI using quantitative color modification to transparently represent the cumulative relationships between larger q sentence fragments and the words within would be desirable.

In order to preserve all of the information in an author-written text, text presentation, by definition, must systematically present the sequence of words written by the author in an unambiguous structure. It is also desirable that the additional information to be depicted in enhanced text presentation for improved reading would be incorporated into the requisite processes of text presentation itself, placing characters against a contrasting background at various locations in space and time), rather than introducing additional symbols that require additional learning, reading and interpretation by the reader.

SUMMARY OF THE INVENTION According to one aspect of the present invention there is provided a method for enhancing text presentation from a machine readable natural language text based on reader specific parameters including at least the viewing field dimensions comprising: a. parsing said text into punctuation and parts of speech for extracting text specific attributes; b. storing said text specific attributes in relation to the parts of speech to produce an enriched text; c. applying primary folding rules followed by secondary folding rules to said enriched text, applied in Qorder of a folding rule rank thereby dividing said text

(N

into text segments, said folding rules having at least said punctuation attributes and parts of speech attributes as inputs and visual attributes as outputs, said visual attributes including the displaying of the text segments in new lines; applying secondary folding rules until a limit is Mreached, this limit preferably being the minimum line (1 length; e. applying text segment horizontal displacement rules to said text segments to determine a horizontal displacement for each text segment, said horizontal displacement rules including parts of speech as inputs and visual attributes of horizontal displacement as outputs to produce an enhanced text; and f. displaying said enhanced text by cascading the text segments in lines down and across a display.

According to another aspect of the present invention there is provided a device for enhancing text presentation from a machine readable material language text an enhanced text presentation based on reader specific parameters including at least text viewing field dimensions comprising: parsing means for parsing said text into punctuation and parts of speech for extracting text specific attributes; storing means for storing said text specific attributes in relation to the parts of speech to produce an enriched text; means for applying primary folding rules followed by secondary folding rules to said enriched text, applied in order of a folding rule rank thereby dividing said text into text segments said folding rules having at least said punctuation attributes and parts of speech attributes as inputs and visual attributes as outputs;

I

Smeans for applying secondary folding rules until a limit is reached, this limit preferably being the minimum line length; and Mc, wherein the visual attributes include the displaying of the text segments in new lines; Cc means for applying text segment horizontal displacement rules to said text segments to determine a Mc, horizontal displacement descent angle including attributes of the text in the line above for each text segment, said horizontal displacement rules including parts of speech as Sinputs and visual attributes of horizontal displacement as outputs to produce an enhanced text; and means for displaying said enhanced text by cascading the text segments in lines down and across a display.

BRIEF DESCRIPTION OF THE DRAWINGS FIG. 1 is a high-level flow chart of a method utilised in an embodiment of the invention; FIG. 2 is a table illustrating various attributes of the elements in one sentence; FIG. 3 contains an example of enhanced text including cascade and right descent angles; FIG. 4 presents the sentence of FIG. 3 as enhanced text including cascade and right and left descent angles; FIG. 5 illustrates step 110 in detail; FIG. 6 illustrates word sets establishing a hierarchy of secondary folding points; FIG. 7 illustrates folding point locations in a sentence; FIG. 8 illustrates step 136 in detail; FIG. 9 illustrates the sentence of FIG. 7 presented as enhanced text; cI FIG. 10 illustrates a Presentation Interval Sampling cHistory Map, wherein the time needed to read is a function of sentence difficulty and complexity; and FIG. 11 illustrates the sentence of FIG. 7 showing visual pronunciation of emphasized syllables.

SFIG. 12 is a chart illustrating lanes and slots used in CI calculating horizontal displacement; FIG. 13 illustrates a curve suitable for text bending; FIG. 14 illustrates rules for placement of text segments upon the curve of FIG. 13 as a function of the segment head and tail text attributes; FIG. 15 illustrates an intermediate result of a text bending algorithm; FIG. 16 illustrates several bent text segments formed over curves in accordance with FIG. 14 and FIG. FIG. 17 illustrates a computer display having a book on a computer display prior to opening; FIG. 18 illustrates a computer display showing the book of FIG. 17 after opening, exposing the chapters within; FIG. 19 illustrates a computer display having the chapters of FIG. 18 after selection of a chapter, exposing the pages within; FIG. 19A illustrates an alternate embodiment computer display having the chapters of FIG. 18 after selection of a chapter, exposing the pages within; FIG. 20 illustrates a computer display having the pages of FIG. 19 after selection of a page, exposing the paragraphs within; FIG. 21 illustrates a computer display having the paragraphs of FIG. 20 after selection of a sentence, exposing the sentences within; FIG. 22 illustrates a computer display having a cascaded sentence on a page selected in FIG. 21; FIG. 23 illustrates a method for displaying text across multiple display pages; FIG. 24 illustrates another method for displaying text across multiple display pages; FIG. 25 illustrates a method for displaying next which enables reading from front to back, from bottom to top, and which can be made to appear to move forward; FIG. 26 illustrates the text of FIG. 25 after forward movement of the text; and FIG. 27 is a tree diagram of a sentence subdivided for color assignment and prose painting.

-8- DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS FIG. 1 shows a high-level flow-chart of a method utilized in an embodiment of the invention. The steps in FIG. 1 and all such figures may of course be split into additional steps or merged into fewer steps and still Sdescribe the same embodiment.

Cc In step 102, the reading device is started, and passed C- the identification (ID) of the reader and the text file identification to be read. In a preferred embodiment, the C- reader ID is obtained from the operating system. In another embodiment the reader is prompted for identification. In one embodiment, there is but one reader of the device assumed.

Given the reader ID, reader specified parameters are retrieved in step 104. Parameters are discussed in detail below. Given the text ID, the text file and text specific parameters are retrieved in step 104. In a preferred embodiment, the parameters have default values, and do not require reader entry. In a most preferred embodiment, entry of reader preferred specific parameters is allowed, values accepted, and the parameter values stored as in step 108.

In one embodiment, the user is allowed to pull down menus containing dialog boxes, view default parameters, and modify them using standard user interfaces including numerical entry, text entry, slide bars, and buttons. In a preferred embodiment, given suitable access permission, modification of global parameters applying to all readers of the text is also allowed. The reader is allowed to have parameters set for only the current text, all texts of the same type as the current text, or all texts read by this reader.

Reader specific parameters include: Vilwing field dimensions; color palette for background and tekt; word set C-i inputs for rules; minimum phrase length; maximum line length; Sminimum and maximum text segment weight; descent angles; horizontal displacement rule inputs; horizontal justification preferences; inter-phrase gp length; cadvancement rate; inter-sentence and inter paragraph interval proportions; Tagging definitions; animation -pec(ifications; Sdefinitions of parameters to derive content dependent presentation intervals; definitions of par4meters and i weighting to derive phrase/text matter, weight density, complexity, and content; and identities of special event scripts.

Viewing Field Dimensions Viewing field dimensions can specify the size of the window for viewing text in the two X and Y dimeisions and a third dimension, time. This field can include the entire screen or a smaller portion of the scree n. The time dimension can include the time over which tite dependent animations such as blinking occur.

Color Palette Color palette for background and text allows specification of preferred background and text color that varies with the text. In a preferred embodiment, the hue and brightness of the background color vary with bcth sentence position within a paragraph, and paragraph position within a chapter. In a most preferred embodiment, thr brightness is set at one level for the first sentence in a Paragraph and a second level for the last sentence in the parkgraph, with che brightness of each sentence in between progressing from O the first level to the second level. In a preferred embodiment, the background hue is set at a first value for the first paragraph in a chapter and a second value for the last paragraph in a chapter, with the hue of each paragraph progressing from the first value to the second value. In a most preferred embodiment, the first hue is green and the second hue is violet.

In preferred embodiments, text color can vary with a word's categorical and continuous attributes. A categorical C' attribute is a function of the category a word fits into.

In a most preferred embodiment, categories include parts of speech e.g. verb, and technical words. Continuous attributes include those that can be measured such as educational level e.g. twelfth grade, number of syllables, and word length.

In a preferred embodiment, different parts of speech have different text colors. In a most preferred embodiment, text color is varied with background color to provide optimal, reader-preferred contrast.

Word sets can be used to specify some parts of speech.

For example, is a pronoun, "had" is a verb, is an article, and "farm" could be a noun, verb, or adjective.

The parts of speech need only be likely parts of speech as with the word "farm", which could be a noun, verb, or adjective. An example of the parts of speech in a single sentence is shown in FIG. 2. The multiple possible parts of speech column 150 is shown as is the disambiguated part .of speech column 152.

Word sets are input in preferred embodiments utilizing two methods. The first method utilizes a structured text format such as Standard Generalized Mark-Up Language (SGML), -11allowing for import and export of large word sets and O dictionaries from diverse sources. Standard SGML parsers are then utilized to store the word sets in a suitable database.

SThe second method allows interactive editing of word sets c 5 using a hierarchy of pull menus and dialog boxes. In a preferred embodiment, the reader specified word sets are Cc stored in a separate database from the dictionary or glossary word sets, to allow for separate storage, editing and faster C-i retrieval.

C-I Minimum Text Secment Lenth Maximum Line Length1 A preferred embodiment includes a reader specified minimum text segment length. A text segment is a piece of text that is presented on one line on a display. The minimum text segment length, which can be denominated in words, is the minimum the reader wishes to see on a separate line. A maximum line length is the maximum length the reader wishes to see on one line. Lengths can be measured in number of characters, words, syllables, educational level, or any sum of products of these. In one embodiment, the minimum line length is ten characters and the maximum line length is thirty five characters.

Phrase Weight Phrase weight is a derived attribute of a phrase (text segment or potential text segment) giving some measure of the amount of material in a phrase. In one embodiment, the phrase weight is simply the number of words in a phrase. In preferred embodiment, phrase weight includes phrase density and phrase complexity. Phrase density can include the number of technical words or number of words exceeding a certain -12grade level. Phrase complexity can include the number of r- O spelling similarities between words in a phrase, number of c ambiguous words, and total weight of reader weight-specified words.

Minimum and Maximum Text Segment Weight SA preferred embodiment includes a reader specified C- minimum and maximum text segment weight. The text segment weight, is the quantity of text the reader wishes to see on a separate line. In preferred embodiments, the text segment C1 weight is the sum of phrase weights within the text segment.

Horizontal Justification Rules Horizontal justification rules specify the horizontal justification of a line of text relative to the line above.

Justification can include the justification type for the line or phrase being positioned, i.e. right, left, or center justification. Justification can also include the text portion of the line above from which the justification is measured, i.e. the entire line of text versus or one phrase, the point of that portion used, e.g. left-most, right-most, or center. Horizontal justification in one embodiment is simply measured within the line being positioned rather than relative to the line above.

In a preferred embodiment, the first phrase on a line is center justified, measured from the center of the last phrase in the line immediately above. In another embodiment, the entire line of text is center justified below the center of the line of text above. In yet another embodiment, the text segment "center of gravity", calculated using the -13ditticulty of each word, is used as the text segment center O for justification purposes, DeS2cent Angle A descent angle is the amount of horizontal displacement for each new line, modifying the horizontal position called for by the horizontal 'justification rules alone. By definition, each text seghent is presented on a new line.

Cq In a preferred embodiment, the descent angle is specified in units of characters. Tlhe descent angle and horizontal justification at least partially determine the "'ext cascade" down and across the scren in preferred embodiments. An example of cascading text is illustrated in FIGS. 3 and 4.

A descent angle may be zer, meaning that, without more, the text segment horizontal position is determined by the horizontal justification -:ules alone. A descent angle can be left, where the line be-ow is to be shifted left relative to the line above, or right, where the text shifts right.

FIG. 3 illustrates only right descent angles.

In one embodiment, the decent angle in a constant for each new line. In a preferred embodiment, the descent angle is a function of the text segment weight of the line above.

In another preferred embqdiment, horizontal justification rules call for center justification below the center of each line immediately above, and the descent angle is calculated to present a substantially straight path, when all text lines are presented, from center- of line to center of line, from upper left to lower right on the display surface.

In a preferred embodiment, the inputs to descent angle rules include attributes of the text in the line above. In one preferred embodiment, inputs include the reason for -14tolding the line above, i.e. primary folding point, secondary folding point, or collapse rule. In a preferred embodiment, Cl a more positive descent angle is called for when the line immediately above folded due to a primary folding point than M 5 a secondary folding point. In another preferred embodiment, the inputs include the text segment weight of the current Sline and the line above.

It is recognized that the horizontal justification rule could call for left justification and measuring horizontal displacement from the left margin, as well as a zero descent angle, combing to result in left justified text on each line.

It is also recognized that horizontal text positioning can be accomplished in numerous equivalent ways to the above example. In particular, calculations of text position can be accomplished by first justifying then shifting, or first shifting then justifying with equivalent results.

Gap Lenqth In one embodiment, gaps are associated with folding points whose locations have been determined, but because of other rules, remain on the same line and do not cause folding. A gap of zero or more spaces is added after a folding point where that folding point has failed to cause new line creation. In a preferred embodiment, the gap length is a reader determined parameter, where a gap length of zero results in a no gaps being created. Gaps allow a visual cue as to the existence of phrases even where the phrases have not caused new line formation.

Advancement Rates Aavancement rates specify display duration times and the time intervals between the presentation of one grouping of CI text and the next. In a preferred embodiment, one sentence is presented on one screen. In a preferred embodiment, the C 5 display duration rate can be a function of the text's appropriately weighted number of words, educational level, M pronunciation time, number of phrases, number of syllables, or phrase weight. Time intervals can include the interval between sentences and paragraphs. In a preferred embodiment, the time interval between sentences and paragraphs is CI different. In this way, the constant arrival rate of text in current systems can be replaced with a rate dependent on where the text lies and what the content of the text is.

Stations Stations are display surface locations at which text is displayed. Stations may be identified windows within which text is presented or points at which text presentation begins. A preferred embodiment includes an active text station, where text being read is presented. Another preferred embodiment includes a pre-reading station displaying about to be read text and a post-reading station displaying already read text. In one embodiment, the total text presented at all stations on the viewing surface is controlled by the expected reading or pronunciation time.

In one embodiment, only about one minute of material is allowed on the screen at any one time.

Animation Animation is the time dependent presentation of text.

Examples of animation include blinking, dissolving, and -16swiping, that is, the brightening of successive text portions S at a specified rate. In a preferred embodiment, a modified scroll effect is accomplished by stacking about to be read text in flat rows at a pre-reading station at the top of the screen, stacking previously read rows at a post-reading station at the bottom of the screen, and cascading the active sentence at an active station across the middle of the M screen. In one embodiment, pre-reading and post reading text and/or background colors differ from that of the currently being read text.

Taqainq Tagging includes designating whether a sentence will be tagged with a definition or a drawing related to the word.

In a preferred embodiment, a word set specifies which words will be tagged with a definition. When a tagged word is selected by a reader, a definition, drawing or other electronic illustration of the word can be displayed. In a preferred embodiment, a tagged word, when selected, displays a pop-up window containing the definition or drawing.

Preferred methods of selection include using a mouse and right shift button.

Reading Event A reading event includes the appearance and disappearance of a sentence and any additional time before the appearance of the next sentence. A reading event contains a series of reading moments. A reading moment is a freeze frame appearance of text to the eyes of the reader trying to comprehend the text. To many readers, the reading moment will find the eye centered on a phrase, the phrase alone on a line, the reader comprehending the single phrase.

-17- Readinq Rate The presentation interval is the duration of a sentence presentation on the screen. The presentation interval may be reader controlled or automatically determined. Reader control in one embodiment may be via mouse clicks. In another embodiment, reader control may be via eye movement M detection or brain wave change detection. Brain wave detection operates by generating a new sentence request when the brain waves corresponding to reading activity change to brain waves associated with havinc finished reading the C, sentence.

Reading rate is measured in text content per unit of time. Such Text Content Units may be measured in length units such as number of words, syllables, pronunciation time, or any of previously discussed measures of phrase length or weight or any sum of products of these. Text Content Units may also be measured in complexity units such as those previously discussed with respect to phrase complexity or weight or any sum of products of these. The Text Content Units in a preferred embodiment are calculated for a presented sentence and used to compare the actual reading rate in Text Content Units per time with reader controlled presentation intervals, and to set the text presentation interval for automatically controlled text presentation intervals. In this way, the presentation interval can be made content dependent, and be made to track the likely reader preferred presentation rate.

Special Events Special events are time dependent events that occur during the reading event. Special events include a visual -18or audib±e cue giving a time remaining signal. For example, when 75% of the content dependent presentation interval has elapsed, a visual cue would appear on the screen. Special events may be arranged by a special event script. Complex special events include a series of special events.

Visual Pronunciation Visual pronunciation involves staggering in time the color or brightness changes of individual phrases, words, and syllables, though not necessarilN all of these. One embodiment denotes emphasis on longer, more difficult words requiring longer to pronounce by using highlighting. Visual pronunciation is an example of a complex special event.

Pulling Down the Cascade In one embodiment, text is displayed at a pre-reading station for a percentage of the content dependent presentation interval, then presented in a sentence cascade one text segment or one line at a time. Pulling down the cascade is an example of a complex special event.

Non-Linear Text Linear text is text presented and read from beginning to end. A book designed to be read from beginning to end is a common example of linear text. Non-linear text includes the presentation and reading of text in an order other than linear. One example of non-linear text is Hypertext, where certain words are presented in such a way to indicate linkage e.g. the text is angle bracketed or colored blue. Non-linear text presentation can be both reader directed and automatic.

One example of this is automatic presentation of Hypertext -19ina depth first order, following the first Hypertext link, ignoring cycles. Another example is a breadth first C-i presentation, presenting the first level of text, followed by any Hypertext links from the main level, followed by any Hypertext links from the first level, continuing until the lowest depth is reached or the reader intervenes. Yet another example is pruning either depth or breadth first presentation to include only portions having certain keywords of interest.

c-.

Magic Readinq Glass In one embodiment, text presentation may begin by presenting common, unformatted text. When the reader desires to see text presented as enhanced text, the reader designates a portion of text for enhancement by placing an icon such as a "magic reading glass icon" over some part of the sentence.

That sentence and following sentences are presented as enhanced text. This enhanced text presentation continues until the reader deactivates the magic reading glass. This magic reading glass can serve as an alternate point of entry into the enhanced text processor.

Method In FIG. i, step 110, the text is preprocessed. Step 110 is shown in detail in FIG. 5. The text is parsed to identify paragraphs, sentences, words, and punctuation. Paragraphs may be identified by blank lines, paragraph markers, indentation characters, tab characters, or any other suitable characteristic in the text. Sentences may be identified using grammar rules including periods, spacing, capitalization of first words, and abbreviations or the lack tnereor. In a preferred embodiment reading well behaved O text, a period, question mark, or exclamation point, either alone or followed by a period, followed by two spaces or end C- of paragraph, signals the end of a sentence.

In step 124, the sentence is tokenized into words and punctuation. Original author specified emphasis, e.g.

italics or underlining, is preserved in preferred embodiments. A standard lexical scanner such as Lex C' (TRADEMARK) is used in one embodiment, where the end of a word is denoted in the grammar rules by white space or C" punctuation. Another embodiment utilizes a hand written lexical analyzer. One embodiment stores formatting characters such as tabs and indents as punctuation. The location of a word is preferably stored as an attribute of the word, to provide links to, and searching within, the original work.

A preferred embodiment also allows groups of words to be "clamped" together, and be recognized as a group of words.

In one embodiment, such groups of words are recognized by the lexical scanner. In another embodiment, such words are recognized by a preprocessor preceding the lexical scanner to insure recognition as a phrase rather than as merely individual words. Clamped words, for example, "Prince of Whales", would be recognized as a single phrase, and preferably not broken by the preposition into two phrases displayed on two lines.

In step 126, the words are looked up in dictionaries, glossaries and tables to determine word attributes. The text is further processed to determine categorical and continuous attributes. In a preferred embodiment, important categorical attributes include parts of speech, and important continuous -21attributes include word location, education level, 0 pronunciation time, and syllable number, location, sound, and C' vocal emphasis level. Identifying parts of speech with 100% C- accuracy would require extensive programing to determine the real-world context of the text. Such accuracy is not required to practice the invention, as errors are of minor Mconsequence because the reader is a human, not a machine.

SThe possible parts of speech are first determined by looking up the word in a dictionary or glossary. This dictionary or glossary need only have the likely parts of speech for a C' word, not a definition. For example, the word "farm" could be a noun, verb or adjective. A preferred embodiment stores the parts of speech attribute using a bitmap to preserve the multiple possible parts of speech. One embodiment explicitly stores an ambiguity attribute, indicating whether the word still has multiple possible parts of speech. Another embodiment uses the existence of more than a single possible part of speech as an ambiguity indication.

In a preferred embodiment, default parts of speech exist in a dictionary and may be looked up. In a most preferred embodiment, a word set may be added to override or supplement the default set. In another embodiment, technical words are specified by user entered word sets.

FIG. 6 illustrates nine word sets specifying parts of speech arranged in a hierarchy. Such word sets and hierarchies are also used as input to folding rules, described later. A preferred embodiment checks the word sets illustrated in FIG. 6, beginning with Class i, Subclass 1, and ending with Class 3, Subclass 1. Search is terminated once a word or phrase is found. Given the limited vocabulary and static nature of the word sets in FIG. 6, a most -22prererred embodiment utilizes a fast, hand written parser to O search for the words and phrases in FIG. 6. Preferred embodiments include parsers written in C, Perl, compiled c- AWK, AWK and AWK to C, C with regular expression functions, or any suitable language. One parser embodiment utilizes

YACC.

In one embodiment, the dictionary is a commercially Cc available dictionary on electronic media such CD-ROM. The CI standard dictionary is parsed for word attributes such as parts of speech and number of syllables. As word definitions are not needed in many enodiments, storage of numerous words with associated number of syllables and parts of speech is possible. In a most preferred embodiment, the most commonly used and most recently used words are stored in fast access memory such a solid state Random Access Memory (RAM). In embodiments where dictionaries are to be hand crafted, a fast method utilizing hashing, collision detection and buckets is preferred. In embodiments where the word sets are fixed before reading, perfect hashing without buckets is preferred.

In yet another embodiment, the level of pronunciation emphasis is derived as an attribute depending in part on the part of speech. In a most preferred embodiment, pronunciation emphasis is categorized as primary, secondary, and none. In one embodiment, the pronunciation time and actual sound, e.g. as found in a sound file, are also retrieved from the dictionary or glossary and stored as attributes of the word.

In step 128, reader and text specific word sets are searched. In a preferred embodiment, the reader specified word sets are stored in a separate database from the dictionary or glossary word sets, to allow for separate -23storage and faster retrieval. In one embodiment, reader specified word sets are checked before dictionaries, and the CI dictionaries only checked if needed words and attributes are not found in the reader specified word sets. Preferred embodiments utilize a hierarchy of databases to look up words.

r^ In step 130, multiple parts of speech are disanmbiguated.

In one embodiment, a microgrammar is used to determine the Slikely parts of speech. A microgrammar utilizes adjacent or nearby words to more accurately determine the most likely C< part of speech for a word. For example, the word "farm" in the phrase "a farm in would likely be a noun as it is preceded by an article and succeeded by a preposition. As another example, if a word could be either a noun or verb, and the word is preceded by "could", "will", "shall", or then the word is likely a verb. If the word "farm" were preceded by "will", the word is likely a verb. In another embodiment, all disambiguation is done simply by choosing the statistically most likely use of the word. In yet another embodiment, there is no automatic disambiguation, only manual disambiguation using human editing. In a preferred embodiment, an attribute of ambiguity is stored for each word, indicating whether multiple possible parts of speech still exist after disambiguation. In yet another embodiment, an ambiguity attribute is not stored but derived from the existence of multiple possible parts of speech stored for a word. In one embodiment, ambiguity is inferred from the visual display of striped or alternating text colors associated with each part of speech. For example, if verbs are orange and adjectives are yellow, then a possible verb -24oraaacective could have alternating yellow and orange stripes or text characters.

cl In one embodiment of the invention, parts of speech are Cdetermined in part by searching for morphemes (root words) and assigning part of speech attributes based on endings e.g.

-ly, -ing.

In step 132, the attributes determined for the words and Cc phrases are stored, creating an "enriched sentence", which C-i will likely remain unchanged between readers. For example, the educational level and part of speech of a word will cI remain unchanged for different readers, even though they may desire different text segment lengths and presentations rates. In one embodiment, the enriched sentence is stored in persistent storage such as a file. In another embodiment, the enriched text is stored on CD-ROM. In a preferred embodiment, the enriched sentence is implemented as a linked list of nodes, each node having the word and phrase attributes described above including the position of the word in the original text.

In FIG. 5, step 134, primary folding points are determined by applying primary folding point rules. Folding points are text dividing points located between letters. In a preferred embodiment, folding points are classified as primary and secondary. Primary folding points are determined using primary folding rules which determine primary folding point locations based on punctuation marks. FIG. 7 illustrates a primary folding point after the comma following "Africa". Primary folding points divide text into "SuDerphrases". In a preferred embodiment, primary folding points are located at every comma, colon, semi-colon, and left parenthesis, brace, and curly bracket. The folding point iocation can be stored as an attribute in a node in a linked list of nodes forming the enriched sentence.

Secondary folding points are determined applying secondary folding boint rules. In preferred embodiments, secondary folding Pbints and rules are ranked in a hierarchy and secordary foldirig rules accept parts of speech as inputs.

SIn a most preferred embodiment, secondary folding rules m include a s rule inpits attributes of the text content of the text segments and Fhrases being processed. For example, a secondary folding )oint may be called for by a segment of text exceeding a reader preferred maximum text segment weight even though a maximum text segment length has not been reached.

Continuous attributes such as phrase difficulty, density, complexity, power and pronunciation time may be used as inputs to a rule modifying the ranking established by a table such as that )f FIG. 6, using parts of speech alone to determine secondary folding part rankings. For example, a segment of text having a weight greater than 35 percent above the text average vould have a Class rank of 1 assigned regardless of the rank otherwise called for by the table of FIG.6. In one prefrred embodiment, phrase weight or power is used exclusively to determine secondary folding point rankings, rather than solely parts of speech.

In an alternate embodiment, folding rules call for foldine based on the number of characters on the line, and the parts of speech are displayed using colors corresponding to a word's part of speech. The later embodiment may not offer the advantagbs of cascading, but does offer visual display cues based bn text content.

-26- FIG. 6 illustrates a table used in a preferred embodiment to determine secondary folding points. For example, prepositions determine Class 3, Subclass 1, secondary folding points. In FIG. 7, there are class 3 secondary folding points before the prepositions "in" and Secondary folding points divide Super-phrases into "Mini-phrases", as illustrated in FIG. 7. Mini-phrases are related to text segments in that Mini-phrases are often the same as text segments and are often displayed each on a new line. Once Mini-phrases are identified however, rules may C' dictate that more or less than one Mini-phrase appear as a text segment on a line.

Primary folding rules are applied first, followed by secondary folding rules, applied in order of the folding rule rank. An example of secondary folding rule ranking is shown in FIG. 6, established by the ranking of the word sets giving rise to the secondary folding points. Some preferred embodiments use either phrase weight or power to determine secondary folding point rank rather than solely using parts of speech. A most preferred embodiment allows reader entry of a preference for parts of speech or phrase weight/power determination of secondary folding point ranking. Some readers prefer text segmentation based on structure, while others prefer text segmentation based on complexity or estimated time to read a text segment.

In a preferred embodiment, secondary folding rules are applied only until a limit is reached. This limit is often the minimum line length. One method of determining the location of secondary folding points is shown in FIG. 5, step 136, and in detail in FIG. 8, step 136. In one embodiment, where the application of a secondary folding rule to a Super- -27phrase would result in a Mini-phrase length less than the O minimum specified line length, the folding rule is not applied and no further folding rules are applied to that c- Super-phrase. Conversely, when no folding point would otherwise exist in a line exceeding the maximum line length, a collapse rule is applied, forcing the folding of the text into two lines. When all Super-phrases are to have no S further folding rules applied, the folding process is CN' complete.

In FIG. 1, step 111, the parameters, attributes, and C' folding rules can be used as inout to the horizontal displacement rules. The horizontal displacement rules determine the horizontal location of the text segment. In a preferred embodiment, horizontal displacement rules include both horizontal justification rules and descent angle rules.

Horizontal displacement in this embodiment is the sum of the results from the horizontal justification rule and the descent angle rule. In an easy to implement embodiment, the horizontal displacement rule is simply the descent angle as applied to the center justified text segment. Such an embodiment does not utilize the folding rule terminating the preceding text segment as input and provides minimum eye movement while reading the sentence cascade. Another embodiment adds left descent for preceding Class 1 folding points, and right descent for preceding Class 3 folding points. A preferred embodiment allows reader specified additional right or left displacement for folding points, including reader entered values for primary folding points, and each class and subclass of secondary folding points. One embodiment stores the added displacement in a table in units -28o£ characters. With the horizontal displacement determined, 0 presenting the text remains.

SIn the example of FIG. 9, Super-phrase 1, "I had a farm in Africa" is divided from Super-phrase 2, "at the foot of S 5 the Ngong Hills.", by the primary folding point after the comma. Super-phrase 2 is divided into two Mini-phrases by the secondary folding point before the preposition In the embodiment illustrated in FIG. 9, two miniphrases are Spresented as the same text segment as the mini-phrase "in l 10 Africa", is below che three word minimum length specified by the reader. The first Mini-phrase, "at the foot", in a new Cl line is center justified below the text segment above and right shifted by a right descent angle. The last Mini-phrase of "of the Ngong Hills", is left shifted because of a horizontal displacement rule calling for left shifting when the preceding line terminated with a secondary rather than primary folding point. Thus, in the FIG. 9 example, the total horizontal displacement is determined by a combination of the decent angle and the justification of the line.

Paired punctuation marks including parentheses, braces, brackets, and quotation marks, serve as punctuation marks which determine primary folding point locations in a preferred embodiment. In one embodiment, such paired punctuation is included as an input to the horizontal displacement rules, including horizontal justification and descent angle rules. For example, a long multi-phrase text segment contained within parentheses might have reduced horizontal shifts and reduced vertical displacement, i.e., less line to line spacing than the other text in the active display area of the display surface.

-29in step 112, the enhanced text is created. In this step, the codes needed to create a properly displayed text C-i segment are created. For example, where the reader specifications require coloring technical words red, and the Mf, 5 enriched text indicates a word is a technical word, an escape sequence may be created that will be interpreted by the 'c display step as requiring red text. Similar coding may be required for animation. The enhanced text may be stored at this point for later display.

c-i In step 113, the enhanced text is presented on the display device, one text segment per newly formed line. The enhanced text can also include the animation, background color, text color, tagging, and presentation rates discussed above. In a preferred embodiment, the background color is presented as a function of the sentence and paragraph positions.

In another embodiment, illustrated in FIG. 11, some text is initially presented in one color or brightness for a time period, followed by presentation in a second color or brightness. In FIG. 11, the sentence is presented on three newly formed lines indicated by arrows 200, 202, and 203.

Within those lines, the text is presented all on the same line, with the words of FIG. 11 shown on different lines to illustrate the temporal change of initial text color/highlight 206 to text color/highlight 207, and initial text color/highlight 208 to 209. The "Af" text at 206 is the emphasized syllable in "Africa", and is initially shown in color/highlight 206 for this reason. The "gong" text is also the emphasized syllable in "Ngong', and has initial color/highlight 208, followed by color/highlight 209.

The preferred embodiment depicted in FIG. 1 allows the 8 enriched text from step 132 and enhanced text from step 113 to be edited. In a preferred embodiment, a word pointing device such as a mouse is used to select a portion of text to be edited. The enriched sentence corresponding to that portion of enhanced text is internally selected, and the attributes made available for editing. This may be (f implemented using pull-down menus allowing each attribute of CI the enriched text and enhanced text to be edited. Where edits to the enriched text could effect presentation, the text is preprocessed according to step 110 again. In the edit session of step 114, reader annotations, either hidden or immediately visible, are accepted and can be stored in the enriched and enhanced sentences. Such annotations perform the same function as hand written notes in a book. The features editable include parts of speech, definition, color, text, folding points, and horizontal displacement.

The edited attributes and display of this embodiment may also be stored, in step 116, preserving the changes. In one embodiment, only the enhanced sentences are stored. In a preferred embodiment, both the enriched and enhanced text are stored. In preferred embodiments, some edits are recorded either as reader specific or global, for presentation to all readers of the text. Manual editing of the enhanced text is especially useful where the same text will be viewed repeatedly by others, as with an electronic book.

In a variation of the embodiment of FIG. 1, steps 113, 114, and 116 are omitted, with no human intervention and no immediate display. In this embodiment, enriched and enhanced text is created and stored for future display. The enhanced text can be stored in standard word processing format such -31as Microsoft Word (TRADEMARK) or Corel Word Perfect (TRADEMARK) binaries. In this embodiment, the presentation software can be simple, small, and fast relative to the software required to look up words and analyze text. Such a presentation is a preferred embodiment for the mass distribution of enhanced text for reading as "electronic books." SIn a related variation of the embodiment of FIG. 1, C' enhanced text itself is retrieved in step 106, with no preprocessing required and no edits allowed. Such a presentation is also a preferred embodiment for the mass distribution of enhanced text for reading as "electronic books" Referring to FIG. 1, step 113, the presentation of each screen of enhanced text may be manually triggered as by a mouse click. In a preferred embodiment, the rate of presentation is controlled by reader specified parameters including display time for the text, and inter-sentence and inter-paragraph arrival times. In a most preferred embodiment, the Text Content is measured, and the text presentation interval is dependent on this Text Content. In a preferred embodiment, the pronunciation time of the text is used as a measure of Text Content to determine presentation interval. In another embodiment, the phrase weights are used to measure Text Content and to determine presentation interval.

A preferred embodiment allows readers to extend the presentation time for a sentence, as well as speed up and slow down the presentation rate. This rate can be recorded, as can the sentence length and sentence difficulty corresponding to a particular presentation interval. FIG.

-32l0iliustrates an example 3-D plot, a "Presentation interval Sampling History Map", depicting the time needed to read a sentence as a function of two attributes, sentence difficulty and sentence length. The time needed to read can be measured by reader preferences or by previously mentioned eye movement tracking or brain wave activity. The time can then be correlated against two attributes as in FIG. 10, or any number of attributes. The presentation interval in a c' preferred embodiment is continuously adjusted to match the predicted required reading time.

In yet another embodiment, most appropriate where reading rate is closely measured, the inputs to folding rules are varied and the resulting reading rate tracked. In an embodiment suitable for creating an optimal mass market reading product, inputs to folding rules are varied and reading rates recorded for a sample population. The inputs are varied for optimal reading speed and comprehension. For example, the importance accorded by secondary folding rules to parts of speech relative to phrase weight is varied for optimal reading speed.

In another embodiment, reader manual editing of initial sentence display is analyzed for relative contribution of sentence structure, parts of speech, length, and complexity.

After such an initial "tuning" or "training" period, similar weightings are used in folding rules for automatic text enhancement. Subsequent edits may be used to further refine the folding rules.

In step 118, text presentation stops when there is no more text or a reader request to stop. Steps 120 provides for clean-up including final enriched and enhanced text -33storage, as well as historical storage about the reading session.

EXAMPLE OF METHOD Overview of Example Method The present invention includes a method for text folding and displacement that applicant believes is an improvement over the method previously described under the heading "method" above.

C' One embodiment of the present invention classifies punctuation marks as either "serial" or "bracketing." Serial o punctuation marks include colon, semi-colon, comma, a single double dash and the ellipse. Bracketing or nesting punctuation marks include quotation marks, parenthetical marks, brackets (as in and and a pair of double dashes.

The present invention works best when bracketed segments do not start inside of one bracketed segment and end within another. This is normally the case.

Segments within bracketing punctuation marks can be serially divided with serial punctuation marks, be further bracketed with inner bracketing punctuation marks, or both.

There is one case where two commas in a segment produce a kind of bracketing: the middle segment of the three new segments formed is "bracketed" by the other two segments.

This is common and can distinguish a relative clause from a non-restrictive one. It is also the way "for instance" and other such phrases are imbedded in larger phrases. Some bracketed text e.g. text within quotation marks, can span more than one sentence.

In the method described below, bracketed segments are identified. The text outside of the bracketed segment is examined for serial punctuation marks. These are used for -34tolding tirst. Then non-bracketed text and bracketed text are separated with folding steps that put each type of text on separate rows. Bracketed text is placed on its own row, with the bracketing punctuation marks at the very beginning and end of each segment. This cycle is repeated as needed.

A segment is ready to leave the algorithm portion dealing with "punctuation mark folding" and begin with "word-only M folding" (word based folding), when it contains no punctuation marks other than those found at the beginning and end. Steps 1 through 15 in the algorithm below deal with punctuation mark folding.

Word-only folding begins with step 16 and will fold each segment to fit between maximum and minimum targets. Folding points in the text will be specified by "hinge words." The hinge words can be specified by a word list. In a preferred embodiment, words or phrases in the word list are associated with a classification code or code word that includes information about the word. In the example illustrated below, the classification include a letter prefix and a number rank or level number. Letter prefixes include "b" (fold before this word, not after), (reserved for future use, fold after this word) (definitive subjective pronoun list) (pre-determiner list) (postdeterminer list). Examples of words and associated codes are contained in Appendix 1.

In the example below, the level numbers range from 0 to 99. The lower level numbers have higher rank and are handled first by the algorithm. The variable is used to denote this value in the steps described below. The 10 39 range includes the before hinge words. The 40 46 range includes verbs, including 40 (modal), 41 (have, auxiliary The term refers to the hinge word level number. In the example below, the hinge word level ranges from 1 through 99.

Referring now to FIG. 12, a table 320 includes a header row 322 listing the spaces from left margin for each column below. A header column 324 contains a list of hinge word levels. The rows are divided into a series of lanes 328 which are subdivided into 4 slots each. A series of Xs, for CI example at 330, denote the location of the start of a text segment, given the hinge word level and lane.

The Xs in the table shows the slot and lane into which a segment's first character will be placed as a function of the hinge word used to make the fold that produced the segment. Each hinge word level, blO, b20, can be placed in any of the Lanes from Lane 2 on up. The lane used is determined by which generation of the sentence folding did the fold for the segment occur in. Then, within that lane lane 3 for a third-generation fold), a slot is determined by the hinge-word level, as shown in the table.

In this way, a third-generation fold made with a b20 level word will be placed in Lane 3 slot B, and a third-generation fold made with a b70 word will be placed in Lane 3 slot D.

The levels shown in the table at level b85 will have folding occur BOTH before AND after the word--in effect placing the word by itself on a its own line. The instructions on the table state where the b85 word should be placed, and where the segment following the b85 word should be placed. The criteria for a b85 word is not on the Hinge Word list, but instead has been defined as any word exceeding 12 characters. As its position on the Hinge Word table implies, b85 words are only used for folding if there are no -36alternatives higher on the list in the sement being interrogated for folding. It is otherwise not explicitly described in the algorithm.

Method The Recursive algorithm is begun by settirl X=1. Thus initially, the first lane will be used and the algorithm on generation is 1.

Punctuation Folds c 1. Other than punctuation marks (pms) that may b1 present at the very beginning or very end of a sentence/segment, are there any other pm's in the sentence/segment? YQ!-go to step 2. No-go to step 16.

2. Are there bracketing punctuation mark:s in the sentence/segment? Yes-go to step 3. No-go to Step 4.

3. Are there serial punctuation marks "outside of" the bracketed text in this sentence/segment? If a sentence begins without a quotation mark, but has the second member of a pair of quotation marks later in the sentenc the quotation started in a previous sentence and is ending in this one], then the segment from the beginning of the sentence to the space following the second member of the pair is also a "bracketed" segment; a similar rule applies if a sentence contains the first but not the second Qf quotation mark pair.) Yes-go to step 4. No-go to step 9.

4. Are there colon/semi-colons in the non-bracketcd text from step 3? Yes-go to step 5. No-go to step 6.

-37- Fold the sentence/segment after each colon/semi-colon and place the text that follows each colon/semi-colon (which may c-s contain bracketed text) onto a new row and indent each new row at position Lane X, Slot A; then go to step 6.

6. Are there commas in the non-bracketed text? Yes-go to Mcf step 7. No-go to step 8.

7. Fold the sentence/segment after each comma and place the text that follows each comma (which may contain bracketed text) onto a new row; indent the new row of text that comes after the first comma at Lane X, Slot D; indent the new row of text that comes after the second comma at Lane X, Slot B: indent any additional rows of text that follow odd-numbered commas at Lane X, slot D; indent any additional rows of text that follow even-numbered commas at Lane X, Slot B. Then go to step 8. Are there any single "double dash" or ellipse punctuation marks in the non-bracketed text? Yes-then fold the sentence/segment immediately before each single "double dash" or ellipse: place each new segment that contains the single "double dash" or ellipse and the text distal to it (which may also contain bracketed text) onto a new row; indent each new row at Lane X, slot C. Then go to step 9. Is there any text between the very beginning of the sentence/segment (which itself may be a bracketing punctuation mark) and the first "intra-sentence/segment" -38- DracKeting punctuation mark? Yes-go to step 10. No-then keep O the first bracketing punctuation mark at the same position (do not fold or indent at this bracketing punctuation mark; c-s for example a parenthetical mark could lead and conclude a sentence or segment but be followed immediately by a quotation mark-these would then be left together according to this step); and go to step 11.

C-I 10. Fold the sentence/segment immediately before the first bracketing punctuation mark (which, per step 9, does have text proximal to it); place this first "intra-sentence/segment" bracketing punctuation mark and all of the text distal to it onto a new row; indent this new row at Lane X, Slot D. Then go to step 11.

11. Is there a matching bracketing punctuation mark in the sentence/segment, the second member of the bracketing pair)? Yes-go to step 12. No-then the second member of this bracketing pair of punctuation marks is in a later sentence; go to step 12. Is there any text distal to the second member (as per step 11) of the bracketing pair of punctuation marks? Yes-then go to step 13. No-then the second member of the bracketing pair of punctuation marks is the end of this sentence/segment; go to step 13. Fold the sentence/ segment immediately after this second member of the bracketing pair; indent the new row of text at position Lane X, Slot B; then go to step 14.

-39- 14. Is there another bracketing punctuation mark in the text distal to the second member of the previous bracketing pair? CN Yes-go back to step 10 and repeat steps 10-14 for this next Z bracketed segment distal to (not nested in) the previous one.

No-go to step 15. Take each of the new segments formed in steps 1 through S14, add 1 to X, and go back to step 1. Repeat folding and indenting instructions using Lane X 1. Repeat steps 1 through 15 until all segments have punctuation marks located CI only at the beginning, or end, or both of each segment. In this way all segments are eventually sent to step 16 for "word-only" folds.

Word Only Folds 16. Word-only folds begin with this step. For any value of X at step 1 where the answer is "no" and sends the segment to step 16, use the greater of: 2 or X for a Lane to begin the word-only folds. For example: If a sentence going through the cycle for the first time, (where had no punctuation marks between its beginning and ending, and the answer to Step 1 was then begin word-only folds not with X (which equals 1) but with 2 (which is greater), i.e., in Lane 2. However, if two cycles through steps 1-15 were required to obtain a segment ready for word-only folds, then, at the beginning of the third cycle, where X now equals 3, and the answer to step one is so at step 16 word-only folds begin with Lane X=3) This will place word-only folds one lane to the right of the last lane used for punctuation mark folds. After determining which lane a sentence/segment's ritst generation word-only fold will be made indents in, go 0 to step 17.

(N

17. Does the sentence/segment exceed the Maxium Row Value number of words or characters in the segrment), or does it extend beyond the available space to the rig[ht No--stop.

n Yes--go to step 18.

Cs 18. Are there words in the segment that can be found in the 0 10 Hinge Word list at level level bl0 when n Cl No-go to 18 A. Yes-go to step 19.

18A. Repeat the question for Hinge Word list level [n+l] level b20, then b30, etc.); if there are no words in the segment that can be found at any level in the Hinge Word List, up to n 95, then go to step 29. NB Wtrds labelled 99, with prefixes w, x, y, z are used in the verb loop, but are not directly used as hinge words for folding 19. Is there more than one word in the segment at Hinge Word level Yes-use the first word that appears in the sequence of the segment, and go to step 20. 1 qo-go to step 21.

20. Does the segment formed with the use of the word in step 19 exceed the Minimum row value? No--theh repeat the question using the next word in the sequence of the segment that is in the same Hinge Word level if none of the words at Hinge Level produce a segment that exceeds the Minimum row value then go back to step 18 ard repeat for Yes--go to step 21.

-41not required to practice the invention, but provides additional folding processing for verbs as described below.

(N

It is desirable for a text folding process to avoid breaking a string of related verbs that should remain on one segment; and to have folding occur at verbs when a discrete verb phrase O is identified; as such, this does not require that a verb M necessarily gets put on the beginning of a new row when identified, identification of an appropriate phrase boundary involving one or more verbs might result in a new row being formed after the verb (in English).

It is also desirable for a text folding process: to avoid separating the word "to" from the infinitive form (and only the infinitive form) of a verb that follows it; to similarly link, if they precede a verb, the words "and" and "or" to verbs identified for folding (all verb forms, not just the infinitive forms); to process specific verb+preposition (or verb+adverb) combinations as a single word so that subsequent folding would avoid separating these prepositions or adverbs from the combination they belong to "they broke_up the fight" v. "he walked up the hill"); and to similarly process the "not" forms of auxiliary verbs as a single word so that they may be recognized as a single word at positions relative to other verbs "could not go" should be processed in the same way as "could It would also be desirable for the pronoun contractions of auxiliary verbs to be labeled as verbs, "you'll", "we've", "you're") Verbs are complex. They can be auxiliary, modal, or ordinary in function. They may be regular or irregular in conjugation. They may be transitive, intransitive, or either -42t in their ability to take an object. They may be ambiguous 0 in Whether they are really verbs and not other parts of

(N

speec:h. They can be combined with specific prepositions or adve-bs to mean something different from the verb alone.

0 5 They have several forms which may or not be combined with auxiliary verbs to create various tenses and which agree with Sthe T)erson and case of the subject.

n At the present time, it is not practical to direct cascade formatting using all of these variables in all their combinations. However, it is possible to classify verbs into four major function groups (modal, have, be, and ordinary); to Sub-classify the ordinary verbs into unambiguous v.

ambiguous and transitive v. strictly intransitive; and, for each of these groups, to identify up to five different forms (infinitive, past participle, gerund, other, and combined infiritive-past participle).

Definitions for Verb Folding Method The (lower case x) denotes the index or offset into the sentence or sentence fragment at a particular word. The x is incremented/decremented in words, not characters.

The term refers, in the verb loop algorithm, to the word in text position relative to the word being investigated. The variable does not refer to hinge word level as in the previous algorithm Introduction For each of these function-form verb categories there are b)asic operations that will determine if folding should occur- at a verb, x, (that was otherwise identified for folding by MIN and MAX criteria). These operations look at -43the word immediately before the verb in question and determine if it belongs to a set of words specifically defined for that function-form group. Based on this, then c-s the potential fold is either skipped, (and the next verb in the segment that is a candidate for folding at that hinge level is considered) or the potential fold is kept open to consideration pending the results of operations.

c The content of these sets for the operation will C< include: words that would disambiguate a verb-noun into a noun; and words that are appropriate helping verbs for the verb in question.

In a subsequent operation, a potential fold before x will be deferred if x+1 is a verb. The x+l1 verb is examined to see if the word before it (namely x) is on its list; if x is not on the list for x+l, then the fold is performed before x+1 rather than before x (because x+1 is a verb belonging to a separate verb phrase). If x is on the list for the x+l verb, x is a helping verb for x+l) then folding is NOT performed at x+1, and the word after x+1, is examined. If the word after x+1 is not a verb, then the original deferred fold to occur before x is now performed.

This process exploits the directionality of the folding process: the existing folding procedure moves the inquiry of each potential verb from first to last in the sentence sequence. If two verbs are adjacent to each other, then the x-1 operation performed for the second verb in the adjacent pair of verbs (an operation which looks at the first verb of the pair to see if it is on a list specific for the form-group of the second verb) will determine if a fold should occur between the first and second verbs.

-44- An x-1 operation is required for each verb, even if it O is the first word beyond the MINIMUM ROW VALUE. The reason c for this is that it is possible the helping verbs of a verb, Sx, in question were passed over in previous potential folds because they fell within the MINIMUM ROW VALUE. However, by looking back toward the beginning of the segment, the x-l E3 operation will detect whether a helping verb for x did indeed precede it (even though x-1 is within the MINIMUM ROW VALUE.

C-i In this case, the fold at x is not performed.

Moreover, even if x is followed by another verb in this c-i case (or any case where x-1 is a helping verb for x) this verb following x (at the x+l position), will not be actively looked for once the operation determines that x is preceded by a helping verb for x. Instead, the folding process-- which is still being driven by the fact that the segment remains >MAXIMUJM ROW VALUE (because the potential fold at x was not made)-- will go to the next available verb in the segment; the process will then find the verb that had just been x+l, and analyze it as a new x (called x* here).

The x*-l operation will now look back at the former x and determine whether x*-l is an appropriate helping verb for x*.

The x-1 operation for level 40 verbs (modal verbs, see below) and the x-1 and x+l operations for level 45/46 verbs (ambiguous verbs that may be either transitive [45] or strictly intransitive are slightly different than those used for all of the other verbs. In the case of modal verbs, the algorithm was simplified because only an infinitive form of a modal verb (and there is only one modal verb that comes in infinitive form, can be preceded by another modal verb "could In the case of 45/46 level verbs, the typical x-1 operation is t initially performed but is followed by specialized x-l and

S

x+l questions. These will attempt to disambiguate x, and will include verb-verb interfaces on either side of x.

An inquiry about a potential fold at a verb x on a lower level 40), might end up with a fold being made at the space preceding a higher level verb 46) that follows it, if one or more verbs sequentially follow x. Once a fold M has been made, then the >MAX question is asked again. If I more folding is needed, then the process looks again for lower level verbs first. This filters out the more elementary verb phrases first those beginning with a modal verb, such as "I could have bought some milk"); followed by verbs tenses involving helping verbs "We have been going there for several years"); followed by unambiguous verbs; then ambiguous verbs. Hopefully, the segments will be so frequently brought down to <Max using the other verb levels, that the steps for ambiguous verbs will not be used very often. By the time level 45/46 verbs are used to initiate an inquiry for folding, the higher level verbs will have all been filtered out. This permits some useful assumptions involving verb-verb interfaces that occur between two different 45/46 verbs.

There are four reference lists that are used in addition to verb-verb comparisons: a definite subjective pronoun list (which will make an immediately subsequent ambiguous verb a definite verb, denoted by x99 in the word list); a pre-determiner list (which will make a subsequent ambiguous verb a definite non-verb, denoted by y99 in the word list); a post-determiner list (which will make a preceding ambiguous verb an appropriate phrase boundary, denoted by z99 in the word list); and a gerund helper verb list (which would -46prevent rolding in front bf the gerund verb form), denoted by w99 in the word list.

The above algorithms for Lane and Slot assignments of newly formed segments can still be used. The Lane, X, into which a newly formed segment is put is derived from the progressive folding history of a segments generational S"ancestors". The Slot within each Lane, into which g a newly formed segment i placed, has, up to now been a C function of the hinge worj level of the word. Because all of these words in the VERB LOOP are by definition verbs, the Slot assignment would be the same, Slot b.

However, to try to corvey the differences among the verb loop folds, the Slot assigrments could be modified. Any fold made in the space before the inquiry verb, x, could be in Slot c. Any fold made in t he space before a subsequent verb after x whe h x was followed by one or more verbs, could be in slot b :IF the verb at x+n was unambiguous (43 or 44). Any other fo:Lds in spaces after x could be in slot d.

Proc.ss Overview A lexicon of verbs is built. While this can be done manually, the rules presented here make it possible to automate this with an electronic dictionary. Each verb is labeled according to its function and form.

In the current hinge word list terminology, verbs would be at a "40s level". The four function groups will therefore be: modal 40; have verbs=41; be verbs=42; and ordinary verbs=43/44 and 45/46. Gcoups 43/44 are unambiguous verbs i.e. they are always verbs. Groups 45/46 are ambiguous verbs -47i.e. they are not certain to be verbs. The odd number in each of these pairs represents transitive verbs; the even number represents strictly intransitive verbs. The verbs at either a 43 or 44 level will be looked for simultaneously in 0 5 the search for a new candidate verb for folding; once a verb at either 43 or 44 is found, the x-1 operations are different Sfor 43 versus 44. Similarly, the 45 and 46 level verbs will Sbe initially looked for simultaneously, but handled differently once identified for potential folding.

"Open" verbs, namely those not preceded by a verb, will be identified for potential folding as a function of the number of the function group, as is done for the other hinge words on the hinge word list. A string of sequential verbs (or potential verbs) could have several function word levels in it: "He should_not (40) have (41) been (42) bringing (43) work (43) home." The steps outlined later will allow the then operations to continue through the string independent of the hinge levels of the words in the string.

A preferred method includes five different kinds of verb prefix classifications, based on the form of each verb. I have therefore called each of these five groups: c (for infinitive form, "to walk", but also "I walk", etc.); d (for past participle form, walked); e (for other forms that do not take any helping verb, including: those modal verbs which have only this form, the third person singular form of regular verbs, "he walks", but also "he goes"); f (for forms that are both infinitive and past participle, as in some irregular verbs such as "to cut" and "have cut"); and, g (for gerund form, walking).

-48- Examples of how specific verbs would be labeled with this classification are included below. Further examples are contained in Appendix 1.

c, Do c 40, Did e 40, Does e 40, Could e S 5 Might e 40,Have c 41, Had d 41, Having g 41, Has e 41, Be c 42, Been d 42, Being g 42, Am e 42, m1 Is e 42, Were e 42, SWalk c 45, Walked d 45, Walking g 45, Walks e Cut f 45, Cuts e 45, Done d 45, Doing g Bring c 43, Brought d 45, Bringing g 45, Go c Gone d 45, Going g 46, Goes e 44, Went e 44.

Integration with Example Method Above The following steps 18 and 18A replace step 18 and 18A in the above algorithm in embodiments using the verb loop algorithm: 18. Are there words in the segment that can be found in the Hinge Word list at level (bl0 through b39, and through b99)? No-go to 18 A. Yes-go to step 19.

Step 18A. Repeat the question for Hinge Word list level IF there are no more hinge words at level b39, then before going to level b50, go to the verb loop for algorithms involving level 40-49. After completing the verb loop for level 40-49, then resume folding beginning with level b50 at step 18. If there are no words in the segment that can be found at any level in the Hinge Word List, then go to step 29.

-49- Verb Loop Q1. Are there any words in the segment labeled 40 through 46 (with any letter c,d,e,f, or g before them)? No--go back to step 18 in the usual algorithm.

Yes--go to step Q2.

SQ2. Is there a verb labeled 40 in the segment (or remaining Ssegment)? No--go to step Yes--go to step Q3.

Q3. For the first (or next) 40 level verb in the segment, is x beyond the MINIMUM ROW VALUE (>MIN) No--go back to step Q2 and inspect for the next 40 level verb in the segment distal to x.

Yes--go to Q4.

Q4. Is the verb x a form is it labeled No--go to step Q6.

Yes--go to step Is the word preceding x, a 40 level verb? No--go to step Q6.

Yes--Then DO NOT fold at the space before x. Instead, go back to step Q2 and inspect for the next 40 level verb in the segment distal to x.

Q6. There will POSSIBLY be a fold at the space before x, but there may instead be a fold before another verb at x+n.

Check the word(s) following x, starting with n=l. Is x+n a verb any word labeled 40, 41, 42, 43, 44, or-46, with any of the following letters before the number: 0 c, d, e, f, g)? C No--go to step Q6A.

Z Yes--go to step Q7.

m- c Q6A. Is x a form or an form? c No--then fold at the apace before x. If the word "and" or the word "or" is the word before x then do not fold before x.

Instead, if the "and" wc)rd or the "or" word are greater than O 10 the Minimum Row valie, then fold before the word C- "and" or the word If the word "and" or the word "or" preceding x is less tha h the Minimum Row Value then do not fold--instead gQ back to step 2 and check for verbs in the segment distal t b x.

Place the new segment ir, LANE X (relative to the other prior generations of folding trom the original segment/sentence), SLOT A. Then go to Steb Q19.

Yes--then go to step Q6.

Q6B. Is x-1 the word "tn? No--then fold at the space before x. If the word "and" or the word "or" is the word before x then do not fold before x. Instead, if the "anci, word or the "or" word are greater than the Minimum Row valbe, then fold before the word "and" or the word If the word "and" or the word "or" preceding x is less tha h the Minimum Row Value then do not fold--instead go back to step 2 and check for verbs in the segment distal t( x.

Place the new segment in LANE X (relative to the other prior generations of folding rom the original segment/sentence) SLOT C. Then go to step Q19.

-51- Ys res--tnen fold at the space before If the word 0 immediately before is "and" or then fold before "and" or "or" instead of before If the "and" word or the "or" word and the word "to" are all greater than the Minimum Row value, then fold before the word "and" or the word "or" or the word "to" whichever is first. If the word "and" or the word "or" or the word "to" preceding r n x is less than the Minimum Row Value then do not fold--instead go back to step 2 and check for verbs in the segment distal to x.

Place the new segment in LANE X (relative to the other prior generations of folding from the original segment/sentence), SLOT C. Then go to step Q19.

Q7. Is x+n an form? No--go to step Q7A.

Yes--go to step Q17.

Q7A. Is x+n a form? No--go to step Q9.

Yes--go to step Q8.

Q8. Is a 40 level verb? No--go to step Q17.

Yes--go to step Q18.

Q9. Is x+n an form? No--go to step Q11.

Yes--go to step -52- Is -1 either a 40 ,41, or a 42 level verb? (Note, it is assumed that it will be very rare that such irregular verbs whose infinitive and past participle forms are the same, forms), would also be strictly intransitive, AND that a writer would precede the past participle of such a verb with a "be" verb) No--go to step Q17.

SYes--go to step Q18.

Q11. Is x+n any one of the following: d41, d42, d43 or No--go to step Q13.

Yes--go to step Q12.

Q12. Is either a 41 or a 42 level verb? No--go to step Q17.

Yes--go to step Q18.

Q13. Is x+n either of: d44 or d46? No--go to step Yes--go to step Q14.

Q14. Is a 41 level verb? No--go to step Q17.

Yes--go to step Q18.

Is x+n a form? No--go to step Q17 Yes--go to step Q16.

Q16. Is a 42 level verb OR a GERUND HELPER VERB (w99)? -53- No--go to step Q17.

Yes--go to step Q18.

(N

Q17. ["yes" to Q7 or "no" to Q8, 10,12,14,15,16] Then FOLD but NOT at the space before x. Instead, fold at the space BEFORE x+n. Place the new segment in LANE X (relative to the 0 other prior generations of folding from the original S segment/sentence), SLOT B. Then go to step Q19.

Q18. ["Yes" to Q8, 10, 12, 14, 16]. Then add 1 to n and go back to step Q6 using n+1 instead of n. (That is, inspect the next word that comes after x+1 and see if it is a verb or not, and proceed accordingly.) Q19. Are there still segments that exceed the MAXIMUM ROW VALUE (or the space available to the right)? No--stop.

Yes--go back to step Q2.

Q20. Is there a 41 level verb in the segment (or remaining segment)? No--go to step Q34.

Yes--go to step Q21.

Q21. For the first (next) 41 level verb, in the segment, is x beyond the MINIMUM ROW VALUE (>MIN)? No--go back to step Q20 and check the segment for a 41 level verb distal to x.

Yes--go to step Q22.

Q22. Is x an form? -54- No--go to step Q22A.

Yes--go to step Q33.

c' Q22A. Is x a form? No--go to step Q24.

Yes--go to step Q23.

CM Q23. Is a 40 level verb? 0 No--go to step Q33.

Yes--go to step Q32.

Q24. Is x an form? No--go to step Q26.

Yes--go to step Is a 40, 41, or a 42 level verb? No--go to step Q33.

Yes--go to step Q32.

Q26. Is x any one of the following: d41, d42, d43 or No--go to step Q28.

Yes--go to step Q27.

Q27. Is a 41 or 42 level verb? No--go to step Q33.

Yes--go to step Q32.

Q28. Is x either one of: d44 or d46? No--go to step Yes--go to step Q29.

Q29. Is a 41 level verb? 0 No--go to step Q33.

C Yes--go to step Q32.

Q30. Is x a form? No--go to step Q32.

r Yes--go to step Q31.

Q31. Is a 42 level verb OR a GERUND HELPER VERB (w99)? 0 10 No--go to step Q33.

C1 Yes--go to step Q32.

Q32. ["Yes" to Q23, 25, 27, 29, or 31] Go to Q32A.

Q32A. Is x a 45 or 46 level verb? No--then do NOT fold at the space before x. Instead, go back to step Q20 and check for another verb in the segment distal to x. (If there are no more 41 level verbs in the segment, step Q20 directs inquiry to step Q34 for 42 level verbs.

Similarly, inquiry is directed from step Q34 to step Q36 for 43/44 level verbs, and from step Q36 to step Q38 for 45/46 level verbs).

Yes--go to step Q33. ["yes to Q22 of "No" to Q23, 25, 27, 29, 30, or 31] Go to Q33A.

Q33A. Is x a 45 or 46 level verb? No--then POSSIBLY fold at the space before x, but check (x+n) first. Go back to step Q6.

Yes--go to step Q42.

-56- Q34. Are there any 42 level verbs in the segment (or O remaining segment)? No--go to step Q36.

Yes--go to step For the first (next) 42 level verb, x, in the segment, is x beyond the MINIMUM ROW VALUE (>MIN)? eg No--go back to step Q34 and check for the next 42 level verb C< in the segment distal to x.

0 10 Yes--go to step Q22.

Q36. Are there any 43 level verbs OR any 44 level verbs in the segment (or the remaining segment)? [Note: verbs at either level 43 or 44 are looked for at this step]? No--go to step Q38.

Yes--go to step Q37.

Q37. For the first (next) 43/44 level verb, x, in the segment, is x beyond the MINIMUM ROW VALUE (>MIN)? No--go back to step Q36 and check for the next 43/44 level verb in the segment distal to x.

Yes--go to step Q22.

Q38. Are there any 45 level verbs OR any 46 level verbs in the segment (or the remaining segment)? [Note: verbs at either level 45 or 46 are looked for at this step]? No--GO BACK TO THE USUAL ALGORITHMS AT STEP 18 and resume inquiry using hinge word level Yes--go to step Q39.

-57- Q39. For the first (next) 45/46 level verb, x, in the S segment, is x beyond the MINIMUM ROW VALUE (>MIN)? No--go back to step Q38 and check for the next 45/46 level verb in the segment distal to x.

C 5 Yes--go to step Q22.

["Yes" from step Q32A]. Is x a 45 level verb? SNo-- [x is therefore a 46 level verb]. Fold at the space C< AFTER x. Then go to step Q19 C 10 Yes--go to step Q41.

Q41. [x is a 45 level verb]. Is a verb? [Note: by now, if is indeed a verb, then should be either a or 46 level verb] No--then do NOT fold at the space before x. Instead, go back to step Q38 and check for the next 45/46 level verb in the segment distal to x.

Yes--then DO NOT FOLD at the space before x. INSTEAD, FOLD at the space before (which is also the space AFTER Place the newly formed segment in LANE X (relative to the other prior generations of folding from the original segment/sentence), SLOT D. Then go to step Q19.

Q42. ["Yes" from step Q33A] Is a verb? No--go to step Q43.

Yes--then fold at the space before which is also the space after x. Place the new segment in LANE X (relative to the other prior generations of folding from the original segment/sentence), SLOT D. Then go to step Q19.

Q43. Is x a form or an form? -58- No--go to step S Yes--go to step Q44.

Q44. Is the word "to"? No--go to step Yes--then fold at the space before If the word M immediately before is "and" or then fold before c "and" or "or" instead of before "to".

Ci If the "and" word or the "or" word and the word "to" are all greater than the Minimum Row value, then fold before C the word "and" or the word "or" or the word "to" whichever is first. If the word "and" or the word "or" or the word "to" preceding x is less than the Minimum Row Value (<MIN), then do not fold--instead go back to step 38 and check for 45/46 level verbs in the segment distal to x.

Place the new segment in LANE X (relative to the other prior generations of folding from the original segment/sentence), SLOT C. Then go to step Q19.

Q45. Is a word on the DEFINITE SUBJECTIVE PRONOUN LIST [list x99; these words are I, we, he, she, they, who, etc.]? No--go to step Q46.

Yes--then fold at the space AFTER x. Place the new segment in LANE X (relative to the other prior generations of folding from the original segment/sentence), SLOT D. Then go to step Q19.

Q46. Is a word in the PRE-DETERMINER SET [y99]? No--go to Q46A.

-59- Yes--then do NOT fold at the space before x and go back to step Q38 to check for 45/46 level verbs in the segment distal to x.

0 5 Q46A. Is x an or form? No--go to step Q47.

Yes--then fold at the space before x. If the word "and" or Sthe word "or" is the word before x, then do not fold before x. Instead, if the "and" word or the "or" word are greater than the Minimum Row value, then fold before the word "and" or the word If the word "and" or the word "or" preceding x is less than the Minimum Row Value then do not fold--instead go back to step 38 and check for 45/46 level verbs in the segment distal to x.

Q47. Is x a 45 level verb? No--then do NOT fold at the space before x and go back to step Q38 to check for 45/46 level verbs in the segment distal to x.

Yes--go to step Q48.

Q48. Is a word in the POST-DETERMINER SET [z99]? No--then do NOT fold at the space before x and go back to step Q38 to check for 45/46 level verbs in the segment distal to x.

Yes--then FOLD at the space BEFORE x. (If the word "and" or the word "or" is the word before x then do not fold before x. Instead, if the "and" word or the "or" word are greater than the Minimum Row value, then fold before the word "and" or the word If the word "and" or the word "or" preceding x is less than the Minimum Row Value then do not told--instead go back to step 38 and check for 45/46 level verbs in the segment distal to x.) Cl Place the new. segment in LANE X (relative to che other prior generations of folding from the original segment/sentence), m 5 SLOT C. Then go to step Q19.

(End of Verb Loop).

ADDITIONAL ASPECTS OF THE IINENTiON Text Enhancement Using Reader-Performer Actions SText presentation can be enhanced using actions of a reader or reader-performer as inputs and a variety of audiovisual effects as outputs. In one embodiment, a reader or reader-performer reads text aloud while varying pitch, tone, volume, and pauses for effect. The term "reader-performer" is used to distinguish over the reader of the enhanced text and to note that reading aloud can include many actions in addition to speech. The spoken text is recorded for use in displaying the text. In one embodiment, the spoken text is recorded electronically and analyzed for spoken action attributes such as pitch, tone, volume, and pauses. In a preferred embodiment, spoken text attributes are analyzed and the analyzed attributes are stored in addition to the spoken text itself. The attributes such as pitch, tone and volume, are stored and mapped to correspond to the text letters, syllables and words which are also stored, such that the attributes can be extracted and reproduced in some fashion in correspondence with the original text. Reader-performer action attributes can be presented as display attributes such as font, intensity, font line thickness, font character height, character color, background color, inter-character spacing, inter-word spacing, interline spacing and intersentence spacing.

-61in one example of the invention, the volume of the reader is reproduced as font intensity and the pitch is reproduced as font thickness, with higher pitches being c-s reproduced as narrower characters and lower pitches being reproduced as wider characters. Font intensity can refer to the number of pixels set per unit area to a different color M than the local background pixels. In one examole, intense MH text is black text on white background while less intense <CI text is gray text on white background. In another example of the invention, the character color is varied in accordance with the pitch. For example, higher pitched words are reproduced more toward the red end of the ROYGBIV spectrum while lower pitched words are reproduced more toward the violet end of the ROYGBIV spectrum. In another example, the text is displaced from the normal horizontal base line of the text in accordance with the pitch of the reader-performer.

One embodiment displaces text upward for higher than average pitch and downward for lower than average pitch.

In another example of the invention, the readerperformer's spacing and duration of pronunciation of specific words is reproduced in enhanced text with spacing of letters, words and sentences. The reader's word pronunciation time and inter-word and inter-sentence spacing can be recorded, allowing normal inter-word and inter-sentence spacing to be measured. Words pronounced more slowly than normal can be reproduced with additional spaces between letters of the word. Longer than normal inter-word time spaces can be reproduced using longer than normal inter-word distance spaces on the displayed text. Even longer time spaces can be reproduced by displaying the separated words or phrases on new lines. Extremely long time spaces can be displayed -62as the separated words or phrases on new lines that are S vertically separated by a greater distance than normal.

Hand Gestures In another aspect of the invention, the reader-performer actions recorded during reading include hand and finger gestures. The position and orientation of the hands is preferably recorded and stored to correspond to each word CI read aloud. Orientation refers to the direction pointed to C 10 by the hands, fingers, or palms. In a two dimensional plane,

C

N the orientation may be recorded as a two dimensional vector.

In a three dimensional model, the hands may be recorded as a three dimensional vector. The positions may be recorded as the position relative to the reader-performer's shoulders or other body reference point.

In selected, appropriate text selections, the vertical and horizontal displacement of text can be varied in accordance with text content. One example of this would be display of the sentences "On one hand and "On the other hand The first phrase could be displaced horizontally to the left while the second, opposing sentence could be displaced horizontally to the right. Similar, vertical displacements could be used to contrast hand actions such as "He promised this, but delivered only this".

Hand position and orientation can be recorded using known technology such as the placement of infrared reflecting dots at strategic locations on the hands. The reflections of projected infrared lights can be cap~ured and stored, allowing later reproduction of hand movement. In another method, the reader-performer uses gloved hands and the hand -63positions are recorded directly with a digital camera using Sthe contrast of the hands against the darker background.

Facial Expressions 0 5 In another aspect of the invention, facial expressions of the reader-performer are recorded during reading and stored in correspondence with the words being read. One m embodiment records eye or eyelid shape, eye openness, eyebrow C position, head position, jaw position, and mouth shape. The facial expression actions of the reader-performer can be recorded on a per word basis and displayed back on a per line basis. In one embodiment, the extreme positions are used to represent the position for the entire line. In another embodiment, an average position is used to represent the position for the entire line. In another embodiment, the facial expressions are used to vary the presentation of the text itself. In one example, the degree of openness of the mouth is used to vary the degree of openness of certain closed vowels, such as The more the mouth of the reader-performer is open, the larger the O and the opening within it become. In another example, the degree of height of certain letters such as or is varied depending on the degree of openness of the eyelids. The facial expression representation described above is combined with text swiping previously described in one aspect of the invention. In one embodiment, a line drawing of a human face is reproduced above the word being swiped or highlighted. The facial features on che animated face change dynamically. In another embodiment, o:n face is represente beside eac 11. of text, with the face animated as the line is swiped. In yet another -64ernbodiment, a face is presented near a phrase to be enhanced, 0 and a static facial expression shown on the face.

Text Presentation Along a Curved Line In another aspect of the invention, text is displayed along a curved baseline rather than a conventional horizontal

M

n straight line as a function of reader-performer actions. In one embodiment, the curved line is either invisible or C-i substantially less visible than the text displaced. The curve in one embodiment is varied in accordance with some C-i action of the reader-performer. In one example, the horizontal displacement of the curve is greater for hicher pitches and less for lower pitches. The word pitch can be obtained from a phonetic dictionary e.g. the pitch of the word can be looked up rather than obtained from a speaker's voice. In another example, the horizontal displacement is varied in accordance with the volume of a reader-performer's voice. In one example, the reader-performer action is sufficiently damped so as to maintain an evenly flowing smoothly curved line.

Text Bending Introduction In spoken language, the variability in tone, pitch, and volume from one word to the next creates a signal that can be thought of as a curve or trajectory. From the pronunciation of just a few words of a sentence, the beginning of such a curve is "drawn" on an "aural chalkboard". The listener uses this initial curve to build and anticipate, in the mind, the structure of the phase or sentence as a whole, even before it has been completely 0 uttered by the speaker.

c This pow.erful information makes speaking- istening very c. efficient. However, current text presentation technology does not readily permit a system that could consistently and meaningfully depict these aural curves in graphical form.

M Instead, the reader must resuscitate the elocutionary force of the sentence by decoding the symbols of the text and c-i recreating an aural representation of the sentence as a whole 0 10 in the mind.

0c Within a sentence, segments can be identified using content analysis that would have characteristic prosodic contours. The present invention describes a method to create graphical correlates of these contours of sentence segments, and to integrate these graphical sub-units into a larger structure to depict the sentence as a whole.

The Process 1. Sentence segments are identified using the "sentence folding" algorithms previously described.

2. Instead of horizontal displacement rules, a set of curves is defined and each sentence segment is assigned to one of these curves.

3. The curves are then arranged on the reading field in a pattern that unambiguously depicts the sequence of the segments from top down) the position of segments within these arrangements can be modified slightly, without changing the implied sequence, in order to optimize the perception of the curves and the reading of the sentence.

-66- A sample of a master curve 340 which defines the size S and orientation of segment curves is depicted in FIG. 13.

A series of let ers, such the letter at 34-1, l-e on curve c- 340. An arrow. 342 at the top of the curve inci-cates the Q 5 position with which a sentence segment curve that was drawn from a Head letter (A through G) to a Tail letter (H through S K) would be oriented on the reading field and relative to r< other sentence segment curves.

CI The Head letter position on the curve is determined by the circumstances of the beginning of the segment. The Tail letter position is determined by the circumstances of the end of the segment. These circumstances are summarized in a table 350 in FIG. 14.

After the Sentence folding algorithms have identified segments of the sentence which will occupy separate rows (in this description, they will occupy separate curves), the length of the segment is measured. A line is drawn that corresponds to this length. The line is then "bent" so that the shape (in degrees and orientation) of the newly formed curved line conforms to the Head-Tail letter positions specified in FIGS. 13 and 14. The "dangle" point, or the position of the arrow at the top of the curve, is also identified.

The size of a segment curve is determined by its length, in characters; however, regardless of its length, the shape and orientation of the segment: curve will be determined by the process described above. If a segment is very short, such that the curved segment would be too small and the curve too extreme, a minimum curve would be defined. Short segments with an H tail would then be placed at the Head letter specified by their beginning circumstances-but the -67remainder of the segment would follow the curvature specified 0 by the minimum curve path and the "dangle point" would also be soecified 1;y the minimum curve. Si.ilarl, sh-ort segments with an I, J or K tail would be placed at the Tail letter specified by their ending circumstances, and the remainder of the segment would follow the curvature specified by the minimum curve path and its "dangle point" or Parameters for maximum line length would likely prevent Ci segments from being so long, have so many characters, that their curves would be very large and relatively flat.

Cl The newly formed curves would then be positioned, in this example, with the first segment of the sentence on tcp.

In one embodiment, the location of the dangle point is a function of the next segment could be developed that was a function of the tail of the segment preceding it. After all of the curves are in position, the characters of text for each segment are positioned on the curve and the lines and the dangle point are erased. A sample of this process is shown in FIGS. 15 and 16.

New Qualities Imparted by the Text Bending Process This text presentation method allows the reader to recognize, by a particular curve in the layout of its characters, an individual segment's structure and function (not just sequential position) in the overall structure of the sentence. This provides graphical information that facilitates cognitive processin of the sentence's segments while reading. This is similar co the. way the word-to-word variability in pitch and tone in speech creates an aural curve that helps the listener predict how long a phrase is -68going to be and whether or not it will be the end of a sentence or clause.

Conventional tex1r only usses punctuation marks, w,:hich are h. often not discovered until the reader has already read the entire phrase-this prevents the reader from making, in the mind, a corresponding phonological model of what was read Suntil after the entire segment is read. The curves created by my process allow this type of phrase structure and C< function information to be known in advance, so that this information can be used in word recognition and phrase Cl comprehension. These curves are also defined by content hinge words) other than punctuation marks. They also further differentiate segments created in the sentence folding process, by depicting circumstances of both the head and tail of a segment.

Referring now to FIG. 15, a sentence 380, the first sentence of the Gettysburg Address shown in cascade form as generated by algorithms (Step 1 under "The Process" heading above). A column 386 specifies a start letter 382 and an end letter 384, for each line of the sentence. This determines the type of curve to be curve to be assigned to each segment.

An integration of these individual segment curves, from Step 3, is illustrated in FIG. 16. For example, sentence 408 is bent over a curve 400, starting at an end position A, ending at an end position 404, and having a dangle point denoted by 406. Subsequent segments are placed beneath preceding segments by placing the "dangle point" approximately two-thirdss he wa the cuive above them.

Additional content-based rules could optimize these "dangle point' positions. In one embodiment, the lines and Head/Tail letters do not appear, only the characters of the text -69appear. A series of bent text curves generated by the O example in FIG. 15 are illustrated by curves 410, 412, 414, 416, and 418.

The curves enhance the "visual prosody" of the text presentation. Each sentence acquires an even more unique visual appearance. The curves also break up the monotony of M linear presented text, which may reduce fatigue of the eye M muscles.

Text Hierarchy Display Another aspect of the present invention provides an improved means for displaying text hierarchies and the location of text being read within those hierarchies. A conventional text hierarchy, listed from higher to lower rank, includes volume number, section, chapter, sub-section, page, paragraph, and sentence. The present invention provides a method for informing the reader of both the number of the units in each hierarchy and the position of the text being read within that hierarchy.

Referring to FIG. 17, an unopened book 200 is illustrated. The book is selected for opening by the reader, as indicated at The reader selection can be performed using any method known in the art including cursor movement and selection with a mouse. Referring now to FIG. 18, book 200 has been selected, opened and displayed as a plane 202 filling a substantial portion of a display screen 201.

Opening book 200 has revealed a plurality of chapters 206 each having an exposed margin 204. In one e b-odiment, the chapter margins have the chapter number inscribed thereon as indicated at 208. Reader selection of Chapter 4 is indicated at in FIG. 18, causing the chapters to separate at Chapter 4 as illustrated in FIG. 19.

Referrin now to FIG. 19, the resul: of selecting -s Chapter 4 is illustrated. Chapters 206 are illustrated 5 having separat:ed to display Chapter 4, exposing pages 210 within the Chapter. Pages 210 have vertical margins 212 exposed. In one embodiment of the invention, pages are c numbered and 4rranged to mirror the pages in a print edition CI of the book. In another embodiment of the invention, pages are numbered to take advantage of the enhanced format of the text presentation, with no attempt made to mirror any print edition of the book. Specifically, the pages can be formatted and numbered to contain only a maximum number of paragraphs or a maximum number of words, whichever is reached first. In this embodiment, paragraphs need not be broken across pages. In this embodiment, the number of paragraphs and words in the maximum are preferably set to values approximating those of a normal published page. In yet another embodiment of the invention, the number of pages in a chapter is set to a maximum, for example 50, with the paragraphs evenly divided into those 50 pages. In this last example, at mo,st only 50 pages are presented on the display screen. Readhr selection of page 3 is indicated at in FIG. 19, causing the pages to separate at page 3 as illustrated ir FIG. Referring now to FIG. 19A, the result of selecting Chapter 4 in FIG. 18 is illustrated in an alternate embodiment. ('hapters 206 are illustrated having separated to display Chabter 4,.exposing pages 210A within the chapter.

Pages 210A have horizontal margins 212A exposed. In this embodiment, the nesting of the text hierarchies is denoted -71- I by the margins having a shorter length for lower ranked hierarchies. In the example of FIG. 19A, pages 210A have margins oriented substantially oaralel to Chapters 206, but with the margin 212A of pages 210A having a shorter length than the margins of Chapters 206.

Referring now to FIG. 20, the result of selecting page 0 3 is illustrated. Pages 210 have separated at page 3 to Sdisplay paragraphs 214 within. In the initial display, only paragraph horizontal lower margins 216 are displayed. Reader selection of paragraph 3 is indicated at in FIG. causing the paragraphs to separate at paragraph 3 as illustrated in FIG. 21.

Referring now to FIG. 21, the result of selecting paragraph 3 is illustrated. Paragraphs 214 have separated at paragraph 3 to reveal sentences 218 having vertical margins 220 within. In one embodiment, the first sentence is initially displayed, ready for reading, as illustrated at first sentence 222. In another embodiment, only margins 220 are initially displayed upon opening the paragraph. Reader selection of sentence 4 is indicated at in FIG. 21, causing the sentences to separate at sentence 4 as illustrated in FIG. 22.

Referring now to FIG. 22, the result of selecting sentence 4 at 223 is illustrated. Sentence 4 is the sentence currently being read. In one embodiment, the completion of reading sentence 4 is indicated by the reader activation of a computer peripheral device such as a mouse button. In another embodiment, the completion of reading a sentence is indicated by the passage of a time period calculated to correspond to the weight of the sentence and the reader's recent reading speed. In a preferred embodiment, the -72completion of sentence reading is switchable between a time based mode and a reader actuated mode. In one embodiment, speed is reader adjustable using input such as a sliding scroll bar. This allows the reader to increase and decrease C 5 speed easily to suit the passage being read.

When completion of reading a sentence is indicated, the next sentence is advanced into position. in FIG. 22, in one embodiment, sentence 4 slides quickly to the left to reveal I sentence 5 underneath. In another embodiment, sentence 4 and 1 10 all subsequent sentences in the paragraph slide to the left by the width of one vertical margin, with sentence 4 continuing to slide left to takes its place underneath sentence 3. In this last embodiment, the sentence being read can be maintained in the same position by moving the sentence before the text of the sentence becomes visible to the reader. After the last sentence in the paragraph is read, the next paragraph is opened to reveal the sentences within, resembling closely the illustration of FIG. 21, but with the next paragraph exposed or pulled down.

In a preferred embodiment of the invention, alternate text hierarchies are represented by planes having alternate horizontal and vertical exposed margins. Stated differently using current Microsoft Windows terminology, alternate text hierarchies are represented by alternately horizontally and vertically tiled planes. In one embodiment, the outer, higher ranked text hierarchies appear to be behind the inner, lower ranked text hierarchies, as the opened higher ranked text hierarchies have the next lower ranked text hierarchy displayed thereover.. In another embodiment, the Outer, higher ranked text hierarchies appear to be closer to the reader, with the inner, lower ranked hierarchies displayed -73rurtner rrom the viewer. In this embodiment, the currently O opened text hierarchy is preferably not displayed, except for the currently being read sentence. In one ex:ample using horizontally tiled text units and having a text unit number 3 opened, the text units appear as a deck of cards splayed open, having 1 on top of 2 on top of the top edge of 3 and 4 under 3 and 5 under 4. The units toward the top of the Cscreen appear closer to the viewer than the units toward the C' bottom of the screen. This is the arrangement used in the embodiment of FIG. 22 with respect to paragraphs, but with paragraph number 3 opened.

In another example, text units 1 through 3 are as previously described, but text unit 4 is on top of the lower edge of text unit 3, and text unit 5 is on top of all but the upper edge of text unit 4. In this arrangement, the text units appear to grow further away from the viewer from the outer margins until the opened text unit is displayed, after which the text units appear to become closer to the viewer until the opposite outer margin reached. This arrangement is similar to having a series of nested theater curtains, the outer curtains being in front, the more inner curtains being behind the outer curtains, and the more inner curtains revealing a smaller aperture than the outer curtains.

In the general case, the orientation of the margins is varied depending on the text hierarchy level represented.

While the illustrated embodiments have the margins oriented alternately horizontal and vertical, 90 degrees offset, in the general case, the offset could be less. .In one example, the highest level hierarchy has horizontal margins (extending from 9 O'clock to 3 0' Clock), and the next, lower level is -74onrset oy 30 degrees, extending from 10 O'clock to 4 O'clock.

c' Referring again to FIG. 22, the display o th current c. "book" being read can be explained without reference to how the reader came to be reading a particular %entence. The next lower text hierarchy is indicated by chaPters 206 which n are horizontally tiled, having horizontal lower margins 204 exposed. The next lower text hierarchy, Pages 210, are displayed superimposed over chapters 206. Pages 210 are vertically tiled, having vertical margins 212 exposed. The C-i next lower text hierarchy, paragraphs 214, are displayed over pages 210. Paragraphs 214 are horizontally tiled, having horizontal margins 216 exposed. The nex.t lower text hierarchy, sentences 218, are vertically tiled, having vertical margins 220 exposed.

The reader is able to visualize the lqcation of the sentence being read within the paragraph, the paragraph within the page, the page within the chapter, And the chapter within the book. Using known three dimensional shading and perspective display techniques, e.g. planar edge shading and bevelling, nested planar text hierarchies can ive the reader a three dimensional feel of the location of the text being read.

In one embodiment, the sentences are displayed in horizontally sliding planes, In another elbodiment, the sentences are displayed in a series of hori.ontally tiled planes that slide horizontally, similar to paragraphs 214 in FIG. 22. In one embodiment, sentences are alqays set to be displayed as either horizontally or vertically tiled. planes, with the higher ranked text hierarchies alternating as horizontally and vertically tiled planes.

In some text bodies, there may be a large number of units, for example, a very large number of pages in a chapter, or a book with no chapters. To deal with this c-s situation, an artificial hierarchy can be created. In one Q 5 embodiment, an "Order of Ten" hierarchy can be created. For example, if the number of chapters is greater than 50, an artificial unit of "10 Chapters" can be creat-ed. This C artificial unit can be created as either a higher ranked text (CI hierarchy to be displayed as such or simply as a unit of similar rank to be tiled the same but displayed differently.

In one example, the unit of "10 Chapters" is tiled vertically while the chapters are tiled horizontally. In another example, the units of "10 Chapters" and chapters are both tiled horizontally, but with the "10 Chapter" unit having a darker color or otherwise different appearance.

Text Display Across Opposed Pages In another aspect of the invention, a method for displaying text on subsequent pages is provided. Referring now to FIG. 23, a left page 250 having a right margin 252 is illustrated as is a right page 254 having a left margin 255, obscured by right margin 252. Page number 3 is displayed as left page 250 and page number 4 is displayed as right page 254. Page number 3 is indicated as being read by the Upon completion of reading page number 3, this can be indicated to the reading system by reader action or by completion of a time interval. Upon completion of reading page number 3, indicated by arrow 251, page numaber 4 is displayed "in front of" page number. 3, with right page 254 left margin 255 obscuring left page 250 right margin 252.

Upon completion of reading page number 6, indicated by arrow -76zbJ, new pages are displayed, with page number 5 being displayed as left page 250 and page number 6 being displayed as right page 254. In this fasiion, pages are alternated left and right to mimic the familiar pattern of paper books C 5 readers are accustomed to.

Referring now to FIG. 24, another embodiment of the 0invention is illustrated. A left page 260 having a right CM margin 262 is illustrated as is a right page 264 having a Cl left margin 265, obscured by right margin 262. page number 3 is displayed as left page 260 and page number 4 is displayed as right page 264. Page number 3 is indicated by the Upon completion of reading page number 3, indicated by arrow 261, page number 4 is brought to the front such that left margin 265 obscures right margin 262. In addition, upon the reading completion of page 3, page number 3 is removed and replaced with page number 5. This configuration is not possible using traditional, paper bindings. Upon completion of reading page number 4, indicated by arrow 263, page number is brought to the front and page number 4 is removed and replaced with page number 6.

The embodiment illustrated in FIG. 24 has the advantage of allowing text segments to be broken across pages while always providing the next page to be read on the screen.

This allows the reader to advance from previously displayed text without having to wait for new display of text. For example, a text segment broken across page number 3 and 4 could be read by the reader without requiring a pause for the contents of page 4 to be displayed. A text segment broken across. page numbers 4 and 5 could. be read by. the reader without requiring a pause for the contents of page 5 to be displayed. In essence, the reader is able to read -77electronically displayed text without having to wait for page flipping, while at the same time having the familiar presentation of Lext on opposinqg aaes, a throw back to c- bound, paper books. The middle configuration of FIG. 24 is not bound by binding.

Text Display for Reading from Front to Sack c Referring now to FIG. 25, another aspect of the Cl invention is illustrated in a screen display of text 280, having a first text line 282, a second text line 284, a third text line 286, and a fourth text line 288. Using perspective illustration techniques, first line 282 can be made to appear closer to the reader than second line 284, second line 284 closer than third line 286, and third line 286 closer than fourth line 288. In one embodiment of the invention, text is displayed one screen or display page at a time, with all lines on the display, even the most distant lines, being readable. In one embodiment, the line actively being read is indicated as being brighter or having higher contrast than the lines of text not having been read. The line actively being read can be estimated by advancing a brightening line wide cursor at the rate of the reader's estimated reading speed. In a variation of this embodiment, a line wide translucent filter is advanced over the line actively read, such that text not being on the line being read appears slightly fuzzy and less readable than the text being read.

Referring now to FIG. 26, the display of FIG. 25 is illustrated as a modified display page 290,. with the text having been .advanced one line forward. -First line 282 has been forced forward off the display while second line 284 is now at the forefront. In this embodiment of the invention, -78text can De advanced a line at a time, toward the reader.

0With current technology, the text can be made to scroll smoothly forward toward the reader. A perspective effect can h. be achieved by making the lower text lines with larger fonts and greater interline distances. Three dimensional fonts can also be used. In a preferred embodiment, the text is Sdisplayed as enhanced text after processing, usiric folding on methods described above.

eC In an alternate embodiment, text is displayed to appear as being displayed from front to back when read from bottom 0C to top, as described above. In this embodiment, the text is displayed statically, one screen at a time, with text being paged through in multiple line chunks rather than being smoothly scrolled through.

The embodiments illustrated in FIGS. 25 and 26 take advantage of the natural image processing selected for by human evolution. Specifically, humans, prior to reading, processed information while walking or otherwise moving forward, with newer information appearing first as further away and moving closer with time and movement through the landscape. English is written from top to bottom to allow the writer to read what has been written on the preceding line. Writing from bottom to top would cover up the lines just written with the writer's hand. This aspect is well known to left handed writers in a slightly different form.

English is written from left to right, allowing right handed writers to see the words they have just written. Left handed writers cover up what they have just written, and often assume .a cramped writing hand position in an attempt to compensate for this disadvantage. Publishing has heretofore adapted the top to bottom style used for hand written text, -79without having to. The present invention frees text display O from the top down convention, allowing display in a more natural, albeit unconventional adiso1la:Y format.

Varying Transition Patterns With Text Levels In another aspect of the invention, transition patterns between text displays is varied as a function of the level or hierarchy ranking of the text. In one embodiment, the Cq transition between sentences within a paragraph is different 0 10 than a transition between paragraphs. The transition between C sentences can be varied as a function of the position of the sentence within the paragraph. The transition between paragraphs can be varied as a function of the position of the paragraph within the chapter or other text hierarchy within which it resides.

In one example of the invention, transition between paragraphs is accomplished by dissolving the just displayed sentence, before displaying the next sentence. The dissolve can be a fine grained dissolve for the first paragraph in a chapter, a coarse grained dissolve for the last paragraph in a chapter, and a continuum of granularity therebetween for intermediately positioned paragraphs. In another example of the invention, each successive sentence in a paragraph could be displayed starting either at a more rightward position, a more downward position, or both. In this way, successive sentences in a paragraph would march either rightward, downward, or both with the progression of the paraqranh.

In another embodiment of the invention, a musical key, played by an instrument, is associated with a paragraph to be displayed. In one example, the same key and instrument is associated with every paragraph. A number of notes corresponaing to the number of sentences in the paragraph is played down the scale in that key at the start of the display C' of the paraoraph. At the beginning of- the display of each t, sentence, the corresponding note in the scale is played up or down the scale. In a corresponding example, a number of notes corresponding to the number or location of that sentence is played. In one embodiment, if the sentence Scontained more than one clause, a chord is played. In one C- embodiment, if the sentence contained contrasting expressions 0 10 such as "despite" or "however", a minor or discordant chord C- could be played. In one embodiment, a different instrument and/or key is used for every paragraph.

Temporal Protrusion In another aspect of the invention, certain words are selected to protrude temporally from the other words in the sentence. The selected words in one example are displayed before the remaining words in the sentence are displayed.

In another example, the selected words are displayed after the remaining words are no longer displayed. In yet another example, the selected words are displayed both prior to the remaining words being displayed and after the remaining words are no longer displayed. The displayed before or after the remaining words are believed to receive more attention from the reader. Words displayed prior to the remaining words are believed to register with and partially analyzed by the reader prior to the reading of the remainder of the sentence, Lhis is believed to make the reader aware of the word such that the reader is not caught unaware and having to .back track.

-81in one embodiment, the words are selected based on the 0 difficulty the reader is predicted to have with the selected

C

words. In one example, words above a certain educational level are selected for temporal protrusion. In another M 5 example, words having a sound-spelling mismatch are selected for temporal protrusion. A sound-spelling mismatch refers r to words whose pronunciation is not deriveable from the pq spelling. In yet another example, a word frequency dictionary is used, and words used with low frequency are selected for temporal protrusion.

Prose Painting In another aspect of the invention, color is used in conjunction with text analysis to display the interrelationship of parts of a sentence with another. One embodiment of this method can include the steps listed below.

a. Analyzing the sentence, subdividing the sentence into a plurality of text segments based on relationships of said text segments to each other.

b. Organizing said text segments into a tree having a root parent node and a plurality of parent nodes and child nodes.

c. Assigning each child node a attribute based on the relationship with the parent node.

d. Assigning the root node a color.

e. For each child node beneath the root node, assigning a color to each child node based upo:n the color of the parent node modified by the relationship between the child node and the parent node.

-82- 1. Repeating step e until all nodes have a color assigned.

g. Displaying said text in lines in substantially the c- same order specified by the author using the color assigned in the tree.

Referring now to FIG. 27, in one embodiment of the above recited algorithm, the parent-child relationships are c selected from the group consisting of entity, action and CI modification. An original sentence 300 is subdivided into three text segments 302, 304, and 306, as indicated by a tree C 301. Text segment 302 has an entity relationship to parent 300, segment 304 has an action relationship to parent 300, and segment 306 has an action relationship to parent 300.

Parent 300, in a generation 0 at 308 has an initial color triple (Red Green Blue) of R00-G00-B00, as indicated in segment 300. The amount of color added to each generation of child segments can be a function of the generation number itself. In FIG. 27, 100 units of color are added to a first child generation at 310, 50 units of color are added to a second generation 312, and 25 units of color are added to a third generation 314. The color itself is added as a function of the parent-child relationship. In FIG. 27, Blue is added for Entities, Red for Actions, and Green for Modifiers.

In FIG. 27, segment 302 is has an Entity relationship with parent 300 and is at the first generation level, so 100 units of Blue are added. The amount of color added is less at each generation. The function oi each w.ord in the sentence is thus influenced most by the function of .the largest segment of which it is a part.

-83- New Qualities Imparted by the Prose Paint Process Speech is a communication system in which individual uttered units, such as words, haveI discrete meanings, and a -s larger meaning is constructed by the relationships among a C 5 group of words. The sequence of words is important structural information that is used to build the meaning of Sthe relationships among the words. Within sequences of CMf words, smaller sequences of words have special relationships CI among them. In this way, the order of two or more smaller sequences of words within a larger sequence of words is also a form of structural information that is used to build the meaning of the relationships among all the words.

In addition to selecting the order in which words are uttered, a speaker also pronounces each word, or group of words, with certain sound attributes that are in addition to the simple pronunciation of the phonemes that indicate what the word is. These additional sound attributes include the pitch, volume and timbre with which the words are spoken.

?0 These sound attributes commute and combine in that a particular word within a phrase will be modified relative to what these additional sound attributes are for the phrase as a whole.

These additional pronunciation attributes, such as pitch, volume and timbre, (hereinafter referred to as intonational attributes) therefore represent a multi-dimensional system of values that can be combined and which commute across successive levels of differentiation of word groups into smaller sub-units within the larger word groups. These values represent the relationship of the word -84- [or woras) to the larger unit of the group of words it was D a part of.

c In addition, each sub-unit is keot within a range of intonaional values that is appropriate for its immediate context. A larger unit, which may contain several sub-units of its own, similarly will relate to the yet larger unit r n containing sub-units like itself, with a change in one these same intonational values that is appropriate for this larger C- context.

0 10 For example, a careful audition of the sentence: "The C-i purple hat of the purple clown was put into a purple box."-will discern that the intonational values of each of the words "purple" appropriately set the word "purple" apart, in a tonal way, from the larger units the word "purple" is modifying in each case. In the first case, "The (purple) hat", is the principle entity of the sentence as a whole: the intonational value of "purple" in this case is built up from a baseline set of values used to represent the principle entity.

In the second case, the phrase "of the (purple) clown", itself is a segment that has a modifying relationship to the entire subject phrase. This second "purple" therefore has a lower volume than the first "purple" but still has a pitch that sets it apart, in a tonal way, from "clown", in the same way that the first "purple" set itself apart from "hat".

In the third case, the entire phrase-- "was put into a purple box"-- is varied in a different tonal dimension, (the l e-.C is sharper Lo sec it aparc- from the Eirse hali: of the sentence, as a way of depicting that this segment is in an "action" dimension relative to the sentence. The sub-units of this phrase retain this baseline timbre betore each phrase of a sentence. "Subject-the purple hat-modifier of subject-of the purple clown-verb-was put Ce- into-object-the purple box." However, the existing methods of representing the C 5 relationships between groups of words in a sentence, such as parse trees or grammatical diagrams, are not transparent, but C instead rely on additional annotation and symbols to label S the word groups. These labels, in turn, are fixed and do not combine or commute values across successive levels of c-i S0 sub-unit differentiation, or limit such values according to S their context.

It is therefore desirable that enhanced text presentation should enrich the appearance of the actual words of the text with visual attributes that are similar to the intonational attributes of speech. These visual attributes should create a multi-dimensional system of combinable and commutable values that represent attributes of the words in addition to their word-specific meanings. These visual attributes should also be transparent, perceivable 3 simultaneously with the perception of the words themselves.

Moreover, unless a text presentation method systematically assures that the author-specified sequence of words is unambiguously represented, it is not text presentation as meant in this discussion, but simply a method to demonstrate the results of some narrow aspect of text analysis which subordinates the integrity of the author's message to the objectives of such an analysis.

The "Prose Paint"- process described above uniquely represents these intonational conditions of speech in an unambiguous text presentation. In addition, the "Prose -86- Paint" process is a novel approach to syntactic analysis S itself, in that the properties of entity, action, and

C

modification, whichhe present he relationships betr.,'een child and parent segments, also combine and commute across successive levels of differentiation of a sentence.

In other words, it is not only that enhanced visual S presentation of text should aspire to the conditions of speech, but that syntactic analysis itself should aspire to eC the conditions of intonality. A basic exercise for testing S.0 grammar rules in linguistic analysis is to create a series C1 of words using the grammar rules and then Co listen co the proposed sentence to determine if it "sounds right". It is possible, as illustrated by the previous discussion on intonational values, that the inner "reasoning" by which "complete sentences" are intuitively recognized as valid propositions is mediated by a multi- dimensional system of attributes for the relationships of sub-units to larger units whose outward expression is intonality. The mathematics of the mind may be musical.

0 The method can be used to display text using either the text folding described previously or using standard text displays, but in color. The method can also stop short of segmenting a sentence into words. For example, the algorithm can stop after a fixed number of levels, or leave phrases intact, leaving all words in the phrase with the same color.

Numerous characteristics and advantages of the invention covered by this document have been set forth in the forgoing description. It will be understood, however, that this disclosure is, in many respects, only illustrative. Changes may be made in details, particularly in matters of combining, separating, and arranging steps, without exceeding the scope -87- 0 of this invention. The invention's scope is, of course, CKI defined ii.; hie .:.angua e in whi.ch the aIppe:d d clai.ms are Sexpressed In the claims which follow and in the preceding description, except: where the context requires otherwise due to express language or necessary implication, the word "compri se" or variations su ch as "compr:ses" or "comprising" is used in an inclusive sense, i.e. to specify the presence of the stated features but not to preclude the presence or addition of further features in various embodiment-s of the ivention It .is be understood that, i. iany pri-or art publication is referred to herein, such reference does not constitute an admission that the publication forms a part of the common general knowledge in the art, in Australia or any other country.

-88- APPENDIX I b af ter bl0 as miuch as blo because biG before biG how M blo if' blo inasmnuch as M biG in case ClbiG in order that biG except 0 biG in lieu of biG in spite of but for whom in which for which b20 above which

I

he can can't e40 cannot could could not couldn't I'll e40 he'll it'll I'd he'd have to e40 had to do do not don't does e40 doesn't e1O does not e4 0 did.

dil *not e4G didn't c41 have e41 I've -89e4l e4 1 d4 i.

d4 1 d41 941 e4 1.

e4 1 e4q 1 c42 d4 2 g42 e42 e42 e42 e42 e42 e4 2 e4 2 e42 e42 e42 e42 e42 c45 e43 e43 e43 g4S z e43 z c4,q z d44 a g46 E e44 z c43 a d45 a a e43. a c43 a a g45 a c43 a have not had norhadi 't having has has no'n-asn;r U be been being am i~s are I'm he 's it's am not I'm not are not aren' t isn't he's not awake awakened awaking a wakes awoke ate ibandon ibandone d abandoning abandons ibstain bstained 3bstaining ibstains Lccept ccepted Lccept ing iccept~s ccornpany .cconipanied ccornpanying .ccornpanies c43 account for accounted for accoun:t n.:c for c43 accounts for c43 accumulate accumulated accumulating e43 accumulates c43 accuse d45 accused accusing e43 accuses act acted g45 acting acts c43 add added adding e43 adds c43 adjust adjust adjusting e43 adjusts c43 adopt adopted adopting c43 adopts c43 advise d45 advised advising e43 advises c43 affirm affirmed g45 affirming e43 affirms c46 aggregate d46 aggregated g46 aggregating e46 aggregates c44 agree d44 agreed g46 agreeing.

e44 agrees c43 agree to agreed to -91c 4 3 3 945 c4 3 c4 3 d4 5- 04 e43 c43 94 5 e4 3 1 5 c43 945S d45 g4 5 e4S c44 d4 4 g4 6 e44 c43 e43 c44 d44 g46 e44 c43 d4S e43 c43 d45 945 e43 c43 945 e432 agreeing to agrees .geduo agreeinL upon agreesnc upon all ow o w loe a.

allows allow for allowed for allowirlq for allows -for amend amended amending amenrds answer answered answering answers appear appeared appearing appears apply applied applying applies arrive arrived arriving arrives arrive at arrived at arriving at arrives at ask asked a ski ng 3asks ask for asked for asking for asks for -92c43 e4l 3 c45 d4 rt"3 d45 945 e43 c43 945 e43 c43 g46 e43 c43 e43 z a c43 a a a e43 a c43 a d45 a a e43 a c43 a a 945 a £243 a a a.

e43 a: c43 ai attach attached a.*Lac hlnc:1 aE:aches attack attacked attackinQ attacks a 11O i 1 avoided avoiding avoids abrogate abrogated abroga in abrogates actuate actuated actuating actuates adapt adapted adapting adapts idvocate ~dvocated tdvocating dvocates Iilay illayed .llaying llays liege Ileged lleging lleges ppend ppended ppending ppeno s rraign rraigned rraigning rraigns uthenticate -93authenticated authent-icating c4 3 avow d4 5 avowed avowing e43 avows b~o both ,eI 1,--r b60 again as away besides b6O always b60 during b6O by b6O concerning about above b70 across against along amid among b70 around beneath beside inside between b70 below beyond at into in b75 for b7S from b8O of b8l and all b90 any b9O a n b9&~ anyone.

both e ach b90 either everybody -94- S- b90 w99 9 S, %%99 w99 w99 w99 W99 11: w99 W99 w99 w99 w9 9 2 C w99 X99 X99

Y

9 9 y99 y99 y99 y99 z99 z99 z99 1 z99 1 z99 1 z99 1 z99 i z99 i z99 a z99 b2 z99 e z99 a z99 e S C z99 a z99 e z99 f z99 a z99 e everyone F ew herself himself greater *chan a a dvri s e a-,d I Se d advising advises admit admitted admits anticipate anticipated anticipates avoid avoided avoids

I

he an.

'ier its 'is uer iers -t ~ts 11l )oth verybody ny ach ye -CV tfhI ng nybody ither e w nything nough

Claims (19)

1. A method for enhancing text presentation from a machine readable natural language text based on reader O specific parameters including at least the viewing field o dimensions comprising: a. parsing said text into punctuation and parts of Sspeech for extracting text specific attributes; s b. storing said text specific attributes in relation to the parts of speech to produce an enriched text; c. applying primary folding rules followed by secondary folding rules to said enriched text, applied in order of a folding rule rank thereby dividing said text into text segments, said folding rules having at least said punctuation attributes and parts of speech attributes as inputs and visual attributes as outputs, said visual attributes including the displaying of the text segments in new lines; d. applying secondary folding rules until a limit is reached, this limit preferably being the minimum line length; e. applying text segment horizontal displacement rules to said text segments to determine a horizontal displacement for each text segment, said horizontal displacement rules including parts of speech as inputs and visual attributes of horizontal displacement as outputs to produce an enhanced text; and f. displaying said enhanced text by cascading the text segments in lines down and across a display.

2. The method of claim 1 wherein said text specific attributes include text location within a text body, said text presentation includes background colour and said text -96- processing includes varying said background colour in accordance within said text location.

3. The method of claim 1 wherein said text specific attributes include a text difficulty measure, and said text presentation includes an automatic text advancement rate, and said text processing includes varying said text O presentation rate in accordance within said text 0 difficulty measure. D

4. The method of claim 3 wherein said text difficulty measure includes an estimated pronunciation time of said text and/or an estimated educational level of said text and/or the number of syllables in said text.

The method of claim 1 wherein applying primary and secondary folding rules comprises: providing a minimum text segment length; providing a maximum text segment length; determining primary folding point locations in said text by applying the primary folding rules, thereby dividing said text into super-phrases; determining secondary folding point locations in the super-phrases by applying the secondary folding rules, thereby dividing the super-phrases into text segments; repeating said previous step until all text segments are no greater than the maximum text segment length, and no less than the minimum text length.

6. The method of claim 1 further comprising: accepting user edits of the displayed text folding points; and -97- storing the user edits in machine readable form; and/or accepting user edits of the displayed text parts of speech; and storing the user edits in machine readable form;and/or Saccepting user annotations to the displayed text parts of speech; and storing the user annotations in machine readable form.

7. The method of claim 1 wherein said visual attributes include the displaying of said text segments in a colour depending on said parts of speech and/or the displaying of said text segments on new lines.

8. The method of claim 1 wherein said primary rules include punctuation as inputs; said secondary rules include parts of speech as inputs.

9. The method of claim 8 wherein said folding rules further include a microgrammar for disambiguating said parts of speech.

The method of claim 1 further comprising determining the likely parts of speech of said words from said parsing step.

11. The method of claim 1 further comprising: providing colour display rules, wherein said colour display rules include said likely parts of speech as inputs and text colour as an output; and displaying said text according to said text colour output. -98-

12. The method of claim 8 wherein applying primary and secondary folding rules comprises: -providing a minimum text segment length; nproviding a maximum text segment length; determining primary folding point locations in said Mtext by applying the primary folding rules, thereby dividing said text into super-phrases; determining secondary folding point locations in the super-phrases by applying the secondary folding rules, thereby dividing the super-phrases into text segments; repeating said previous step until all text segments are no greater than the maximum text segment length, and no less than the minimum text length.

13. The method of claim 5 further comprising: accepting user edits of the displayed text folding points; and storing the user edits in machine readable form; and/or accepting user edits of the displayed text parts of speech; and storing the user edits in machine readable form; and/or accepting user annotations to the displayed text parts of speech; and storing the user annotations in machine readable form.

14. A device for enhancing text presentation from a machine readable natural language text an enhanced text presentation based on reader specific parameters including at least text viewing field dimensions comprising: -99- C parsing means for parsing said text into punctuation and parts of speech for extracting text specific attributes; storing means for storing said text specific attributes in relation to the parts of speech to produce an enriched text; means for applying primary folding rules followed by (secondary folding rules to said enriched text, applied in order of a folding rule rank thereby dividing said text into text segments said folding rules having at least said Spunctuation attributes and parts of speech attributes as inputs and visual attributes as outputs; means for applying secondary folding rules until a limit is reached, this limit preferably being the minimum line length; and wherein the visual attributes include the displaying of the text segments in new lines; means for applying text segment horizontal displacement rules to said text segments to determine a horizontal displacement descent angle including attributes of the text in the line above for each text segment, said horizontal displacement rules including parts of speech as inputs and visual attributes of horizontal displacement as outputs to produce an enhanced text; and means for displaying said enhanced text by cascading the text segments in lines down and across a display.

The device of claim 14 wherein said means for applying primary and secondary folding rules comprise: minimum text segment length input means for inputting a minimum text segment length; maximum text segment length input means for inputting a maximum text segment length; primary determining means for determining primary folding point locations in said text by applying the primary folding rules, thereby dividing said text into super-phrases; -100- secondary determining means for determining (N secondary folding point locations in the super-phrases by n applying the secondary folding rules, thereby dividing the super-phrases into text segments; means for repeating said step for determining Msecondary folding point locations until all text segments are no greater than the maximum text segment length, and Mno less than the minimum text length.

16. The device of claim 15 wherein the enriched text includes words and phrases and is implemented as a linked list of nodes, each node having the word and phrase attributes.

17. The device of claim 16 wherein said folding rules include primary rules and secondary rules, said primary rules include punctuation as inputs and said secondary rules include parts of speech as inputs; comprising a table containing word sets establishing a hierarchy of secondary folding points.

18. A method as claimed in any one of claims 1 to 13, and substantially as herein described with reference to the accompanying drawings.

19. A device as claimed in any one of claims 14 to 17, and substantially as herein described with reference to the accompanying drawings. -101-