C4 ara LViJY t:ufl uuuuvuvuuu rage z Australia Patents Act 1990 COMPLETE SPECIFICATION INNOVATION PATENT Invention title: 'Document Display (Screen Reading) System and Method D' The following statement is a complete description of the Invention, including the best method of performing it known to me. COMS ID No: ARCS-232017 Received by IP Australia: Time (H:m) 23:13 Date (Y-M-d) 2009-04-22 . nansu tuu caua UUUUUUJUUUUU Page 4 DOCUMENT DISPLAY (SCREEN READING) SYSTEM AND METHOD 2. This invention.renates to a system and method for refornatting documents for display on varying display types. In particular; it relates to a presentation system that performs 5 rendering and reformattinq of documents including Inaging and navigation. BACKGROUND TO THE INVENTION From the very easiest days of-computing, programmers have 10 been forced to target applications to particular screen eMs and resolutions. Even the flexibility of the Web's HTML imposes these kinds of limits, with some pages looking good on some resolutions while having a poor appearance on others. This situation was awkward yet manageable, whie most of the 15 world's information displays were confined to fixed computer monitors and televisions, most with industry-standard sizes and. resolutions. But now, with the inuoduction of lightweight low power consumption displays with wide size variation, such as liquid crystal displays and plasma units, the former display constraints are being 20 rerhoved. Nowadays programmers and content creators cannot make assumptions about the devices that will display the delivered information..To do so introduces inherent weaknesses in the way their content can be acesed and Iled th other information, Generally speaking, the IT Industry has reacted to this 25 challenge using traditional notions of 'device independence'. The seminal model of ths approach to content presentations Adobe's Page Description Format (PDF). This digital file type uses similar formatting commands across a multitude of devices, thus ensuring the correct display of documents on whatever system is used to 30 render the output to the user. This ensures the infomiulion looks the COMS ID No: ARCS-232017 Received by IP Australia: Time (H:m) 23:13 Date (Y-M-d) 2009-04-22 uuuuuuuuuuuu Page 5 same whatever resolution it displays on, with detail being degraded on less sophisticated rendering devices. However this detail is not lost if vilawed on a low resolution monitor, as the user can often zoorn in if desired. Although this method preserves the original fidelity of the information's look and feel, PDF files are static, typically being rendered to a pre-determined paper size. The end-user's device is expected to navigate around this. This can be dilficult, particularly if the original document was rendered to a size much larger than the 10 end-usersa device, such as trying -to look at e Foolscap page on a cell phone sized display. Intrinsic to this method of information distribution is the expense of converting documents from their original form to the description language of PDR Such conversions can limit a document's reuse, with recipients experiencing difficulty 15 re-editing and thus reusing the information in a similar form in different contexts. The problem is exacerbated in Web pages where formatting is mixed up with the substance of content. It is not always easy to determine whether portent Is a bitmap picture, a textural heading 20 bitmap, a navigational token bitmap or all or only some of the above, It is also difficult to determine whether a table is a logical unit of information or a layout separator. The ambiguity of display formats often compromises the ability of display devices to render the information in a way that suits the end-user environment. So file 25 formats such as PDF and HTML often result in compromised viewing situations when rendered by devices not originally targeted by a document creator. This is becoming Increasingly unmanageable in today's world of device proliferation, where advances in battery, communications and flat display technology are multiplying end user 30 display options. COMS ID No: ARCS-232017 Received by IP Australia: Time (H:m) 23:13 Date (Y-M-d) 2009-04-22 64 irn £UVY z3;V UUUUUUUUUUUU Paqe b To overcome these deficiencies, many in the IT industry propose a different-kind of device-independent format. Using standards such as XML, Informition presentations as diverse as text, pictures and program user interfaces are being described in a 5 manner whereby the substance of the content is stored separately from its layout or display formatting instructions. This provides the display device with the opportunity to make value judgments on how content should be displayed, both in the context of its use and the display environment in which it is to be rendered. While this may be a great step forward in solving technical issues it results in content creators; publishers and developers losing control of how information Is to be displayed. This is unacceptable because the look and feel of documents and programs is an essential ingredient for product differentiation. Moreover, content Is 15 often deliberately presented -by its creator in such a way as to give more or less significance to its various parts. So altering the look and feel of content for convenient display might also change Its intended meaning. Of even greater concern is the arbitrary degradation of 20 information that each end-user device may impose when rendering information on low resolution systems. The look and feel of the document, including the layout and navigation, is sometimes completely different from what is rendered on higher resolution displays. This forces users to leam a different way of using an 25 application or document on each class of device displaying the information. So while abstracting content away from its format is technically possible, in practice formatting and layout provides a contAxt which gives meaning to content. This has been addressed in 30 part by employing templates, which when applied alter XML documents to be more suited to their intended display environments. COMS iD No:ARCS-232017 Received by IP Australia: Time (H:m) 23:13 Date (Y-M-d) 2009-04-22 U00000000000 Pae 7 -LV This offers some improvemnt compared to static formats such as POF or- HTML Indidual document conversion and hard coding can be abolished, being handled once for each device end each document class Or-application type using a complex matrix of 5 templates, However using device and document templates with XML cannot eliminate all of the rendering disadvantages due to Multiple display sizes. The use of templates assumes developers and content creators can know in advance all the device types that will try to 10 render their Information. And having no control over these devices makes content degradation and user interface variation inevitable. This is caused by differences in capabilities between devices, as mandated by their sie, capacity of their power source and indented use, creating learning ourves to effectively present the same content 1U across multiple device types. Altering XML documents with templates this way may also compromise their reusabiity when passed to other rendering engines or editors on more advanced systems. Another problem that is not addressed by either flexible XML or fixed description formats is the formatting issues associated with 20 document readability over different viewing distances. For example, a billboatd sized screen may have four million pixels. A hand-held device may have the same. One user's eyes are 30 centimeters away; another 5 are 30 meters away, But even when two users look at the same device from the same distance, different magnifications 25 of the content may be appropriate, depending on the quality of their eyesight This is a key usability flaw in all device-oentricformatting regimes, be they fixed or flexible. The problems above are further compounded by modern end. users being connected through networks such as the Internet In 30 times past, most users accessed liforrhaioh from one -or at the most two devices - their desktop or notebook PC. But in a world of inter COMS ID No: ARCS-232017 Received by IP Australia: Time IH:ml 9-11 rutn vAA, Annauuuuuuvuvuuu Page connected devices, users may soon expect to be able to read documents located on their PC or business servers on any handheld device via a wireless link, for example a mobile phone. Therefore these problems are being encountered by users sharing documents, 5 as well as where a reader and author are the same pujrson The range of possible display sizes Information must be accessible through is further being proliferated through portal technology Portals assemble information on behalf of end-users, typically presenting it on Web pages made up of a series of 'info 10 boxes'. Even info boxes displaying the same information may vary in size, depending on how important each end-user considers the' contained information to be. Therefore the arbitrary nature of portal information viewing areas-means maintaining a consistent look and feel to content is becoming even harder to achieve. What Is lacking. Is a presentation system which strikes a balance between the competing interests-of end-user preferences, docurnent owner/creator desires, rendering device constraints and network transmission considerations. This must be achieved in such a way as to preserve the content's look and feel, navigability, 20 usabiity and reuse, with high fidelity across a multitude of devices. Prior patent documents have been identified that relate to the reformatting of documents. Reference may be had to United States Patent Application number 2001-0011364, in the name of Stroub. Stroub reformats documents into columns that hiive a fixed number 25 of characters per line, The number of columns is selected to suit a given screen. The approach of Stroub is useful for simple text but is of limited value for more complex content S-Another approach is described in US 6175845 in the name of IBM Corp. The IBM approach is to implement navigation between 30 parts of a document by inferring the significance of content to users. Not having a unique multi-user, multi resolution, multi-magnification COMS ID No: ARCS-232017 Received by IP Australia: Time (H:m) 23:13 Date (Y-M-d) 2009-04-22 u..uuvuuuuu Page 9 cache, the system refbrmats content each time it is requested by the user or when.the user resizes the viewing area. However content based analysis and per-view reformating on demand are particularly processor intensive mechanisms for the display of content As a result, this method is impractical for most applications, In US 5S1 8440 IBM Corp also describes a system for changing user interfaces principally document navigation, again based on analyzing the information being displayed. Using display software to change the user interface according to certain criteria 10 which is compared against the characteristics of the content on hand has some.glaring limitations. The method doesn't deal with critical factors influencing usability such as display area size and requires end-users to team multiple interfaces to view content on the same device. 15 The prior art does not describe a system or method which reformats doc'Oments for display on varying display types while minimizing repeat processing and optimizing display for optional transmission over a computer network. 20 OBJECT OF THE INVENTION it is an object of the present invention to provide a system and method for reformatting documents for display on varying display types. it is a further object to minimize repeat processing and 25 optimize display for optional transmission over a computer network. Further objects will b6 evident from the following.description. COMS ID No: ARCS-232017 Received by IP Australia: Time (H:m) 23:13 Date (Y-M-d 9la.nA uuuuuu...uvu. Page -0 '7 DISCLOSURE OF THE INVENTION In one forrh, although it need not be the only or Indeed the broadest form, the invention resides i a document presentation system comprising: 5 a rendering unit that analyses user or publisher preferences to determine content layout and navigation; a formatting unit that resizes and rearranges content according to the content layout and navigation determined by the rendering unit a display unit That presents content to a user and responds to user 10 interaction: and a cache unit that stores reformatted content for display by the display unit. The rendering unit may further establish device characteristics such as the presence or absence of external buttons, Joysticks or 15 speech recognition, altering the on-screen navigational features accordingly. Alternately, for simplicity or to maximize the content viewing area, devices relying bn such external navigation may mandate that no on-screen navigation featuies be made available to end-users, except perhaps within the content itself, such as 20 hyperlinks. In this case, the rendering unit might simply provide the formatting unit with the boundaries into which content is to be reformatted. The formatting unit may further perform content manipulation including one or more of blank space reduction, blank space 25 insertion text condensing, margin adjustment, and tab stop adjustment. The display unit may also incorporate processes for detecting system misuse, such as attempted unauthorized access and have functions to monitor the efficiency of reformatting and content 30 display. These may offer end-users various alternatives should an error occur, including using. information stored in a management or COMS ID No: ARCS-232o17 Received by IP Australia: Time (H:m) 23:13 Date (Y-M-dl 2009-04-22 uuuuuuuuuuuu Page 11 takeholder subsysteM (such as that described in our co.-pending international application titled "A System for.Secure Distribution of Electronic Content and Collection of Fees) to help appropriately restore the users dI4play. Additionally, the display is also responsive 5 to pre-defined 'copy-zones', whereby users can be otherwise .prohibited from selecting and copying content to their clipboard. In a further-form, the invention resides In a method of reformatting content for display on a display device including the steps of 1 retrieving content from a repository; determining characteristics of a display device that Is to dIsplay the content, Including the area (such as the nurnber and dimensions of pixels) available for display of the content and any associated control structures on the display 15 rendering content areas and any associated control structures to suit the display characteristics; and reformatting content for display of the content in the available content areas, said reformatting including resizing and rearranging, Suitably the reformatting includes: 20 identifying wrapping graphic elements and repositioning said graphic elements to avoid wrapping; Identifying section breaks and inserting blank space to push content to a next column; and identifying bleeding graphic elements and repositioning said graphic 25 elements to avoid bleeding. The Invention may have particular application to the presentation subsystem described in our co-pending application titled, "A System for Secure Distribution of Electronic Content and Collection of Fees, (AU 2002366661), herein Incorporated by reference. COMS ID No: ARCS-232017 Received by IP Australia: Time (H:m) 23:13 Date (Y-M-di 2014.P-nd.2 000000000000 Page 12 BRIEF DES'RIPTION OF ThE DRAWINGS To assist in understanding the invention, preferred embodients will be described with reference to the fIllowing figures In which; 5 'FIG I shows an overview of a presentatIon system; FIG 2 is a flowchart showing pagination steps; FIG 3 is a flowchart of the graphis wrapping process from FIG 2: FIG 4.is a flowchart of the graphics bleeding process from FIG 2 FIG 5 displays a document formatted for a PDA display; 10 FIG 6 shows the'same document as F1G.6 reformatted for a televlinn monitor; FIG 7 shows the some document as FIG S reformatted for a mobile Phone display; FIG 8 shows how one of the invention's a display units hnterets with 15 one of its Cache units: and FIG 9 shows the elements of a reformatted document and rendedrng unit components. DETAILED DESCRIPTION OF THE DRAWINGS Rbferring to FIG i, there is shown a schematic of a 2D presentation system In an appllcalion nsxt. Content Is provided to the system from, for example, a stakeholder subsystem (like that in our co-pending applioaon mentioned above) ta alioft preparation of composite documents containing one or more of text, Images, apples, scaipts and other material. The stakehloIder subsystem will 25 normally also store a master copy of the document. The composite document is passed to the presentation system that consIstsof a number of units including a rendering unit, formattng unit, cache and display unit The presenttion system presents a COMS ID No: ARCS-232017 Received by IP Australia: Tima W-m -n- fl..f. Iv .
-- 000000000000 Page 13 10 refornatted version of the document to an end user for local display or to a display server connected over a computer network to the end-user's display-device viaremote display protocols. Communication with other subsystems may be effected by 5 XML or other messaging, as desenbed in our co-pending application. As depicted in FIG 1, the c6mposite-document is converted to a basic frame structure based on information obtained about the users display. Navigation and layout elements, sch as arrows, page and column links, etc are added within the available display-space. The 10 rendered document is then reformatted to suit the display by rearranging text, graphics, and other elements to satisfy viewing preferences defined by the originator and/or the user, taking into account the characteristics of the display device. The document is stored in a cache for future use In a location 15 which may also identify the display characteristics of the stored document. This leads to a novel document retrieval process as described in greater detail below. A copy of the reformatted document retrieved from cache is displayed to the user. The Presentation system operates in four stages. It consists of 20 four units, for display, cache, fonnatting and rendering of documents: The Display Unit is responsible for maintaining the end user interaction and imaging documents for local display or network transmission using remote display protocols. The display unit also incorporates a number of features lo detect misuse of the system. 25 To help facilitate the previously mentioned portal 'info-box' scenario, the display unit may suitably occupy the entire area of the end-user's screen or may be embedded within a portion of a document or application user interface. The display unit itself may also divide its viewing area into sections between a. number of 30 screen-paginated, fixed sized or traditional scrolling contenL And it COMS ID No: ARCS-232017 Received by IP Australia- Timn (Wnlm fA-1q rl# I A .an 000000000000 Page 14 10a may also allow ond-users to adjust the dimensions of such content areas, or do so automatically, providing more display area for less scrolling of bleeding content not fitting in a column. The multi-resolution, multi-magnification document caches 5 feed documents to the display unit from the cache. This minimizes the need for processing by reusing previously formatted content, If a document is not available in the cache to suit the combination of the end-user's device, magnification or other publisher or end user preferences, an attempt may be made (by either the cache or display 10 unit) to find an appropriately reformatted document in another cache COMS ID No: ARCS-232017 Received by IPAustralix Timo rm-I 2-12 f.@ Io k.
000000000000 Page 15 It If one cannot be found, the formatting and rendering units are activated to provide one. The formatting units minimize the need for end users to scroll though documents by dividing them into smaller; more manageable 5 sections. These reformatted documents may or may not fit completely on the screen, depending on how the content has been marked up, the device size, how the contenf or its images are being viewed, and publisher and end-user preferences. The reformatting process resies and moves content to an appropriate size, as well as 10 employing various 'blank space' reduction and insertion techniques. These include condensing text, reducing tab stops, adjusting margins etc. The basic layout of the documerit (such as the number of columns) in which this is done i predetermined by a rendering unit, The rendering units examine the dimensions of the available 13 viewing area, and analyses end-user and publisher preferences to determine a suitable document layout and navigation system. This basic layout is then used by the formatting unit to measure, and manipulate content. The rendering unit may fill the entire available display area or 20 ronder in dimensions or magnifications wilch are a near match to an end use's or publisher's preference or display device characteristics. Ukewise the display unit can also, request documents from the Cache which are a near match to Its available display area or desired magnification. Using existing reformatted content nearly matching the 25 display aroa.size or desired magnification can speed up document access for end users and increase computing efficiency by reducing the neead for repeat reformatting. The system if required, can determine the next near match incrementally, such as providing the largest.rendering dimensions fitting the display, when made available 30- in fifty-Pixel x and y axis or at 20% magnIfication level increments. A Cache unit may offer a display unit such an available near match if it COMs ID No: ARCS-232017 Received by IPAustrali Timn 1-..a f. .
ZZ AK ZUU9 zj:U1 UUUUUUUUUUUU Page .i cannot yet fulfill the exact-request. Or end users may be given the choice between a near match aind a document exactly fitting their display device and viewing preferences; A near match may also be offered to an end-user for example, while a large document takes 5 time to be reformatted nearer or in accordance with their exact specification. Information pertaining to end-user interactions, Such as document navigation, may be encrypted. This prevents unauthorized display units or external viewers from impropedy accessing content. 10 Documents may- also be password protected to prevent opening by unauthorized display units or viewers. The rendering-unit and formatting unit perform a number of steps that can be best thought of as a pagination process that is shown in FIG's 1-3 and described in greater detail in the following 15 steps: Pagination Procedure The following steps are-suitably performed by the rendering unit 1. Create a new location (such as a file-system directory) in 20 cache with a name or identifier based on documents name, screen resolution, magnification or an indicator of the intended displayable columns and their sizes; 2. Copy document from publisher repository into newly created cache directory; 25 3. Check the document for publisher preferences 4.. Open system's master rendering document; 5. Use current (end-user) display settings and publisher preferences to set basic frame structure. COMS ID No: ARCS-232017 Received by IP Australia: Time (H:m) 23:13 Date (Y-M-d) 2009-04-22 00000U0OUO Page 17 13 The following steps are performed by the formatting unit: 1.. Open the cached document for reformatting; 2. Scan for potentially hostile or problematic contributed code, such as embedded applications and macros; disabling where appropriate 3. Delete all hyperinks from document for navigation encryption, replace with colored underlined text; 4. Remove empty paragraphs, adjust inter-paragraph spacing; 10 5. Compres or expand tab stops, Indentations etc, optimizing document for specified column width and magnification; 8. Optionally group graphical elements which overlap, so as to maintain their relative position with one another during 1$ the rest of this procedure 7. Reduce all graphical elements larger than a column's width or height to fit except those marked-up asscrollabe; B. Adjust width of caption boxes; 9. Check for section break(s) on page, if so insert blank 20 space to start content in next section in next column; 10. Unless marked-up to allow wrapping, detect if graphical elements on screen horizontally 'wrap' next to each other, moving them down so as to appear vertically separated down the document; 25 11. Check for graphical elements which bleed off the botbrn of the display area and: a. If element Is marked-up as 'un-movable' then do nothing, or COMS ID No: ARCS-232017 Received by IP Australia: Time (H:m) 23:13 Date (Y-M-d 200:-nA..2 Li. nra 00000 0UUUUUUUOUU0 Page 18 14 b. If text-above element is of the same attributes as the text next to and below it, move element up-the page to fit it-on the screen if possible, or c. Move the element to the next screen page or 5 column, inserting blank space or moving any text after the shape into the preceding column if appropriate; 12. Record (optionally encrypting) top and bottom visible column boundaries in relation to the current area of the 10 document being displayed (by the number of points from the top of the document for example) or a reference to the document's first and last currently visible characters (see FIG 9); 13. Remove any previous blank space buffers; 15 14.Insert blank space 'buffer' after previously established last character.or column bottom boundary position; 15. Move document up so blank space buffer is at top of screen-colurmn; 16. Write progress to cache location for multi-user splash 20 screen reports; 17. Repeat steps 13 to 20 over entire document; 18. Add more blank-space to the bottom of document; 19. Break document up into more manageable sub-document chunks If necessary 25 20. Suitably add Icons and controls to screen areas used for navigation,. as provided by the rendering unit, according to the number of columns paginated in the document and the number of columns that will be simultaneously dlsplayod on the screen; COMS ID No: ARCS-232017 Received by IP Australia: Time (H:m) 23:13 Date (Y-M-d) 200-nd-9 SAPK ZUU9 u3:ui UuUUUUUUUU g 1: 15 21.Create default summary information from first paragraphs if it has not already been included by the- content's creators 22 Save the changes to the cached document and associated frame set and navigation documents 5 A certain amount of set-up is required to capture user information if maximum advantage Is to be obtained from the method. The information can be captured in a new session process according to the following steps: New Session Process 10 1. Start session / Required start-up document name and location is passed to system; 2, Display full display area splash screen, turn off user interaction; 3.: Obtain username and password if required, evaluate and 15 'grant access; 4. Lock end-user display, stopping e11 input; 5. Create end-user cache if one does not already exist; 6. Load settings from any previous settings oridefault settings such as: 20 a. To-Do lists b. History lists c. Magnification levels d. Number of column preferences e. Searches 25 f. Home page location g. Contents page location COMS ID No: ARCS-232017 Received by IP Australia: Time (H:m) 23:13 Date (Y-M-d) 2009-04-22 uuvuuuuuuuuu Pagq 20 15a h. Optionally load the last used document [ Various other settings; COMS ID No: ARCS-232017 Received by IP Australia: Time (H:m) 23:13 Date (Y-M-d) 2009-04-22 ZZ APR UU9 23:08 000000000000 Page 21 7. Create end user command structure based on available display- area: icon menus with words for larger screens k. loon menus without words for smaller screens 5 L No loon menus (or a reduced set) for some very simple devices such as public kiosks or miniature hand4elds, where navigation and commands might be implemented externally as buttons or joysticks on the device; [0 8. Open start-up document using Open Document Procedure. Open Documnent Procedure 1. User clicks on hyperlink or document to be open Is supplied from other procedure; 2. Obtain usemame and password (I required); 15 3. Check management system for permission to open document (if required); 4. Check if document exists in Publisher's reposltor'; 5. If document is not found, display error message or.page; 6. Display full display area splash screen, tum off user 20 interaction; 7. Close all currently closed documents; 8. Select navigation system according to in the following order of priority, as marked up in document or known by system from end-user inpui: 25 a. Display area restrictions 1. Ascertain or calculate maximum width and height to be used for columns in the display area COMS ID No: ARCS-232017 Received bv [P Auntralia- Time (H -m 23-13 nate (Y-M-di 20flDL22 000000000000 Page 22 iI. Establish the range of rhagnifications permitted In the display area b. -Publisher compulsory preferences i. Scrolling only or scrolling with pagination 5 Ii. Magnification level iii. Number of columns c. End-user preferences i. Scrollirg only or scrolling with pagination i, Magnification level 10 iii, Number of columns d. Publisher's default display preferences i. Magnification level ii Number of columns; 0. Checks if file is cached (see FIG 8), if not, checks if 15 document Is in the process of being paginated and waits or, if not, invokes Pagination Procedure; 10. Determine filename of cached document appropriate for the above navigation system; 1'. Checks If cached file is newer than polished original, if not 20 invokes Pagination Procedure before proceeding; 12. Optionally copies document into end-user specific cache; 13. Loads document into each column; 14. Determines appropriate position of content to be displayed in each column by: * 25 a. Programmatic control, as in the case of forward/back navigation from previous documents or b. Default first and second pages or COMS IDNo:ARCS-232017 Received bv IPAuntrali-ri , -- 4') r C.4 fl VV * Lj;uo 000000000000 Page 23 c. Bookmark position specified by user or d. Boukmark position specified by hyperlink; 15. Decrypts position in document of required columns; 16. Moves (i.e. scrolls) document to populate columns with 5 required content; 17. Insert additional information in document, such as automatic links to other content; IS. Load virtue sidebars with marked-up content; 19. Enable scroll buttons if any content exceeds on-screen 10 column dimensions; 20. Update history list 21. Update to-do list 22. Update command structure (menu) options; 23. Remove full-screen splash screen and enable user control 15 24. Inform management system of successful display; 25. Inform management system of display of monitored items on screen; 26. Start next document preloads if required. The order of steps described may be changed or some steps 20 may be omitted to optimize the system for particular uses. Where these processes occur in memory or behind splash screens, the terms 'visible' or 'displayed' are for explanative convenience only, refening to that part of a process which could be seen should an end-user be allowed to view ito 25 ~ The command structure and navigation set-up by the . rendering unit may also be linked to end-user device hardware, such as page and arrow keys or a microphone (or associated speech COMS ID No: ARCS-232017 Received by IP Australia: Time (H-ml -'j-- $ r Vt. v.._, ->r 000000000000 Page 24 recognition engine) for voice-activated navigatafn, It may also be accessed programmatically by other third-party applications. At various times during rendering, formatting or display, all or . parts of documents may be removed from video or program memory 5 - 'made invisible'- so as to speed up or otherwise optimize their manipulation by the system or end users. The outcome of the system and method can be seen clearly in the examples of FIG's 5-7. A master document was produced by a stakeholder subsystem to contain a graphic element and a number of tO text blocks. In FIG 5 the content is reformatted for display on a mobile device, such as personal digital asistant/communicator combination. The navigation elements described above are evident at the bottom and top of the display. The navigation elements include navigation arrows as well as page or column numbers. 15 - FIG 6 shows the same content reformatted for a television Screen. it should be noted that the magnification is larger and repositioned with respect to a larger image. Similarly, in FIG 7 the same content is reformatted for display on a mobile phone-sized screen. 20 The presentation subsystem may be used in a commercial publishing environment In the following manner: 1. Stakeholder subsystems provide information to the interface subsystems allowing links to the presentation subsystems to be published in Web pages; 25 2. An end-user requests to see a document by clicking on a link; 3. The interface subsystems forward locational information to the management subsystems, such as IP address or vocational information from an end-user device; COMS ID No: ARCS-232017 Received b IP Australix Time (H- 1-m'z- rl-t rv RA ,nnn -- a uv ooa000000000000 Page 25 4. A logon session is created (such as via a secure Web page or remote displayprotocols or locally from the invention's own Interface) to find out who the user is in relation to the publisher, which If succeassfl, causes the management subsystem to create a unique S session number for that users session with the presentation subsystem; 5. The logon session forvards locational information to the Management subsystem; 6. The display unit forwards locational information to the 10 Management subsystems; 7. The display unit finds the document in the Cache unit. If the document is not in cache, the display unit obtaiis the document's location from a management sub-system, allowing it to draw on the publisher's master document repository. The management 15 subsystem evaluates the request authorizing it according to the business models associated with the docunient; these business models may include invoking substitute content based on the provision of such locational Information, 8- Once authorized by the management subsystem, the 20. display unit seeks a document from the cache corresponding to the display area, user and publisher preferences; 9.. If one cart be found (the majority of cases), it is loaded and displayed for image transmission, if not, the rendering unit is engaged to determine an appropriate layout and navigation system 25 and the reformatting unit then handles the content accordingly' 10. The caching or display unit may attempt to obtain a' suitable document from another's Presentation subsystem's cache, rather than initiate fresh reformatting 11. After beiog loaded from the cache, the document image 30 is displayed via remote display'protocols; COMS ID No: ARCS-232017 Received bv IP Australia, Time (H-mi -1a r- rvA_, ,nnl nA uuuvuuvuuuuu age zb 21 12. The display unit accepts navigation commands from the user; 13. The display unit monitors end-user activity in relation to time or session charges as controlled by the management 5 subsystem; -14- - The display unit monitors 'revenue coontent' such as . advertising or any content usage patterns of interest. informing the management subsystem of details of its exposure; 15. The display unit also provides account management 10 tools and activity reports to end users: 16. The display unit also provides end-users integrated searching between sites, documents and the content within a document; 17. The display pnit also provides histories of previously 15 seen documents, favorites and 'to-do'lists of must-see documents; 18. Fram time to time, the caching unit may delete from itself infrequently used documents. Conversely, it may be pre-loaded with previously formatted documents in anticipation of heavy use. This may be implemented under Management system control 20 according tothe business models associated with a document. The above scenario would not contain as many steps if the invention is used to display documents on an end-user's machine to their cell phone over the Internet. Organizations running the publishing system in-house may also require fewer steps. Depending 25 upon the implementation, the order of steps may also vary. However In all cases, the unique four-stage architecture df the presentation subsystems remain the same. The Cache, display, formatting and rendering units may be deployed on different machines for load balancing purposes. COMS ID No: ARCS-232017 Received by IP Australia: Time (H:m) 23:13 Date (Y-M-d) 2009-04-22 uvuuuuuuuuuv rage .zi 21a It will be appreciated that the presentation subsystem, and in particular the display unit, is not limited to on-screen display: For COMS ID No: ARCS-232017 Received by IP Australia: lime (H:m) 23:13 Date (Y-M-d) 2009-04-22 22 APR 2U09 23:09 000000000000 Page 28 example, coti~gat mayW be em't dirod to a camm=tWa pfmt~r ftr Orte-ftne hard copy prlrfig, to a sound stud lo for onie-time INsonig ur to a cinean for v~emkii. COMS ID No: ARCS-232017 Ppnivdr hv IP rt,,I. IV~~ RA*~ Ai -)nrfn (VM A ~AO 22 APR 2009 23:09 000000000000 PacqQ 29 ~Ila In all the embodiment of the inention described, a 'screen' also may be widerstoad ." the display owe in which a dmclnw is iD be kw~ed, auch as within a boxmb a Web page. Also the taim doaimer't' may aisQ refer to other contont sch as video or audio associltOd with docunt It wil also be appreciatd thot the presentation system QptimilzB docijrnn for docamnsn imap trenailsaon ard disply Qvei ximputer networiw. These opnizatitma lndqde:. 1. Suppresng screen updates during documnt reformattiag wlti the Judkcioliu use of f d-displey areas plesh-smeens; 2. StappressiM screen updates during docut~rf nX0ig110n using otwnn-.sked slash-eens; 3. Supprussing sowee udat by teducWi smreling through paginatin ior available display atus. WMere scrOWWig Is required, the system lmandate scrolIng be done in slap rather #=a gradually. All this'is aOllved by moving content up or Wo the neot coliim wbsre pabe. *hile sti llwing individual elements to'bleed' (a puii~ls!"ri term) off the base uf the screen Wo scrolling at the awne tme, whom. size reduton or nxwement Is inappropriate, Oveisize pictures may also be reduced and enlag by all~diig on them. Thus on a single pagp same content may be moved, same conent may be'nuued, some oonlent will Kit while somet my be Allov&d lo bleed for. scrcilngt Thus text will hardly ever need to be mconle Yweas. ftnwduLxbl cadtent (Aub-deata Or marked up as such) sach as maps which ars bigge thm Me~ scree) will Ampay need to be broflad to get the Mil Picture. However whome a bnig skinny strip map appears nuxt to text, the kodAxif be pagiated to fft on te scren WHlO V* strip map Is 'bfod oWl Wa scroing (TN& is twique to an automated 22 APR 2009 23:09 000000000000 Page 30 system.) Thus and users might never need to scroll to read the main body of text and not be interested enough in the ovcrsiwd material to bother scrolling (See FIG 9); 4. Suppressing screen refreshes byfiltering end-user commands 5 likely to have been caused in error, such as over-scrolling due to response time delays over the network; 5. Suppressing screen updates by using 'direct! navigation techniques, which avoid flicking through columnular screen pages to see the desired text, such as between the 'found' 10 words within a document resulting from a search. This reduces the amount of screen refreshes before the desired portion of text is found and makes the end-user experience much quicker; 6. Provision of direct page navigation icons, Go To Page 15 commands and bookmarks -.. Provision of 'next found' and 'previous found' navigation between documents reduces having to redisplay search results documents to go to the next result; 8. Provision of 'Next' and 'Previous' screen navigation to 20 compliment the more traditional 'forward' and 'back' navigation. This has the effect of coordinating inter-document with screen page navigation within a document, reducing the need to start viewing a document from its beginning, The four button approach allows the system to automatically return 25 users- conveniently to the exact screen-page in a document where they were viewing It last So Forward and Back buttons replay where a user has been, While Next and Previous tuttons allow screen page navigation to deviate from the previously established paths; 30 The presentation. system also provides particular advantage for viewing docurnenis in small windows, such as those allocated by an information portal in a typical Web page containing other comSc in mn- aQ,3nv 7 o- -. .. f .. .- ~ , ~ ..
22 APR 2009 23:09 000000000000 Pag 1 20W information. In this scenario a number of deployment options are possible, including: 1. The entire presentation system can run on the end-user's computer platform as their personal reformatting serVice. This makes sense.when the information within a document has been customized for Individual use, requiring reformatting on an individual basis; 2. Being processor intensive; reformatting May be done on a separate server, while display and.caching of an individual's t0 documents remains on their machine oron-a dedicated display server; 3. The display remains on the end-users machine, but to eliminate the same reformatting of a document many times, caching is centralized on a fileserver To further reduce processing, if a document is not in the cache and is presently being reformatted for inclusion.in the cache, further requests will use tho same output without starting a duplicate reformatting process; 4. The entire system can be run on a server, displaying on end 20 user display areas via remote display protocols, being a convenient all-in-one implementation for smaller organizational units; 5. The display unit can be run on separate servers to the other units within the presentation system. This allows computing 25 resources to be allocated on the basis of need, with the faster processors within an.organization being dedicated to. reformatting. Another use of the presentation system is to browse documents In a similar manner to the way in which web pages are 39 browsed, only with better end-user control. This can be done without the need to convert the documents to HTML, thus promoting COMS ID Na: ARCS-232fl17 Rennivnrk u A,,.,um Tim - f ~n r tA n Ze APR 209 23:09 000000000000 Page 32 information reusablity, The presentation'systern provides the following features to support this application: 1. Auto-contents page generation, allowing the flagging of documents to be placed in a to-do list, plus instant summary .detail Information; 2. Auto-search page generation, listing results in the directory hierarchies in which they were found, thus preserving their context in relation to other documents and topics, Such searches can be added or subtracted to each other, as well as 10 having the to-do and summary features contents pages. Searches can be preserved for later viewing or updating; 3. Automatic document usage tracking, allowing administrators to know exactly which documents were viewed by whom and for how long. This is able to use standard network logon information and standard documents, eirninating the need for running a separate management system or membership services. The presentation system can also be deployed in an ASP scenario, In this scenario, end-users send their documents to a 20 service provider, to be custom formatted by the system to an appropriate scrsen-size and magnification. Typically, the service provider may also cache the document. The results are then viewed on display servers via remote display protocols or on the system's display units depbyed on local machines. Features supporting 25 software as a service Include: 1. Document formatting and cache usage tracking - The system allows monitors how documents are drawn from the cache, and provides detits of document usage such'as usernames, IP addresses, times, format popularity and the like in various 30 logs or to a messaging system. This allows the system . operators to better manage and charge for their services, COMS ID No: ARCS-232017 Pniuns , in A...... -r--- ,u.
22 APR 2009 23:09 000000000000 Page 33 2. Multi-user support - The system uses three kinds of caches. A general multi resolution & magnification cache is used to store previously formatted content This may also include temporary storage areas to be used in the process of document formatting. Copies of documents may be placed in an end user's or the display unit's current session cache to enable their personalized use while avoiding contention issues with other users. Finally, a user-cache stores user-defined content that needs to span end-user sessions, such as search results 10 or personal billing and usage reports. In many cases, these may also be stored in concentrated format to be reconstituted on demand by the presentation subsystems for user display purposes. In this way, the bandwidth and disk storage space requirements needed to support thousands or even millions of t S centralized user caches can be minimized, 3. Enhanced document security - Traditionally documents can be password protected and sometimes encrypted to avoid unauthorised use. However, once the document is in a computer's memory it is usually fully exposed, allowing it to be 20 easily copied by hackers. The current.invention limits the usefulness of documents in memory for unauthorised use by encrypting all the references and document structures used by the navigation features described above. These are stored separately and added back into the document as required and 25 decrypted only as they are used; meaning a full version of the content is never loaded entirely into memory at any point in time. The result is a formatted document that Is practically unnavigable by any other system with very economical use of processing power, while providing better protection from 30 hackers. This Is particularly so In.the cake of commercial online publishing, where the content is being displayed for image transmission by a closed system at an Application 22 APR 2009 23:10 000000000000 Pag Service Provider, making the above measure.precautionary rather than a primary means of defense. 4. Anti-donial of service mechanisms - Because the system monitors the use of documents by users, it is also able to search for signs of suspicious activity, such as excessive use of the invention's navigation features or repeated failed logins or repeated 'non-revenue producing sessions'. 5. Management system interface - Part of the system's resistance to attack and maintenance functions for ongoing 10 availability and reliability are a range or self-monitoring functions. The invention also monitors itself for unusual conditions or behavior and may forcefully lock the user interface when appropriate. Access to the rendering, reformatting, cache and display units may also be restricted, 15 requiring authbrisation from an external management system before processing can occur. Based on how it is being used, the system may also automatically or under management system control take remedial actions such as sending information messages, shutting down all or the affected parts 20 of itself, receiving instructions form a management system, disconnecting from a user or restarting. The display unit of the system allows the running of applications embedded within documents. These may include image boxes where the images of applications sent via remote display 25 protocols are viewed. Thus an instance of the system can be embedded within itself if so desired, Applications contained within the system have tremendous reach, with. their accompanying information being suitable for display within a multitude of devices and usage scenarios. Such applications may also be launched in separate 30 windows via a hyperlink or button. The system's application support features include: COMS ID No: ARCS-37n17 Reniudel 1o Amictrolia. Timo nImn 21-1 natv I naru, 22 APR 2009 23:10 000000000000 Page 35 1. Embedded applications - Embedded applications can be monitored by the system, sending their time (on screen or running'times) or CPU usage details to a log or messaging. system. Additionally, any piece of-content can be niarked up for such monitoring, including pictures or text .2. Contributed code - It is common for people to create their own Web sites, launching them unsupervised with the ISP. But online applications are orders of magnitude more expensive to . create and maintain.-The system addresses this problem by 10 allowing macros to be contributed, automatically scanning source code for potentially harmful algorithms or system calls. This code may then be executed in an interpreted environment or compled into an 'authodsed' application. Thus the system supports the execution of publicly 'ontributed 15 code, without the need for human supervision or risk of compromising the system. 3. Applications may also be launched in separate windows via a hyperlink or button, with their usage details also tracked by the display unit. These could be run in an execution environment 20 outside of the system, optionally giving the externally linked program a unique name, tracking its operation using the operating system's process monitoring interfaces. 4. Content pro-loading - In an application environment, it is often easy to predict a user's next move. Therefore to speed up 25 interaction, particularly where custom formatting may be required before display, the pro-loading of documents Is desirable, The system may support this with third party application programmer interfaces, end user commands or its own predictive algorithms. 30 The system can also be used as an advanced document Viewer, making the reading experience mor4 aesthetic by presenting COMSID Na: A~r'S-2q'0.n17 Poato k. DA,.I;.f..L1. 14 n .F, 22 APR 2009 23:10 000000000000 Page 36 naterlal in columns and minimizing scrolling. This is of particular use in making long documents more viewable oi-screen, which would otherwise normally be printed. The invention's desktop viewing support features include: 5 1. Caption resizing - Caption boxes (sometimes called balloons) accompanying pictures must sometimes be resized to fit on small screens, Howevdr this may reduce the size of the text until it becomes unreadable. Instead the system resizes the box in which the caption appears, extending it downwards until alithe text becomes visible. Therefore in some dases reading the text in cases will require the entire document to be scrolled. In order to avoid this, caption boxes may be optionally extended downward but only to the base of the viewing area, with scroll bars then being provided within the 15 captior box to view the remaining caption text not visible on the screen. 2. Blank space formatting reduction & expansion - The reformatting of documents, particularly for small screen devices, often requires a reduction of blank'space. If a 20 document is formatted with tab stops or indentations, when displayed on small screens these can potentially fill most of - the viewing area with blank space, causing text to wrap after only a few words. However often such indentatidns should not be removed, as they indicate the relative importance of one 25 piece of content with the next. On the other hand, the same document viewed on a large screen, particularly With negative magnification, needs blank-space for readability. Therefore the system is able to reduce the amount of blank space in some situations, while increasing it in others, providing the best 30 readability.for the display area provided. The system does this automatically or using publisher preferences using a number rlflMSQ Ifl M- ADrC~2 9nI 7 D -i RA.I .. *-.fi..'...(... 22f.*.L Al WnO0 . u o000000000000 Page 37 Of, Methods, iuncIudinig adjusting qpargins, Indentations and tab stops, 3. Blank space creation - in Some situations it 1s desirable for the system t& Pad a doc1rriert with addltIonatlank space. For 5 example, a large pictures at the end of a document may be better displayed fully in the next column than force the User to scroll. In this situation the text at the end of the document would finish before the end of the column, requiring padding with enough blank space afterward to push the picture to the 10 next column. (Blank space has to be inserted into computer generated content, which is unlike a paper document, where blank space is present befbre prinig occurs.) 4. Iconic command structures - A desktop PC environment will probably have enough room to display a command selection 15 system, such as menus. or hypeujinks, which contain words. However this Is not usually the case in smaller viewing areas, suich as within a box in a Web portal or on a mobile device. Desktop users should not be forced to learn a new interface because screen space precludes the efficient use of wordy 20 commands- To bridge the interface consistency gap between small and large display areas,'the system uses lcons in as' many command structures as possible; autornauically Presenting only the icons on smaller display areas. The function Of these commands can alWays be accessible via 25 mouse-over tool-tips, however the loon-only representation of commands means Identical command structures can operate on both large and small viewing areas, minimizing4he learning curve to operate the: invention over different iewing enlvironments. 30- 5- Watchword navigation - A user or content creator may create a list of words or phrases to be searched upon, These are COMS ID No: ARCS-232o17 Received hi, EP Aatn; .... Ij.. .
22 APR 2009 23:10 000000000000 Page 38 5M a'utomaticaiiy hyper-linkid to the search document to a matching phrase's occurrence in documents. 6. Document splitting - The longer the document'and the smaller the display size, the longer refrmattlng taket to occur Additionally, the larger the document, the rmore system resources such as memory are required to display it. Therefore it makes sense for the system -to break longer documents into smaller chunks for quicker and easier' flandling. This is done by using the natural breaks in a 10 document created by pagination into scretn-colurnns. This automatic breaking up anrd presentation of documents as an unbroken whole is transparent to end users, but helps speed up document initlaiasation times. 7. Virtual sidebars - Magazines often contain supplementary information in sidebars, however these don't translate easily into readable documents on screen. The problem is further compounded in smaller display areas, which have barely enough space for the main body of content to be presented. The answer is to use pop-out sidebars, which are actually 20 another area of the same document presented in a window which slides out from the edge of the screen, sliding back again when no longer required. Optionally being a part of the main document, virtual sidebar content may also be screen paginated. Content can also be inked-in virtual sidebars, 25 effectively creating pre-loaded hyperlinks for greater speed and efficiency, only also with greater accessibility. 8. Word processor companion - Perhaps the best place for a desktop implementation is within or connected with a word processor. After creating a document, often it is best to proof read it on a printer. By paginating the document into a'clearer, more readable form, more proofing can be done using the COMS ID No: ARCS-232017 Reeived hv P Aimtrlin Time (Hmi 2l1 natt (v-l 9flo-rl-22 22 APR 2009 23:11 000000000000 Page 39 same application a document was created in, simply by. selecting a 'Screen-Page' item in the View menu. This saves time waiting for print outs as well as printing aid network bandwidth costs associated with printing In large corporations. 5 In all cases, the term 'picture' may refer to any graphic element or embedded application within a document. Throughout-the specification the aim has been to describe embodiments-of the invention without limiting the invention to any specific combination of atternate features. 10