AU760816B2 - System for capturing, annotating and transmitting images of internet web pages - Google Patents

Description

WO 00/07110 PCT/US99/17432 SYSTEM FOR CAPTURING. ANNOTATING AND TRANSMITTING IMAGES OF INTERNET WEB PAGES TECHNICAL FIELD The present invention generally relates to the art of electronic communications, and more specifically to a system for capturing, annotating and transmitting images of internet World Wide Web (WWW) pages.

BACKGROUND ART As a facet of the exponential explosion of digital computer and electronic communication access a huge volume of information and also to communicate directly with each other. This network is known as the "intemet", and consists of a large number of computers which have information stored therein and are interconnected by telephone, satellite and other data links.

These computers function as "servers", and can be accessed from an external location by a user with a personal computer to enable the user to obtain information stored therein. Most users connect their personal computers to the internet through service providers, although others are equipped to access the internet through direct connections.

The most widely known and used functionality of the internet is known as the "World Wide Web In this system, the server computers are designated by addresses known as Uniform Resource Locators (URLs) and present their information to remote users in the form of "web pages".

These web pages are created as text document files in the HyperText Markup Language (HTML), and include embedded hypertext links to graphic images, other URLs, etc.

A remote personal computer user accesses the WWW using a software program known as a "browser," several of which, such as Microsoft* Internet Explorer and Netscape* Navigator, are currently available to individuals at no cost. The user types the URL to be accessed into the browser, or clicks on a link or icon representing the URL using a mouse. The browser then communicates with the URL over the internet and downloads -a WWW page which is displayed by the browser on the computer monitor screen.

The data which is actually downloaded from the URL to the user includes an HTML document file and graphic images which are referenced or pointed to by hypertext link elements in the HTML file.

The browser interprets the HTML file, and generates and displays an image of the WWW page on the screen including any graphic images which are inserted at the locations designated by the elements in the HTML file. The displayed page can also include hypertext links in the form of highlighted text, icons, buttons, etc. which can be clicked on using a mouse to download another web Page, image, etc.

from the same or another URL.

It is desirable for a user to be able to save a complete web page and/or send a copy to a friend or business associate. However, a user-friendly way to provide this capability has not heretofore existed in the an.

Although browsers provide an option to save a web page under a filename, all that is saved is the HTML document file. Graphic image files which constitute parts of a displayed web page. for 2 example, are not saved with the document file. Although graphic images are temporarily stored in a cache directory, this directory is periodically purged.

If the saved HTML document is loaded into the browser after the cache is purged, the images will be missing from the displayed page. In order to restore the images, it is necessary to download them again from the Server (the UYRL is the address). Another limitation of conventional browsers is that they do not have any capability to annotate web pages prior to saving them, or save selected portions of web pages.

If a user wants to send a copy of a web page to another person with annotations, it is necessary to perform a procedure such as printing out the web page, annotating it manually, and faxing it to the recipient.

This is very cumbersome, time consuming and "low-tech", and manual annotations made by a typical user can be oooo seriously lacking in artistic. quality.

For this reason,'there is a pressing need in the art to provide a system which will enable an internet user to save, annotate, and transmit a received web page in an efficient and aesthetic manner.

DISCLOSURE OF THE INVENTION According to an aspect of the present invention, there is provided a programmed digital computer system, comprising: a display unit for displaying a hypertext document; an input unit; a processor; and a storage for storing a program which is executed by the processor, the program including an annotator for annotating the page by modifying the document in response to input from the input unit, said annotation including painting an image on the page at a position designated by the input unit.

H:\flanagan\keep\SPECIFICATIONS\51367-99.doc 21/03/03 3 BRIEF DESCRIPTION OF THE DRAWINGS FIG. 1 is a block diagram of a programmed digital computer which executes a software program according to an embodiment of the present invention; FIG. 2 is a diagram illustrating a World Wide Web (WWW) page displayed on a monitor screen of the computer of FIG. 1 according to an embodiment of the invention; FIG. 3 is similar to FIG. 2, but further illustrates a pull-down file menu; FIG. 4 is also similar to FIG. 2, but illustrates annotation of the WWW page using a graffiti pen; FIG. 5 similarly illustrates annotation by highlighting text; FIGs. 6 and 7 illustrate pasting a cyber note on 15 the page; FIG. 8 illustrates blocking text for copying to a 9* 99 scrapbook file; FIG. 9 illustrates a scrapbook file including the text blocked in FIG. 8; FIG. 10 is a block diagram illustrating an overview of the software system running on the computer of FIG. 1; FIG. 11 is a block diagram illustrating the main S* :component elements of a software program according to an embodiment of the invention which is part of the system of FIG. FIG. 12 is a block diagram illustrating a main module of the present software program; FIG. 13 is a block diagram illustrating annotation files representing digital stickers and cyber notes which can be pasted on displayed WWW pages using the present program; FIG. 14 is a block diagram illustrating the components of a WWW page which are stored in accordance with an embodiment of the invention; FIG. 15 is a flowchart illustrating a system tray operation of an embodiment of the invention; \\melbfiles\home$\flanagan\keep\SPECIFICATIONS\51367-99.doc 21/03/03 3a FIG. 16 is a flowchart preprocessing operation; FIG. 17 is a flowchart pen annotation operation; FIG. 18 is a flowchart highlighting operation; FIG. 19 is a flowchart pasting operation; FIG. 20 is a flowchart mail transmission operation; FIG. 21 is a flowchart operation; and illustrating a illustrating a graffiti illustrating a text illustrating a cyber note illustrating an electronic illustrating a printing o• FIGs 22 to 25 are block diagrams illustrating server-based embodiments of the present invention.

BEST MODE FOR CARRYING OUT THE INVENTION STRUCTURAL OVERVIEW The present provides a system for effectively capturing, annotating and transmitting images of internet 20 World Wide Web (WWW) pages. The system includes a method which is embodied by a software program using a digital computer. The program is supplied to users on a computerreadable medium such as a CD-ROM.

FIG. 1 illustrates a computer system 10 for 25 implementing the present invention. In this embodiment of the invention, the system 10 is a stand-alone configuration. The system 10 includes a \\melbfiles\homeS\flanagan\keep\SPECIFICATIONS\51367-99.doc 21/03/03 WO 00/07110 PCT/US99/17432 Random Access Memory (RAM) 12 in which a software program for implementing the functionality of the invention is stored, and a processor 14 for executing the program. The system 10 further includes a visual display unit or monitor 16 for providing a visual display of relevant information on a screen, a read-only memory (ROM) 18 for storing firmware, and an input-output unit 11 for connection to a printer, modem, etc.

The system 10 further includes a mass data storage 20 which can be any combination of suitable elements such as a fixed (hard) magnetic disk drive, a removable (floppy) disk drive, an optical (CD- ROM) drive, etc. The program which implements the invention may be stored in the storage 20, and modules of the program loaded into the RAM 12 for execution as required.

User access to the system 10 is provided by an input unit or device 22 such as alphanumeric keyboard and/or a pointing device such as a mouse. The elements of the system 10 are interconnected by a bus 24. The system 10 can be implemented by a conventional personal computer.

As viewed in FIG. 1, the system 10 is connected to the internet 26 via a modem 28 and telephone line 30. However, other means of connection to the internet 26 are within the scope of the invention. The system 10 functions as a user terminal.

The computer software program which implements the functionality of the present invention is a commercial product of Insight Development Corp. of San Ramon, CA, entitled "Hot Off The Web" (HOTW)". The program is currently supplied to end users on an optical CD-ROM disk in compressed form. It is expanded and loaded onto a hard drive of a user's computer by running a setup program.

However, it is within the scope of the invention to provide the HOTW program on a computer-readable medium other than an optical element, such as on one or more magnetic elements such as floppy disks, or stored in a ROM or other computer memory element.

As presented above, the present invention enables a user to capture, annotate, print and transmit images of WWW web pages downloaded from the intemet 26. Each web page includes a hypertext document file in the HTML language, and ancillary files such as graphic image files which are referenced by links in the document. As will be described in detail below, the HTML document is interpreted by a web browser, and the web page defined by the HTML document is displayed on the monitor 16 together with the graphic images represented by the image files inserted in their proper locations.

The functionality of the present invention will be described first with reference to FIGs. 2 to 9. Then, the internal structure and operation of the software will be presented in detail.

FUNCTIONAL DESCRIPTION FIG. 2 illustrates the display on the monitor 16 generated by the HOTW program. The display includes a menu bar 32, toolbar 34, taskbar 35, icon tray 36, and a web page 38.

The menu bar 32 includes text names of pull-down menus which can be accessed using a keyboard or mouse. The toolbar 34 is optional and provides a fast way to perform the operations which are also accessible from the pull-down menus. As illustrated, the toolbar 34 includes the following icons: "Open Page" 40, "Save" 42, "Print" 44, "Print Preview" 46, "Cut" 48, "Copy" 50, "Paste" 52, WO 00/07110 PCT/US99/17432 "Undo" 54, "Graffiti Pen" 56, "Highlighter" 58, "Digital Sticker" 60, "Cyber Note" 62, "Scrapbook" 64 and "E-mail" 66. These elements are omitted from FIGs. 3 to 9 except as necessary for simplicity of illustration.

In its currently marketed form, HOTW is designed to be used in conjunction with the Microsoft Internet Explorer 4.0 web browser. It is a Windows- 95/98/NT 4.0 application, and is launched either by selecting the program from the "Start" menu or by clicking on an HOTW icon 68, which is located in the icon tray 36 in the right-hand corner of the taskbar TRAY ICON If Internet Explorer is running, the user can click on the HOTW tray icon 68 using a mouse, and the web page currently displayed in the browser will be loaded into HOTW. The user may then open and work on multiple web pages simultaneously.

If Internet Explorer is not running, a sample starter web page (the page 38 illustrated in FIG.

1) containing a brief introduction-to HOTW is displayed.

PULL-DOWN

MENUS

FIG. 3 illustrates one of the pull-down menus, in this case a "File" menu 70, which is accessed by mouse clicking on "File" in the menu bar 32 or pressing Alt-F on a keyboard. "Open" brings up a dialog box which allows the user to enter a web address (URL) and open a page directly from the Internet. If Internet Explorer is running, the current page will be displayed.

A user may click on the arrow to the right of the URL to display a list of recently viewed pages, or click "Browse" to open a page from the user's file system. HOTW will open HTML files only.

"Save" and "Save As" save the contents of the HOTW page into a folder in order to preserve all the different ancillary files that make up the page. A HOTW page is opened by opening the folder, then clicking on the HTML file that shares the same name as the folder.

"Import" allows a user to import PNG, GIF or JPEG graphic image files for use as Cyber Notes and Digital Stickers as will be described below. "Print" causes the page to be printed. "Print Setup" contains options the user can use for standard adjustments such as orientation, paper size and source.

Although not explicitly illustrated, the "Edit" pull-down menu includes the following selections.

"Undo" reverses a previous action.

"Cut" removes a blocked section and saves it to the clipboard.

"Copy" copies a blocked section to the clipboard.

"Paste" places the clipboard content into a page.

"Delete" removes the selected content from the displayed page.

"Select All" selects all of the content, including the web content and annotations.

"Clear All Annotations" deletes all the HOTW annotations the user has made on a page.

The "View" selection on the menu bar 32 accesses a pull-down menu (not shown) which enables the user to change the view and set preferences.

ANNOTATION

TOOLS

WO 00/07110 PCT/US99/17432 The "Annotate" selection on the menu bar 32 enables a user to select one of four annotation tools, and also to edit the text in a cyber note. The annotation tools are "Graffiti Pen", "Highlighter", "Digital Sticker" and "Cyber Note", and can also be activated by clicking on the icons 56 to 62 respectively.

After a tool is selected, a floating dialog box or palette appears which allows the user to make specific choices regarding the use of the tool (the descriptions appear under each individual tool). The palette is closed by clicking on the X in the upper right-hand comer of the screen or by selecting a different tool.

The "Scrapbook" selection on the menu bar 32 enables a user to create a new scrapbook and annotate and organize portions of multiple web pages, add, edit and format a scrapbook title, and send selected content to an open scrapbook or a new scrapbook.

GRAFFITI PEN The "Graffiti Pen" tool is illustrated in FIG. 4, and is activated from the "Annotate" menu or by clicking on the icon 56. A "Graffiti Pen" palette 72 pops up which displays the current pen choice in terms of size and color, and allows the user to change the attributes of both. This palette 72 also displays a selection of 15 pen colors.

After the user chooses a pen size and color, the cursor changes into a cross-hair 74. To draw a line such as indicated at 76 on the displayed web page, the left mouse button is held down and the cursor 74 is moved around the screen. The mouse button is released to complete the drawing.

HIGHLIGHTER

As illustrated in FIG. 5, selection of the "Highlighter" tool causes a "Highlighter" palette 78 to pop up which allows a user to select the color of the highlight marker. The color selected is displayed at the top of the palette.

When the user selects "Highlighter", the cursor changes into an I-beam 80 when the cursor is over text. The user clicks the left mouse button and drags the cursor 80 over text to be highlighted.

The highlight appears when the mouse button is released. To change the highlight color, the user selects a new color from the palette 78 and repeats the selection.

To immediately undo a highlight, the user may select "Undo" from the "Edit" menu or click on the "Undo" icon 54. Or, the user may select "None" from the palette 78, then select the text highlight the user wishes to erase by left-clicking and dragging. When the mouse button is released, the highlight will disappear.

CYBER NOTES The Cyber Notes annotation tool is illustrated in FIG. 6. This tool enables a user's thoughts and comments to be added to a web page as an annotation along with professionally created images and graphics. More specifically, a Cyber Note is a graphic image or sticker which can be pasted onto a web page. Each Cyber Note includes a central box area into which the user can type a message using a keyboard.

Activating the tool causes a "Cyber Notes" dialog box or palette 82 to pop up. The "Cyber -6- WO 00/07110 PCT/US99/17432 Notes" palette 82 is organized according to theme. The palette 82 includes a text window 83 which displays the name of the currently selected theme, in the illustrated example "Pop Secret Agents", and thumbnail images 87 of the Cyber Notes in the theme. Clicking on an arrow icon 85 causes another drop-down dialog box (not shown) to pop up which displays the names of all of the available themes.

The set of thumbnails in a theme is viewed by selecting a theme using the scroll bar 89. By default, the displayed theme is the last theme used. This may be changed by selecting "Preferences" from the "View" menu, and choosing the theme to be used as the default.

The most frequently used Cyber Notes can be collected in the "Favorites" theme for easy reference. To remove a Cyber Note from "Favorites", the user selects the thumbnail in the "Favorites" theme and clicks on the "Remove from Favorites" button. The user will still be able to find that Cyber Note in its original theme.

After selecting a Cyber Note 86 from the palette 82 using a mouse click, the cursor changes into a rectangular outline 84 of the selected note 86. To place the note, the user moves the outline 84 to the desired location on the page, then clicks the left mouse button. The Cyber Note then appears in the web page as indicated at 88 in FIG. 7. The cursor 80 appears in the Cyber Note 88, allowing the user to enter a text message 90 using a keyboard.

Messages may be edited by double-clicking on the Cyber Note 88. A "Cyber Note Editor" dialog box (not shown) appears, allowing the user to format a message by typing text into a box in the cyber note 88.

A user can import a graphic image file from a source outside the HOTW program clip art) and use it as a Cyber Note. This is done by selecting "Import" from the File menu, and then selecting the "Cyber Notes" menu item. The user chooses the theme to add the note to, then browses the file system for the PNG, GIF, or JPEG graphic image file the user wishes to add. The user selects the file and clicks the "OK" button.

A dialog box (not shown) appears which allows the user to set the text margins. The image being imported will be displayed, along with an outline representing the text box area for the Cyber Note. To adjust the text area, the outline is dragged to the size the user wants to be able to enter text in.

DIGITAL STICKERS Although not explicitly illustrated, the "Digital Stickers" annotation tool is similar to the "Cyber Notes" tool in that it enables a user to add graphics to a web page to get points and messages across.

A Digital Sticker is a graphic image which differs from a Cyber Note in that it does not include a text box for entering a user's message. Digital Stickers are arranged according to themes and are selected and pasted onto a web page in a manner which is essentially similar to the Cyber Notes as described above. External graphic images can be imported for use as Digital stickers in the same manner as for Cyber Notes.

-7- WO 00/07110 PCT/US99/17432

SCRAPBOOK

The "Scrapbook" tool allows the user to easily organize information found on the web. For example, if the user is shopping for a new car, the user can select the information needed from each of the automobile web sites visited and place each item of interest into a single scrapbook. The annotation tools can also be used to make notes on the content after it is brought into the scrapbook.

The user can select "New Scrapbook" from the "Scrapbook" menu or click on the "Scrapbook" icon 64. A dialog box (not shown) appears which prompts the user to name the new scrapbook and to select an option for listing the URLs and times of each web page in the user's scrapbook.

To add content to a scrapbook, the user selects items from a web page, then clicks the right mouse button and selects the "Send To Scrapbook" menu item, or selects "Send to Scrapbook" from the "Scrapbook" menu.

A "Send to Scrapbook" dialog box (not shown) appears, listing the user's open scrapbooks.

Selecting an open scrapbook automatically adds the content to the end of that scrapbook. Clicking "New Scrapbook" brings up the "New Scrapbook" dialog box described above under "New Scrapbook." Once the user names the new scrapbook, selects its URL and time stamp option, and clicks OK, the user has the option of copying or sending the selected content of the new scrapbook. The content is automatically added to the new scrapbook.

As illustrated in FIG. 8, a section 92 of web page content can be blocked for selection using a mouse. This brings up a dialog box 94 which enables the user to send the blocked content to a scrapbook. FIG. 9 illustrates the section 92 stored in a scrapbook named "My Scrapbook".

HOTW

further provides the option of automatically adding a stamp 96 following the content 92 which indicates the origin of the content and the time and date it was added to the scrapbook.

Saving a scrapbook is like saving any other HOTW page. All of the related files are saved into a folder. When a new scrapbook is created, it is automatically saved in the "Scrapbook" subfolder under the "Hot Off The Web" program folder. To open an existing scrapbook, the folder is first opened, the HTML file that has the same name of the folder is selected.

E-MAIL

An annotated page can be sent to a friend or business associate using the electronic mail or "Email" function. This can be accessed by selecting "Send Mail" from the "File" menu or clicking on the icon 66.

This causes a message composition form (not shown) to pop up from the user's default E-mail program with the HOTW page automatically attached as a self-extracting CABINET (.exe) file.

If the recipient of the E-mail has HOTW, the attachment will be opened as a HOTW file. If not, the attachment will be opened in the recipient's default web browser. This annotated page may exist as an attachment, or as an integral part imbedded in the e-mail message.

DETAILED STRUCTURE The internal structure of the present invention will now be presented in detail. FIG. illustrates the various software components which in combination provide the functionality of the -8- WO 00/07110 PCT/US99/17432 invention.

As described above with reference to FIG. 1, the computer system 10 is a programmed digital computer including a digital data storage which collectively includes the data storage 20, RAM 12 and ROM 18. The software for practicing the invention is stored in the storage, and includes an operating system 100, in the present case Microsoft Windows* 95 or 98 or NT 4.0, the Hot Off The Web (HOTW) program 102, one or more web pages or HTML documents and ancillary files 104, and a web browser 106.

In the currently marketed version of HOTW the browser 106 is Microsoft Internet Explorer although, as will be described below, the invention is not limited to any particular web browser, and can alternatively be used without a web browser.

The detailed configuration of the browser 106 is not the particular subject matter of the present invention, and will be described only to the extent that is necessary for an understanding of the present invention. Full details regarding the Internet Explorer web browser are available from Microsoft.

The browser 106 generally includes a browser control 106a which is used to access URLs, display menus and dialog boxes for various options, and download and display selected web pages from the internet. The browser 106 further includes a parsing and rendering engine 106b which is controlled by the browser control 106a for interpreting a downloaded document in HTML format and rendering a corresponding web page on the monitor 16, including all graphic and other ancillary files.

The engine 106b in Internet Explorer is a file entitled MSHTML.DLL, which is also used as a stand-alone module by the HOTW program 102. Further included in the browser 106 are a suite of multimedia controls and plug-ins which process ActiveX scripts, Java Applets, etc. under control of the browser control 106a and engine 106b.

The software components illustrated in FIG. 10 are stored in the various elements of the storage.

Generally, these components will be stored on a hard drive of the data storage 20, and various modules will be loaded into the RAM 12 for execution by the processor 14 as required. The manner in which the software is stored is not limited within the scope of the invention. For example, although the preferred manner of providing the HOTW program 102 to a user is to store it on a CD_ROM and install it onto the computer storage, it is possible to store part or all of it in the ROM 18 or in an auxiliary ROM (not shown) as firmware.

The main components of the HOTW program 102 are illustrated in FIG. 11. The program 102 includes a tray icon module 110 which enables the user to launch the HOTW program by clicking on the tray icon 68 illustrated in FIG. 1. A preprocessor 112 is provided to convert an HTML text document file which is downloaded from the internet or obtained from another source into a predetermined format which facilitates annotation.

An annotator or main program 114 provides overall control of the other components of the HOTW program 102 and enables the user to annotate a web page after preprocessing by the preprocessor 112 using the input unit or devices 22 of FIG. 1. The preprocessor 112 and program 114 use the engine 106b to get and set attributes in HTML files using Microsoft's Document Object Model (DOM) WO 00/07110 PCT/US99/17432 language. Detailed documentation on the manner in which DOM can be used to perform these operations is available from Microsoft.

The HOTW program 102 further includes an E-mail module 116 for compressing a web page into a self-extracting CAB (.exe) file and transmitting it to a recipient as an attachment to an E-mail message. A printing module 118. enables a web page to be printed without any portions thereof being cut off.

The HOTW program 102 further includes a collection of annotation files, more specifically sets of Digital Stickers and Cyber Notes, which are organized by theme as described above. The parsing and rendering engine 106b, which is actually part of the web browser 106, is also illustrated in FIG. 11 because in an alternative embodiment of the invention as will be described below, the engine 106 can be provided as part of the HOTW program 102 itself and used as a stand-alone component.

The major components of the main module 114 are illustrated in FIG. 12. A controller 130 provides overall control and coordination between the component modules for annotation and other operations.

A module 132 uses the browser control 106a to generate the toolbars and icons which were described above with reference to FIGs. 2 to 9. The particular manner in which the toolbars and icons are generated is not the particular subject matter of the invention and will not be described in detail.

Documentation on the manner in which the browser control 106a can be used to perform these operations is available from Microsoft.

A module 134 also uses the browser 106a to generate the dialog boxes and palettes for the four annotation tools as described above. A module 136 is used to insert Digital Stickers and Cyber Notes into web pages using Cascading Style Sheets (CSS) as will be described in detail below. A module 138 operates in conjunction with the module 136 to enable a user to position a Digital Sticker or Cyber Note at a desired position on a web page.

A module 140 enables a user to import an external file clip art) for use as a Digital Sticker or Cyber Note. A module 142 is used to create and edit a scrapbook, and insert selected portions of web pages into a scrapbook. A file manager module 144 is used by the other modules to open folders and files, save and print web pages, and perform other file oriented operations as required.

FIG. 13 illustrates the format for storing the annotation files (Digital Stickers and Cyber Notes) 120. The Digital Stickers and Cyber Notes for each theme are stored in separate folders. As illustrated, the Digital Stickers for a certain theme include a Digital Sticker HTML file 150 which includes hypertext elements or links that reference or point to the graphic image files for the Digital Stickers.

The Digital Sticker images themselves are preferably in the Portable Network Graphic (PNG) file format (but can be GIF, JPEG or other type of file format) and have a filename extension of ".png".

As shown, Digital Sticker images 150a to 150n referenced by the HTML file 150 are designated as "Digital_Sticker_l.png" to "Digital_Sticker_n.png" respectively. An element in the HTML file 150 referencing or pointing to the file 150a, for example, is preferably provided as an element of a table in the HTML file 150, and will have a configuration such as: WO 00/07110 PCT/US99/17432 <tr> <td align="center" height="47" width="123"><img width="115" HEIGHT="84" alt="Digital Sticker </tr> Further illustrated in FIG. 13 are a Cyber Notes HTML file 152 and PNG image files 152a to 152n which are arranged in the same manner as the files 150 and 150a to 150n.

FIG. 14 illustrates the arrangement of an HTML document file and ancillary files as indicated at 104 in FIG. 10 which were downloaded from the internet This collection of files is stored in its own folder. As illustrated, the files for one web page include a main hypertext document 154 in HTML language format which includes a link element to an ancillary graphics image file "ImageJ_.png" 154a.

The HTML file 154 is also linked to image files 154b, 154c and 154d which are annotation files created by the user. The file 154b is for a Cyber Note and is named "Cyber Note_l.png"; the file 154c is for a Digital Sticker and is named "Digital_Sticker_l.png"; the file 154d is for Graffiti Pen annotation and is named "Graffiti_l.png". Although one file of each type is shown in the drawing, it will be understood that a particular HTML file 154 can reference no ancillary or annotation files, or alternatively any number thereof.

DETAILED OPERATION The detailed operation of the present invention will now be described with reference to flowcharts illustrated in FIGs. 15 to 21.

TRAY ICON FIG. 15 illustrates the operation of the tray icon module 110 which is illustrated in FIG. 11.

The HOTW program 102 is preferably launched by mouse clicking on the HOTW tray icon 68 illustrated in FIG. 2. The first step 160 is to determine if an instance of the tray icon 68 (the main program 114) is running. If so, the module 110 gets a web page (URL) from the open window of the browser 106 in a step 162. If not, the module 110 opens with the default page 38 as shown in FIG. 2.

The module 110 then starts the maim HOTW program 114.

PREPROCESSING

The operation of the preprocessor 112 is illustrated in FIG. 16. The preprocessor 112 is called by the main program 114 after it has received control from the tray icon module 110 in a step 170. The preprocessor 112 first determines in a step 172 if the URL or web page is a local file (created locally or previously stored). If so, it copies the file and any ancillary graphic images or other files to a temporary directory in a step 174. If not, it copies the file(s) from the browser's cache memory (if available) or downloads it from a URL server via the internet in a step 176.

The preprocessor 112 then parses the HTML file which is stored in the temporary directory to determine if any ancillary files are present in a step 178. The preprocessor 112 then preprocesses the HTML using the parsing and rendering engine 106b in a step 180.

More specifically, the preprocessor 112 uses the parsing and rendering engine 106b and 11 WO 00/07110 PCT/US99/17432 Document Object Model (DOM) language to determine if a hypertext link which references an ancillary file of a predetermined type is present in the HTML document. The most common type of file which is linked to an HTML file, and desired to be included in the preprocessed web page, is a graphics file in a GIF, PNG, JPEG. Other formats such as AVI, Shockwave, Quicktime, way or au, for sound graphics and video, may also be used since the scope of the invention includes linking and/or displaying any suitable type of ancillary file.

The preprocessor 112 uses DOM language instructions to search for an element in the HTML document referencing or pointing to, for example, a graphics file. An element pointing to a GIF file named "Image_l.gif" will have a typical format: <img src="http://www.mysite.com/images/Image_.gif"> If a link is present, a local copy of the ancillary file is stored in the temporary directory, and the element referencing the ancillary file is modified to reference or point to the local copy.

In the present example, the element pointing to the file "Image_l.gif' was downloaded from a directory or folder named "images" at an URL named "http://www.mysite.com". The element is modified to reference or point to the file "Image_l.gif", but in this case in a local directory named "temp" on a hard drive in the storage 20 of the present system 10. The modified element will have the form: <img src="/temp/http wwwmysite.comimagesImage._.gif"> The image file "Image_l.gif" itself is renamed as "http www mysite.com_imagesjmage_l.gif' by changing the slashes and dots to underscores Alternatively, the prefix can be dropped, and the file renamed as "wwwmysite.com_images_mage_l.gif'.

If the folder "temp" can be implied as current, the element can be simplified by omitting the folder name and have the format: <img src="http www_mysite.com_images_Image_l.gif'> or <img src="www_mysite.comimagesmage_l .gif'>.

After preprocessing of the HTML document, the ancillary files are renamed by the file manager 144 to correspond to their new names in the HTML document and stored in the temporary directory or folder. If the user wants to save the web page with or without annotation, a new folder will be created with a name selected by the user, and the HTML file and ancillary files will be saved into the new folder.

As presented above, the preprocessor 112 also modifies the HTML document to facilitate annotation by the controller 130 and associated modules of the main program 114. This modification includes a number of changes.

1. The HTML content is placed into a fixed-width table which allows the annotation tool images to be easily inserted into the table. This also allows WYSIWYG printing preprocessed documents, which is preferably employed with the present invention. As such, the preprocessed

HTML

pages will have exactly the same layout regardless of the settings on the computers to which an HOTW 12 WO 00/07110 PCT/US99/17432 page is sent by E-mail and will also look like the actual annotations made.

Since browser settings such as default font and window size affect the layout of an HTML document, the same HTML document may look quite different on different machines therefore compromising the effectiveness of annotations. The appearance of the displayed web page is fixed by defining the width of the encapsulating table.

A fixed width table is created by inserting an element into the body of the HTML document having a format such as: <TABLE style="CURSOR: text" width=600> This element defines a table having a width of 600 pixels. The elements of the HTML document which generate the displayed web page follow the table definition element. This allows the printing of a web page on an 8 1/2" x 11" sheet.

2. Attributes designating fixed font properties (font and size) are assigned to all text elements. This causes the displayed page to have a predetermined appearance regardless of the default settings on the computer which is running the HOTW program, as described above.

An element which performs this function has a form such as: <STYLE> HI font-size: 24pt; H2 font-size: 18pt; H3 font-size: 13.5pt; H4 font-size: 12pt; font-size: 10pt; H6 font-size: 7.5pt; TABLE, TH, TR, TD font-size: 12pt; TT font-size: 10pt; SELECT font-size: 9pt; </STYLE> 3. Java script and ActiveX elements are replaced by boxes of the same size as the original objects. Comments are placed in the boxes indicating their original functions.

4. Marquee objects are made static.

Buttons, forms, and mouse events are removed or disabled.

ANNOTATION

After the HTML and ancillary files have been preprocessed and stored in the temporary directory, the web page is ready for annotation by the user. The web page defined by the HTML document is rendered or displayed on the monitor 16 by the engine 16b together with the ancillary images in their proper locations.

GRAFFITI PEN FIG. 17 illustrates the steps for annotating a web page using the Graffiti Pen. As described above, the user selects the Graffiti Pen tool from the menu bar or by clicking on the icon 56, and selects the color for lines to be drawn using the palette 72. These operations are performed by the browser control 106a under control of the module 134 of the main program 114.

To draw a line, the user holds down the left mouse button and drags the cursor over the displayed web page. The program 114 gets the mouse positions or points from the browser 106 in a step 190. and converts the points into pieces in a step 192.

13 WO 00/07110 PCT/US99/17432 Releasing the left mouse button provides an input to the program 114 which indicates that the line drawing operation has been completed. In response, the program 114 converts the pieces into a PNG file in a step 194 and stores the PNG file in the temporary directory as a position file together with the HTML and ancillary files in a step 196.

The final step 198 is to insert an element into the HTML document which references the stored PNG file. This is preferably performed by the module 136 which inserts a Cascading Style Sheet (CSS) element into the page table of the HTML document with a position for the line and a Z-Order for the file specified. A Graffiti Pen placement element for a PNG file named "Graffiti_l.png" will have a format such as: <P style="CURSOR: auto"> <IMG border=O src="Graffiti_l.png" style="CURSOR: auto; LEFT: 186px; POSITION: absolute; TOP: 114px; VISIBILITY: visible; Z-INDEX: 268"></P> The above is the format specified by Microsoft. However, it may not display properly in Netscape Navigator. A format which will work for Netscape is: <style type="text/css"> Graffiti_l.png left: 186px; top: 114px; position: absolute; visibility: visible; z-index: 268; </style> <body> <DIV ID="Graffiti_l.png"><IMG SRC="Graffitil.png">

</DIV>

</body>

HIGHLIGHTER

FIG. 18 illustrates the steps for annotating a web page using the Highlighter. As described above, the user selects the Highlighter tool from the menu bar or by clicking on the icon 58, and selects the color for the highlighted text using the palette 78. These operations are performed by the browser control 106a under control of the module 134 of the main program 114.

In a step 200 in FIG. 18, the program 114 determines that the Highlighter tool has been activated and gets the text to be highlighted from the browser control 106a in a step 202. The program 114 then changes the color attribute of the highlighted text in the HTML document to the color selected from the palette 78 in a step 204. The final step 206 is to save the modified HTML document to the temporary directory.

CYBER NOTES FIG. 19 illustrates the steps for annotating a web page by pasting a Cyber Note thereon. As described above, the user selects the Cyber Notes tool from the menu bar or by clicking on the icon and selects a Cyber Note using the palette 82. These operations are performed by the browser control 106a under control of the module 134 of the main program 114.

In step 210 in FIG. 19, the program 114 determines that the Cyber Notes tool has been activated 14- WO 00/07110 PCT/US99/17432 and gets the mouse position from the browser control 106a in a step 212. Since Internet Explorer does not support drag and drop, this functionality is emulated by the present module 138.

More specifically, the module 138 uses the browser control 106a to generate an outline display or rectangle at the position of the cursor in a step 214. Preferably, the upper left corer of the rectangle is at the cursor position. Then, the module 138 checks to see if the left mouse button was clicked in a step 216, indicating that the position for the Cyber Note has been decided by the user.

If yes, the module 138 inserts the Cyber Note at the cursor position in a step 218. If not, the module 138 loops back to the step 212 to get the new cursor position and redisplay the rectangle at the new position.

The next step 220 is to utilize the browser control 106a to display a text box or dialog box to enable the user to enter text into the Cyber Note using the keyboard. A step 222 determines if a left mouse key click has been generated, indicating completion of the text insertion operation. If yes, the HTML file is modified and saved in the temporary directory, and the Cyber Note is saved as a PNG file.

If no, the module 138 loops back to step 220 to input more text.

The program 114 modifies the HTML document by inserting a CSS element into the above described table which references the stored Cyber Note PNG file. The CSS element specifies the position of the Cyber Note and the Z-order of the file. The format specified by Microsoft for a file named "Cyber_Note_l.png" is: <DIV dataFormatAs=TEXT id="Note:LM Cyber_Note_l.png" style="COLOR: #000000; CURSOR: auto; HEIGHT: 114px; LEFT: 202px; OVERFLOW: hidden; POSITION: absolute; TOP: 358px; VISIBILITY: visible; WIDTH: 189px; Z-INDEX: 258"><FONT face="Courier New" size=2 style="CURSOR: auto; TEXT-DECORATION: none">Cyber Note 1</FONT> </DIV> Again, this format may not display properly with Netscape Navigator. A format which will work is: <style type="text/css"> LM_Cyber_Note_l.png color: #000000; height: 114px; width: 189px; left: 202px; top: 358px; overflow: hidden; position: absolute; visibility: visible; z-index: 258; </style> <body> <DIV ID="LM_Cyber_Note_l.png"><IMG

SRC="LM

Cyber_Note_l.png<FONT face="Courier New" size=2>Cyber Note 1</FONT>

</DIV>

15 WO 00/07110 PCT/US99/17432 </body> DIGITAL STICKERS An annotation operation for pasting a Digital Sticker on a web page is essentially similar to that for a Cyber Note as described above, except that a Digital Sticker does not have a text box for inserting a message. The operations for inputting text are therefore omitted.

Digital Stickers are saved as PNG files. A CSS element referencing the PNG file is inserted in the table in the HTML document with the position of the Digital Sticker and the Z-order for the file specified. The format specified by Microsoft for a Digital Sticker file named "DigitalStickerl.png" is: <P style="CURSOR: auto"xIMG id=Sticker src="DigitalSticker_l.png" style="CURSOR: auto; LEFT: 225px; POSITION: absolute; TOP: 232px; VISIBILITY: visible; Z-INDEX: 256">&nbsp;&nbsp; A CSS format for a Digital Sticker which will display properly in Netscape is: <style type="text/css"> Digital_Sticker_l.png position: absolute; top: 232px; left: 225px; visibility: visible; z-index: 256 </style> <body> <DIV ID="Digital_Sticker_1 .png"> <IMG SRC="Digital_Sticker_l.png"></DIV> </body>

SCRAPBOOK

The scrapbook module 142 uses the browser control 106a and engine 106b to get the elements from an HTML document which correspond to blocked text on the displayed web page. The operation is similar to conventional cut and paste except that the content is pasted into a scrapbook HTML document.

A scrapbook is created to encapsulate the pasted HTML content in a fixed width table in the same manner as described above for the preprocessor 112. Preferably, each block of HTML content is inserted into the scrapbook HTML document in a separate subtable. This prevents any attributes for one block of content from affecting another block

E-MAIL

The operation of the E-mail module 116 is illustrated in FIG. 20. As described above, the user selects the E-mail function from the menu bar or by clicking on the icon 66.

The first step is to set up the HOTW program 102 as an E-mail client. This causes a message composition form (not shown) to pop up from the user's default E-mail program, into which the user inputs the E-mail address of the addressee, an optional message and any other desired information.

In the next step 232, the HTML document and any ancillary files are compressed into a self- 16- WO 00/07110 PCT/US99/17432 extracting cabinet file. This is "CAB" file with an extension of and does not require a separate decompression program for extraction. The cabinet file is then transmitted to the addressee as an attachment to the message.

If the recipient of the E-mail has HOTW, the attachment will be opened as a HOTW file. If not, the attachment will be opened in the recipient's default web browser.

PRINTING

The printing functions provided by the MSHTML.DLL file which constitutes the parsing and rendering engine 106b are limited. MSHTML.DLL is unable to do print preview. Also, if a document is too wide, part of the document is cut off. The present printing module 118 overcomes these deficiencies.

The printing operation is illustrated in FIG. 21. As described above, the user selects the print function from the menu bar or by clicking on the icon 44. The first step 240 of the process is to draw the content of the HTML document to a virtual screen, such as a block of memory locations in the RAM 12. Then, the individual actions are parsed, broken down and rendered into pages which fit a printer in a step 242. Preferably, each page will have a fixed size such as 8 1/2 x 11 inches. Finally, the individual pages are rendered to the printer in a step 244.

SERVER IMPLEMENTATION FIGs. 22 to 25 illustrate alternative embodiments of the present invention in which the storage of documents and/or execution of the annotation program are performed on a server under control of a user at a local terminal which is connected to the server by the internet or other network (Local Area Network (LAN), Wide Area Network (WAN), etc.).

FIG. 22 illustrates a system 300 in which a user terminal 302 is connected to a server 304 over the internet 306. The user terminal 302 can be a personal computer including the conventional elements illustrated in FIG. 1. Alternatively, the terminal 302 can be a "dumb" terminal, including a display unit 308, an input device 310, and an elementary processor and small amount of memory for running a low level control program (not shown).

The user terminal 302 can also have a configuration other than a conventional desktop or laptop personal computer. For example, the present invention can be applied to a set-top unit such as used to connect a user to the internet using a conventional television receiver. In this case, the display unit 308 is the television receiver and the input unit is a remote control unit for the set-top unit. Another applicable type of user terminal 302 is a small hand-held unit.

The server 304 is generally larger than a personal computer including, for example, a mainframe computer. The server 304 includes a storage 312 which typically includes a bank of hard drives. An annotation program 314 according to the present invention is stored in the storage 312 and executed by a processor 316.

The storage 312 stores at least one hypertext document 318, preferably a number of linked documents and ancillary image and other files, including original documents for normal browsing and modified documents including user annotations. The annotations files 320 themselves are stored in the 17- WO 00/07110 PCT/US99/17432 storage 312 in the same folder with or in a different folder from the modified documents.

The user terminal 302 may or may not run a browser program. In the latter case, the functionality of the browser is provided on the server 304. A user utilizes the input unit 310 of the user terminal 302 to browse the web site on the server 304 and cause a selected web page corresponding to a document 318 to be displayed on the display unit 308. The program 324 enables the user to annotate, save, print and E-mail a web page in the manner described above.

The main difference is that the program 314 is stored and executed on the server 304 instead of on the user terminal 302. Also, annotated hypertext documents, ancillary files and annotation files are stored on the server 304. This configuration enables a user to access the internet using a very simple and inexpensive user terminal 302 with a minimal or no file storage capability. A variation of this arrangement is that a modified version of an annotated hypertext document need not be stored on the server 304. A modified version can be created and the annotations 320 applied thereto "on-the-fly" in response to a user accessing the hypertext document at a subsequent time.

FIG. 23 illustrates another server implemented embodiment of the invention in which like elements are designated by the same reference numerals used in FIG. 22. In a system 322, a user terminal 324 is provided with a small amount of storage which is sufficient to store the annotation files 320. The hypertext document files 318 are stored on the server 304. The program, here designated as 326, is modified to download the annotation files 320 to the user terminal 324.

FIG. 24 illustrates another system 328 which is similar to the system 322 of FIG. 23 except that both an altered hypertext document 330 and associated ancillary and annotation files 320 are downloaded to a user terminal 332 by a program 334 after the annotation operation has been performed on the server 304.

FIG. 25 illustrates another server implemented system 336 which is similar to the stand-alone embodiment described above with reference to FIGs.-1 to 21 except that the documents 318 are stored on the server 304. A user terminal 337 includes the processor 326 which executes an annotation program 338 stored in the storage 312 which is located in the user terminal 337. The server 318 can store modified documents as well as original documents 318, or modified documents can be created onthe-fly as described above.

In the embodiments described above, the present annotation program has been described as a conventional program which is executed directly by a processor. However, the present invention is not so limited. It is within the scope of the invention to provide the annotation functionality as a Java applet, ActiveX control, or XML functionality, which is stored on a server and downloaded to a user terminal together with a hypertext document. The applet or control is launched by a browser or other application program and executed using the user terminal's processor.

INDUSTRIAL APPLICABILITY

MODIFICATIONS

Various modifications will become possible for those skilled in the art after receiving the teachings of the present disclosure without departing from the scope thereof. The following 18- WO 00/07110 PCT/US99/17432 modifications are exemplary only, and are not intended to be inclusive or in any way imitating the scope of the invention.

1. The present invention has been described above as being used in conjunction with the Microsoft Internet Explorer 4.0 web browser. However, the invention is not so limited, and the present system can be modified to work in conjunction with Netscape Navigator or any other commercially available or proprietary web browser. The present program can also be provided as a module plugin) of a web browser or other application program such as a word processor.

2. It is also within the scope of the invention to modify the system to function in a stand alone manner without an external web browser. This can be accomplished using non-MSHTML.DLL dependant software with comparable functionality, in combination with software which provides the functionality of the browser control 106a and associated elements.

3. Although the invention has been described above as operating on hypertext documents written in the HTML language, the scope of the invention is not so limited. The present invention is applicable to any type of hypertext document which can include a link to another document.

Other hypertext document formats which are compatible with the invention include Standardized Generalized Markup Language (SGML), Dynamic Hypertext Markup Language (DHTML) and Extensible Markup Language (XML). These formats are listed as examples only, and are not limitative of the scope of the invention. For example, the invention can be applied to a hypertext format in which the text elements are binary encoded.

4. As another typical modification within the scope of the invention, it is possible to use another file format. The PNG format is preferred because it is possible to easily display a PNG file with a transparent background in Internet Explorer. This enables the PNG annotation files to be overlaid on the main web page content without the background areas of the PNG image obscuring underlying main content.

However, Netscape Navigator does not support transparent PNG files. For this reason, the transparent GIF file format can be substituted for the PNG format. Transparent GIF files are supported in both of these web browsers. Other suitable file formats are further contemplated within the scope of the invention.

Alternative methods of positioning the Graffiti pen, Digital Sticker and Cyber Note graphic images on a page are possible within the scope of the invention. The method which is described above and implemented in the HOTW product is to enclose the page in a fixed width table in which a graphic image is referenced by an element in a cell of the table. The position of the cell determines the position of the image on the page.

It is further possible to define a position for a graphic image by overlaying the page with a virtual grid, and defining the image position as a pair of coordinates on the grid. Yet another alternative is to specify the image position as a offset from a predetermined position or anchor on the page, for example from the upper left corner of the page.

6. The embodiment of the invention described above preprocesses a hypertext document 19- 20 perform corresponding operations to consolidate annotation after the annotation has been completed. This arrangement can be described as "postprocessing".

In the postprocessing embodiment, the web page is displayed on the display unit, and the annotation images are created and displayed as overlays on the web page.

The page is not modified at this time to incorporate the annotations. After annotation is completed, the page is modified and saved.

The modifications which are performed prior to saving preferably include enclosing the page content in a fixed width table as described above with the front attributes fixed. Cascading style sheet elements pointing to the annotation files are also inserted as described 0 15 above. Active and external links can be de-activated or left active. The links to the ancillary files can be t changed to reference the local copies as part of the postprocessing operation.

It is also within the scope of the invention to 20 combine preprocessing and postprocessing. For example, the links to the ancillary files can be changed during preprocessing, and the other modifications performed during postprocessing.

7. A primary application of the present 25 invention is to capture, annotate, save and transmit web pages downloaded from the internet or other network.

o However, the web pages which are processed using the invention can be produced locally using a text editor to create a text document consisting of HTML elements and tags. Web pages can also be obtained from a friend or business associate on a floppy disk or other portably storage element and loaded into a user's personal computer which runs the program.

8. It is also within the scope of the invention to annotate graphic images which may be in a propriety file format and which are not desired to be \\melbfiles\homeS\flanagan\keep\SPECIFICATIONS\51367-99.doc 21/03/03 21 altered. This modification is accomplished by painting a graphic image on the display unit, and using the present program to create a hypertext document which is overlaid on the graphic image.

A hypertext document overlay which does not include any display elements (text or images) will be transparent. However, using the above described annotation tools to create Graffiti lines, Digital Stickers and Cyber Notes will cause CSS elements to be inserted into the hypertext document and cause the corresponding annotation elements to be superimposed on the graphic image. This method can be used for any suitable type of graphic image file, such as Autodesk Autocad files (.dwg) Adobe Portable Document Format (PDF) files etc.

INDUSTRIAL APPLICABILITY .The present invention generally relates to the art of electronic communications, and more specifically to a system for capturing, annotating and transmitting images of internet World Wide Web (WWW) pages.

It is to be understood that, if any prior art publication is referred to herein, such reference does not constitute an admission that the publication forms a part of the common general knowledge in the art, in Australia or any other country.

For the purposes of this specification it will be clearly understood that the word "comprising" means "including but not limited to", and that the word "comprises" has a corresponding meaning.

H:\flanagan\keep\SPECIFICATIONS\51367-99.doC 21/03/03

Claims (56)

1. A programmed digital computer system, comprising: a display unit for displaying a hypertext document; an input unit; a processor; and a storage for storing a program which is executed by the processor, the program including an annotator for annotating the page by modifying the document in response to input from the input unit, said annotation including painting an image on the page at a position designated by the input unit.

2. A system as in claim 1, in which: the system comprises a user terminal which is connected to a network having accessible hypertext *documents; the user terminal comprises the display unit, input unit, processor and storage; and the program comprises instructions for downloading a hypertext document from the network. S A system-as in claim 1, in which: S 25 the system comprises a user terminal which is connected to a network having accessible hypertext documents; the user terminal comprises the display unit, input unit, processor and storage; the system further comprises a browser for browsing and downloading hypertext document from the network; and the program is part of the browser.

4. A system as in claim 3, in which the program is a plug-in for the browser. \\melbfiles\homeS\flafagaf\keep\SPECIFICATIONS\5136799.doc 21/03/03 23 A system as in claim 1, in which: the system further comprises an application program; and said program is part of the application program.

6. A system as in claim 1, in which: the system comprises a user terminal and a server which are interconnected by a network; the user terminal comprises the display unit and the input unit; and the server comprises the processor and the storage.

7. A system as in claim 1, in which: the system comprises a user terminal and a server A*4 which are interconnected by a network; the user terminal comprises the display unit, input unit and processor; and the server comprises the storage.

8. A system as in claim 1, in which the page is an overlay for.a separately created graphic image file.

9. A system as in claim.8, in which: 25 the image is substantially opaque; and areas of the page other than the image are L, P substantially transparent. f A system as in claim 1, in which: the system comprises a user terminal and a server which are interconnected by a network; the user terminal comprises the display unit and the input unit; the document is stored on the server; and the program modifies the document on the server.

11. A system as in claim 10, in which: \\melb files\homeS\flanagan\keep\SPECIFICATIONS\51367-99.doc 21/03/03 24 the server comprises the processor and the storage; and the processor executes the program on the server.

12. A system as in claim 10, in which: the image is represented by an image file which is stored in the storage; and the user terminal comprises the storage.

13. A system as in claim 1, in which the program is configured to modify the document to establish a positional reference for the image.

14. A system as in claim 13, in which the positional *ee. reference comprises a.virtual grid which is overlaid on the document. S* 15. A system-as in claim 13,: in which the positional reference comprises a..table in.which the document is enclosed.

16. A system as in claim 15, in which the table comprises a plurality of cells. 25 17. A system as in claim 15, in which the table has a Sfixed width. t

18. A system as in claim 13, in which the positional reference comprises an anchor point in the document.

19. A system as in claim 18, in which the program is configured to calculate the position designated by the input unit as an offset from the anchor point.

20. A system as in claim 19, in which the program is configured to modify the document so as to have a fixed width. \\melb files\homeS\flafagaf\keep\SPECIFICATIONS\51367-99.doc 21/03/03 25

21. A system as in claim 1, in which: the image is represented by an image file; and the annotator is configured to modify the document to include an element which references the image file.

22. A system as in claim 21, in which the element comprises a Cascading Style Sheet (CSS).

23. A system as in claim 22, in which the CSS specifies the position for the image and a Z-order for the image file. 15 .24. A system as in claim 21, in which the image file comprises a Portable Network Graphic (PNG) file. A system as in claim 21, in which the annotator is configured to cause an outline representing the image to be displayed on the page at a position designated by the input unit, and to cause the image to be painted onto the page at the position of the outline in response to a subsequent input from the input unit. 25 26. A system as in claim 21, in which the annotator *is configured to enable a user to select an image from a set of available images.

27. A system as in claim 21, in which the annotator is configured to cause a dialog box containing a set of available images to be displayed which enables a user to select an image from the set.

28. A system as in claim 21, in which the annotator is further configured to enable a user to create an image by importing an image file. \\melbfiles\home$\flanagan\keep\SPECIFICATIONS\51367-99.doc 21/03/03 26

29. A system as in claim 21, in which the annotator is further configured to enable a user to input text into a box in the image using the input unit.

30. A system as in claim 29, in which the annotator is configured to cause an outline representing the image to be displayed on the page at a position designated by the input unit, cause the image to be printed onto the page at the position of the outline in response to a subsequent input from the input unit, and subsequently enable a user to type text into a box in the image.

31. A system as in claim 29, in which the annotator is further configured to enable a user to select an 15 attribute for the text.

32. A system as in claim 29, in which the annotator is further configured to: cause a dialog box to be displayed which enables a user to select an attribute for the next.

33. A system as in claim 1, in-.which the annotator is 2 configured to enable a user to produce the image by drawing a line on the page using the input unit.

34. A system as in claim 33, in which the annotator is configured to sense a cursor positions designated by the input unit to represent the line, store a representation of the cursor positions in a position file, and modify the document to include an element which references the position file. A system as in claim 34, in which the element comprises a Cascading Style Sheet (CSS).

36. A system as in claim 35, in which the CSS specifies a position for the image and a Z-order for the \\melb_files\home$\flanagan\keep\SPECIFICATIONS\51367-99.doc 21/03/03 27 position file.

37. A system as in claim 34, in which the position file comprises a Portable Network Graphic (PNG) file.

38. A system as in claim 33, in which the annotator is configured to enable a user to select a color for the line.

39. A system as in claim 33, in which the annotator is configured to cause a color pick palette to be displayed which enables a user to select a color for the line.

40. A system as in claim 1, in which the annotator is configured to enable a user to highlight selected text on the page.

41. A system as in claim 40, in which the annotator is configured to modify the document by changing a color attribute of highlighted text to a selected highlight color.

42. A system as in claim 40, in which the annotator is configured to enable a user to select a color for the text.

43. A system as in claim 40, in which the annotator is configured to cause a color pick palette to be displayed which enables a user to select a color for the text.

44. A system as in claim 1, in which the annotator is configured to enable a user to create a scrapbook file, and insert a selected section of a page into the scrapbook file using the input unit. \\melb_files\homeS\flanagan\keep\SPECIFICATIONS\51367-99.doc 21/03/03 28 A system as in claim 44, in which the annotator is configured to enable a user to block and copy a selected section and paste the selected section into the scrapbook file.

46. A system as in claim 1, in which the program further comprises an electronic mail module configured to transmit the document as an electronic mail message.

47. A system as in claim 46, in which the electronic mail module is configured to compress and transmit the document as a self-extracting file.

48. A system as in claim 46, in which: a local copy of an ancillary file which is referenced by a hypertext link in the document is stored Sin the storage; and the electronic mail module is further configured to-transmit the local copy together with the document. .r 49. A system as in claim 48, in which the-electronic- mail module is configured to compress and transmit the document and the local copy as a single self-extracting file.

50. A system as in claim 1, in which the program further comprises a printing module configured to print the page.

51. A system as in claim 50, in which the printing module is configured to draw the page to a virtual screen to capture actions performed and thereby render content, parse and break down the actions and render the actions into pages which fit in a designated printer, and render the pages to the printer.

52. A system as in claim 1, in which the program \\melbfiles\homeS\flanagan\keep\SPECIFICATIONS\51367-99.doc 21/03/03 29 further comprises a preprocessor for preprocessing the document into a predetermined format which facilitates annotation.

53. A system as in claim 52, in which the preprocessor is configured to determine if a hypertext link which references an ancillary file of a predetermined type is present in the document and, if a link is present, store a local copy of the ancillary file in the storage and modify the document by changing the link to reference the local copy.

54. A system as in claim 52, in which the predetermined type of ancillary file is an image file.

55. A system as in claim 52, in which the preprocessor is further configured to modify the document such that the page has a predetermined, fixed.width.

56. A system as in claim 52, in which the preprocessor is configured to modify the document such-. S that the page is enclosed in a table having a -predetermined fixed width. 25 57. A system as in claim 52, in which the preprocessor is further configured to modify the document to disable word wrapping. *0

58. A system as in claim 52, in which the preprocessor is further configured to determine if an external hypertext link of a predetermined type is present in the document and, if present, deactivate the external hypertext link.

59. A system as in claim 52, in which the preprocessor is further configured to determine if an active hypertext link of a predetermined type is present \\melb_files\home$\flanagan\keep\SPECIFICATIONS\51367-99.doc 21/03/03 30 in the document and, if present, deactivate the active hypertext link. A system as in claim 52, in which the preprocessor is further configured to modify the document by fixing a default font setting.

61. A system as in claim 1, in which the program further comprises a postprocessor for postprocessing the document into a predetermined format which consolidates annotation.

62. A system as in claim 61, in which the postprocessor is configured to determine is a hypertext link which references an ancillary file of a predetermined type is.present in the document and, if a link is present, store a local copy of the ancillary file in the storage S and modify theldocument by changing the link to reference the .local copy. 0 *0 0 9 0*O 9 V00' *00. 9* U @9 9 9* 09 A system as in.claim 61, in which the predetermined type of ancillary file is an image file.

64. A system as in claim 61, in which the 25 postprocessor is further configured to modify the document such that the page has a predetermined, fixed width.

65. A system as in claim 61, in which the postprocessor is configured to modify the document such that the page is enclosed in a table having a predetermined, fixed width.

66. A system as in claim 65, in which the postprocessor is further configured to modify the document to disable word wrapping.

67. A system as in claim 61, in which the \\melb_files\home$\flanagan\keep\SPECIFICATIONS\51367-99.doc 21/03/03 31 postprocessor is further configured to modify the document by fixing a default font setting.

68. A system as in claim 1, in which at least part of the program is downloaded from a server together with the document.

69. A system as in claim 68, in which the program comprises an applet. A system as in claim 68, in which the program comprises a control.

71. A system as claimed in any one of claims 1 to substantially as herein described with reference to the accompanying drawings. V,06 9. 9. 0 0050 0*00 a 0 S S 5*5 Dated this 21st day of. March 2003 INSIGHT DEVELOPMENT CORPORATION 20 By their Patent Attorneys GRIFFITH HACK Fellows Institute of Patent and Trade Mark Attorneys of Australia \\melbfiles\homeS\flanagan\keep\SPECIFICATIONS\51367-99.doc 21/03/03