JP2006518055A - Technological innovation on web page display - Google Patents

Abstract

The web page is displayed in simultaneous view and enlarged view. The indicator can display the overview portion in the enlarged view. Both views can be displayed so that one is above the other, exceeding the maximum width of the same screen. The user can select among split views and other views such as an overview-only view, an enlarged-only view, and a view where text selected to fit the width of the enlarged view is laid out. The navigation input can move the layout directly in the enlarged view or the cursor, and can scroll both the overview and the enlarged view. An enlarged view can display text in an anti-aliased font designed for that resolution. A magnified view can be created to function like a magnifying glass. The text width in a multi-column layout can be limited to fit the width of the view window, such as an enlarged view.

Description

  The present invention relates to a method, system, and program for displaying a web page, and in particular, a small screen, a low resolution screen, or a screen that occupies a relatively small portion of the user's field of view and is sufficiently distant from the viewer. The present invention relates to a method for displaying a web page useful for display on a screen.

  This patent application is wrist- or head-mounted for browsing handheld computers, cell phone computers, the World Wide Web (WWW), or similar media, although several features are available on large screen computers. It has many features relating to improving the use of computer devices with small screens or low resolution screens, such as computers with type displays.

  At the time of filing of the present application, there are a plurality of handheld computers, such as so-called “personal digital assistants (PDAs)” having about 240 × 320 pixel screens about 4 inches diagonal. These include Compaq's iPaq Pocket PC, Casio's Cassiopeia, and Hewlett-Packard's Jornado. Unfortunately, such resolutions are too low to display most current web pages in a way designed for viewing. Currently, most web pages can be viewed fairly well on a 640 x 480 resolution screen (although some websites cannot even be viewed easily even at this resolution). It would be desirable to be able to view most web pages using such a handheld device in a way that makes it easier to view the website on a larger screen.

  Also, at the time of filing this application, there are a plurality of mobile phones that are specifically designed to allow users to view web pages. For example, Microsoft is supporting the software standard for “smartphones” for mobile phones that have screens of 176 × 220 pixel size and have web browsing software. Unfortunately, the browsing software provided as part of the package allows users to browse most web pages on a small screen in a way that makes it easier for users to view web pages on larger computers. Absent.

  Other phone manufacturers offer mobile phones with other screen sizes. For example, Nokia currently sells mobile phones with a 176 × 208 pixel size screen. However, at this stage, on the screens of these computers that make it as easy to browse as many web pages on a screen that is commonly found on desktop and laptop computers. There is no software for browsing web pages.

  Currently, liquid crystal display manufacturers can produce small screens with substantially higher resolution than display devices currently on the market. Organic LED display manufacturers claim that even higher resolutions are feasible. This means that a diagonal 4 inch screen of the size currently installed in the handheld computers listed above can have a resolution of 480 × 680 or greater. Such screens provide acceptable resolution for many websites, but a more effective high resolution is desirable for viewing many web pages.

  Furthermore, in order to view such a screen at a relatively high resolution, the screen must be kept close to the user's field of view. This can be met in many applications, but in many cases the user may feel tired or inconvenient to keep the handheld computer close to the field of view. is there.

  Also, such advances in display resolution mean that a 320 × 240 pixel screen can be manufactured with a diagonal length of 2 inches or less. Such a display device is smaller than the size of a display device generally mounted on many multimedia mobile phones at present, and can be adapted to a wristwatch. Such a display would make many application types currently used in handheld computers available on cell phones, watches, or other similar small computers. Unfortunately, these display devices have the problem of having a relatively low resolution, which tends to make it difficult for users to view most web pages, and because they are physically too small, at these resolutions by the user. In order to enable browsing, the handheld computer has to be installed at a position very close to the user's field of view. Furthermore, maintaining such a device close to the user's field of view would be fine for a predetermined time, but may be inconvenient for long periods of time or under certain circumstances.

  Currently receiving light reflected to the user's eyes via an external device such as glasses or from a mirror placed at a position offset above, below, or sideways of the user's eyes. There are several companies that provide head mounted display devices that allow users to view images on a computer screen. In many cases, such projected computer screens occupy a relatively small area of the user's field of view so that the user can easily interact with the surroundings while using such a head mounted display. It is desirable to make it. As such, users of such display devices may face many of the same problems faced by users of small handheld screens.

  Some aspects of the invention relate to a method for optimizing the browsing of web pages on a computer having relatively limited computing power, memory, or Internet line capacity. For example, standard web browsers of the type currently used on most desktop and laptop computers require many megabytes of memory and a relatively large amount of computing power. They also require an Internet connection with at least the speed of a high-speed modem in order for the web content types contained in many of the more frequently used web pages of the World Wide Web to operate at the desired speed. And Unfortunately, many handheld computers do not have storage or computing power that can effectively display many of these web pages. Also, the most commonly available wireless systems have a line capacity that is substantially lower than the preferred line capacity for many web pages. As a result of these factors, one of the focus of some of the technological innovations included in this application is that, with limited storage, communication speed, and line capacity, the computer The present invention relates to a method capable of browsing the wide web and similar media more appropriately.

  The present invention relates generally, but not necessarily, to a method for displaying media on a small computing device such as a cellular phone, personal digital assistant (PDA), wristwatch computer or the like.

  The first feature is that the present invention has a plurality of features, all of which are related to a media display method having the use of split views.

  Such split view methods include receiving digital content representing the displayed web page, the layout of the web page, and the creation of a split view of the web page. Simultaneously displaying a portion of a web page laid out at the first magnification in the overview window, and a second magnification that makes the layout elements appear larger in the enlarged view window compared to the overview window Then, using the display in the enlarged view window, the divided parts are formed by displaying the subparts of the layout portion displayed in the overview window in the enlarged view window. The split view also includes a display in the overview window of an enlarged view indicator showing some subparts of the layout displayed in the overview window currently displayed in the enlarged view window.

  The split view method of the present invention also includes a method that allows the user to navigate the boundaries of the portion of the layout displayed in the enlarged view. In such magnified view navigation, the subparts of the layout displayed in the magnified view window are moved, while the magnified view indicator is moved relative to the layout displayed in the overview window. In response to navigation input from the user, so that the subpart of the layout shown in the enlarged view window will continue to correspond to the subpart of the layout indicated by the enlarged view indicator after such movement .

  In some enlarged view navigation methods, the overview window and the enlarged view window are displayed simultaneously on different parts of a single rectangular display screen having a predetermined width and height, and the overview window and Each of the enlarged windows is as wide as the screen, but each occupies a different portion of the screen height.

  In some embodiments relating to features of the present invention, the height of the enlarged view window occupies less than half the combined height of the enlarged view window and the overview window.

  In some embodiments relating to this aspect of the invention, the screen has a vertical direction relative to the direction in which the web page layout is displayed in the enlarged view window and overview window. In some such embodiments, each of the overview window and the enlarged view window has a width that is longer than the height, so that the screen on which they are both displayed has a vertical orientation. Despite the fact that each of them will have a lateral direction.

  Some of the split view methods are multi-view methods. As described above, these methods create a first view of a web page and a second view that does not have a combination of split views with respect to an overview window and an enlarged view window. Includes responding to the first view of selecting input from the user by responding to the second view of selecting input from the user. When a split view is displayed, the method corresponds to navigation input from the user, as described above.

  In some embodiments relating to this aspect of the invention, the first and second correspondences are different hardware or software button or key presses, and in other embodiments it is If the method user interface is in a different state, the same button may be pressed.

  In some such multi-view methods, the second view is an overview only view. In this view, a portion of the web page is displayed at a magnification that allows the layout elements to appear smaller than in the enlarged view. In this overview only view, the sum of the display size and the combined size of the overview window and the enlarged view window in the divided view is substantially equal.

  In many embodiments, the magnification of the overview window for the overview only view and the split view is the same.

  In some multi-view methods, the second view is an enlarged only view. In this view, a portion of the web page layout is displayed at a magnification that makes the layout elements appear larger in the overview window. In the enlarged only view, the sum of the display size and the combined size of the enlarged view window in the overview window and the divided view is substantially equal.

  In many such embodiments, the magnification of the magnified view and the magnified view window of the split view are the same.

  In some multi-view presentation methods, the second view is a reflowed text window. The response to the second view selection input can be, for example, a column of text in which the cursor or magnified view indicator is located before receiving the second view selection input, or at this time the cursor or magnified view indicator. Performing a second layout of the selected portion of the text of the web page, such as a column of text closest to.

  The second layout is new with one or more font sizes associated with the new column line width greater than the selected text font size associated with the selected text layout width in the split view. Includes reflowing selected portions of text beyond the line of the text column. The maximum width of this new text column is displayed on the screen in the reflowed text window.

  In some such embodiments, the reflowed text window of the second view is substantially as large as the combination of the enlarged view window and the overview window of the split view.

  In some embodiments using such a reflowed text window, the second view is a second form of split view that simultaneously includes the display of the reflowed text window and the second overview window. The second overview window is a part of the web page layout, with a scale factor that makes the layout elements look smaller in the enlarged view and text in a smaller font in the reflowed text window. indicate.

  In such a reflowed text split view, the second overview window preferably includes a display of a portion of the text currently displayed in the reflowed text window. For example, such displayed text indicators include displaying text using an overview window in which the reflowed text window in reverse video is located, or a box surrounding it.

  In embodiments having a reflowed text window, the method preferably includes responding to navigation input from the user by moving a portion of the text displayed in the reflowed text window. . In those embodiments having a text indicator displayed in the simultaneously displayed overview window, the displayed text indicator is moved relative to the layout displayed in the overview window in response to such navigation input. Thus, it is desirable for the text subpart displayed in the reflowed text window to continue to correspond to the text subpart indicated by the displayed text indicator after such movement.

  Of course, the use of the present invention for reflowed text windows of the type just described in media display also includes a multi-screen mode that can also display an overview window of the type described above and an enlarged window. It is not limited to the view method.

  For example, according to some aspects of the present invention, there is provided a media display method for receiving digital content displaying a displayed web page and then performing a first layout and a second layout of the web page. Is possible. The second layout has one or more larger font sizes corresponding to the new column line width compared to the selected text font size corresponding to the selected text width in the first layout. To reflow the selected portion of the text beyond the line of the new text column. This method then creates a split view of the web page. This split view includes a display of a portion of the web page layout at the first magnification in the overview window. It also includes the simultaneous display of the reflowed selected text in the reflowed text window. The maximum width of the new text column is that the second part of the text that is displayed in the reflowed text window is displayed in a larger font in the reflowed text window on the screen at a second magnification. Is displayed.

  According to some aspects of the present invention, the web page layout displayed in the overview window prior to generation of the navigation input over the previously hidden portion of the web page layout, corresponding to the navigation input. A split view of the overview / enlarged view is provided if it includes responding to navigation inputs that move a predetermined portion (ie, all or part) of the enlarged view indicator beyond a portion of The The method responds to such input by scrolling the web page layout in response to the overview window, thereby scrolling the previously hidden portion into the overview window. It moves a predetermined portion of the enlarged view indicator over the previously hidden portion of the web page layout. Also, in contrast, it scrolls the layout subparts displayed in the enlarged view window, so that the subparts displayed in the enlarged view window are indicated by the enlarged view indicator after such movement. Corresponds to the web page layout subpart.

  According to some features of the invention, a split view of an overview / enlarged view is provided when the correspondence to the navigation input includes corresponding to the first form and the second form of the navigation input separately. Is done. The method corresponds to a first form of navigation input by directly moving an enlarged view indicator and a portion of the web page layout displayed in the enlarged view window. It corresponds to the second form of navigation input by moving the cursor directly in relation to the part of the web page layout displayed in the enlarged view.

  In general, when viewing a web page, the user is interested in selecting a link and entering information in the web page controls and fields. To do this, the user often wants to be able to navigate the cursor in relation to the enlarged display. However, when browsing or reading a web page, the user is often most interested in moving the magnified view itself. This feature of the invention allows the user to have both of these two forms of navigation associated with an enlarged view.

  In some embodiments relating to this dual navigation feature of the present invention, the correspondence to the second form of navigation input is the web displayed in the enlarged view window prior to the formation of the second form of navigation input. It includes a response to such input that moves the cursor over a portion of the web page layout that was previously hidden in the enlarged view window beyond the page layout subpart. Such embodiments include (1) moving the cursor over a portion of the web page layout that was previously hidden in the enlarged view window beyond the sub-part of the web page layout displayed in the enlarged view window. (2) Scroll the web page layout in relation to the enlarged view window, so that the part that was hidden before the cursor was moved is scrolled in the enlarged view window (3) in contrast, scrolling through the enlarged view indicator in the overview window, so that the sub-part of the web page layout indicated by the enlarged view indicator is magnified Web page layout displayed in the selected view window And to respond continuously to subparts out, corresponds to such input by. Thus, the user can scroll a part of the web page layout displayed in the enlarged view by moving the cursor with respect to the boundary of the view.

  In many embodiments of the dual navigation feature of the present invention, a cursor is displayed at the same location in relation to the web page layout in both the enlarged view window and the overview window.

  According to another aspect of the present invention, it is described in the previous paragraph, except that it does not necessarily include a correspondence to the form of navigation input, which is different from that for moving the cursor directly. A split view of the overview / enlarged view similar to the features of the present invention is provided. In accordance with this aspect of the invention, the response to navigation input has been (1) previously hidden in the enlarged view window, beyond the subpart of the web page layout previously displayed in the enlarged view window. Move the cursor over a part of the web page, and (2) the part that was hidden before the cursor was moved will be scrolled and displayed in the enlarged view window. Scrolling the web page layout in relation to the magnified view window; and (3) correspondingly scrolling the magnified view indicator in the overview window, thereby zooming in The sub-part of the web page layout indicated by the view indicator is an enlarged view window. By continuing to respond to the web page layout subparts displayed in the dough, it is possible to zoom in beyond the web page layout subparts displayed in the enlarged view window before forming the predetermined navigation input. Corresponding to a predetermined navigation input that moves the cursor over a portion of the web page layout that was previously hidden in the displayed view window.

  This scrolling preferably includes both horizontal scrolling and vertical scrolling, both in an enlarged view window and an overview window, relative to the web page layout, so that it is moved in response to navigation input. In this case, it is desirable that the cursor is displayed at the same position.

  According to one aspect of the present invention, when the magnified view window moves relative to the overview window, a split view of the overview / magnified view is provided and looks like a magnifying glass. . According to this magnifying glass feature of the present invention, the navigation input from the user moves the enlarged view window relative to the overview window and correspondingly displays it in the enlarged view window. Moving the enlarged subpart of the layout to be handled. As a result, the portion of the web page layout displayed in the enlarged view window continues to correspond to the subpart of the covered subportion of the layout displayed in the overview window after such movement.

  Some embodiments of this magnifying glass feature of the present invention further illustrate the location of the subparts of the layout displayed in the enlarged view window relative to the layout displayed in the overview window. Or displaying a plurality of enlarged subpart markers. In some such embodiments, the one or more enlarged subpart markers extend along a width and / or height subpart of one or more edges of the enlarged view window. , Horizontal and / or vertical extents, and layouts displayed in the enlarged view window in relation to the layout displayed in the overview, respectively.

  In some embodiments relating to the magnifying glass features of the present invention, the magnified view is smaller in width than the overview window so that it can be moved horizontally and vertically in response to navigation input. And both have a height.

  However, in some embodiments relating to this aspect of the invention, the enlarged view window has substantially the same width as the overview window. Such an embodiment separately accommodates most horizontal and vertical navigation inputs. They were often displayed in (1) the enlarged view window in relation to the overview window and (2) in the enlarged view window in relation to the layout. Moving the position of the enlarged subpart of the layout in the vertical direction; and (3) if necessary, in the vertical position of the enlarged subpart of the layout in relation to the layout displayed in the overview window. To accommodate changes, one or more enlarged subpart markers are moved in the vertical direction to accommodate vertical navigation input. Such embodiments often include (1) horizontally moving the position of the enlarged subpart of the layout displayed in the enlarged view window relative to the layout, and (2) as required. In relation to the layout displayed in the overview window, the layout is expanded in relation to the layout displayed in the overview window to accommodate changes in the horizontal position of the expanded subpart of the layout. By moving one or more magnified subpart markers to accommodate changes in the horizontal position of the subpart horizontally, the magnified view window is associated with the overview window. Corresponds to horizontal navigation input without substantial movement.

  In this description, the horizontal direction refers to the direction in which text is laid out on a line on the display. The advantage of having the enlarged view as wide as the screen is that if the text is most readable, it will allow the enlarged window to show the wider part of the layout, In this way, the user's ability to read the entire width of the text column is improved without having to move the enlarged view.

  Other embodiments having a magnifying glass that is substantially the same width as the screen can be used in a horizontal direction by moving the overview display of the layout in the horizontal direction relative to an enlarged view that is fixed in the horizontal direction. It is possible to deal with navigation input.

  Another aspect of the invention relates to a method for displaying a web page that limits the width in which the web page text column is displayed.

  The method relating to this aspect of the invention accesses the contents of a multi-column web page, including a plurality of displayable text strings and a plurality of column specifications. Each column specification includes a character string specification in the column, a column layout width, and a column horizontal layout replacement. Such a specification may be explicit or implicit. In some cases, they define the column width relative to the screen area in which the column is displayed, and sometimes specify it in terms of the entire pixel.

  When accessing the web page, the method lays out the web page in a multi-column format having a layout magnification. If the horizontal layout position of each column is determined as a function of the column's horizontal replacement specification, this layout includes determining the layout position for each of the columns.

  The method allows a user to select a predetermined subpart of a multi-column layout from among a plurality of different horizontally substituted layout subparts, which has a predetermined horizontal resolution at a predetermined display magnification. A portion of the layout selected by the user on a predetermined screen window is displayed. The multi-column layout is a multi-column layout from the leftmost end of the leftmost column of a set of columns to the rightmost end of the rightmost column of this set of columns. It has a set of laterally arranged columns having a total width. The layout includes limiting the width in which text in one or more horizontally arranged columns is laid out to a width that is less than or equal to the maximum desired column width. This restriction changes the width of the column if the width corresponding to the specified layout width of the column is longer than the maximum desired column width, as scaled by the layout magnification.

  The maximum desired column width is a width in a layout equal to or less than the width of a predetermined screen window when displayed in the window at a predetermined magnification. In many embodiments, the maximum desired column width corresponds to a pixel width that fits within the display window, with a resolution that allows accurate and / or easy reading of the column text.

  In some embodiments relating to the column width limitation feature of the present invention, the content of the accessed web page includes one or more images, each column specification being text and / or one. Includes a specification where one or more images are located in a column. Web page layout includes laying out the text and / or the location of one or more images located in the column in each column. The width of text that is laid out in one or more horizontally arranged columns is limited if the width corresponding to the image is larger than the maximum desired column width, as scaled by the layout magnification. Reducing the size of such images includes limiting the width of any image laid out in such columns to a width less than or equal to the maximum desired column width.

  In many column width restrictions embodiments where one of the horizontally arranged columns has a layout width corresponding to a specified width of the column that is less than the maximum desired column width, the text in that column is In many cases, the layout is performed with a layout width corresponding to the specified width of the column.

  In many column width limitation embodiments, the width laid out to the right of the first column is reduced if the width in which the first column of text in the horizontally arranged column is reduced by the width limitation. The second column of orientated columns is related to the horizontal replacement specification of the second column, as scaled by the layout magnification as a function of the reduction in the layout width of the text in the first column. Laid out with reduced horizontal right permutation.

  In many column width limiting embodiments, the layout scale selected by the user on a given screen window at a given display magnification, depending on the layout scale and / or display scale, may be related to the text column layout width restriction. Independently, it displays web page elements containing text characters at a reduced pixel resolution. In such embodiments, their specified column widths would otherwise make their laid out widths too wide to fit within a given screen window even with such reduced pixel resolution. If this is not possible, the column width is limited.

  Some column width limited embodiments create a split view of a web page that displays both an overview window and an enlarged view window simultaneously, similar to the previous embodiment, where the enlarged view window is This is a window in which a portion selected by the user of the layout is displayed. In these embodiments, the maximum desired column width is selected to ensure that a portion of the layout having a width equal to or less than that width fits within the enlarged view window. ing.

  In many embodiments of the present invention described above having an enlarged view window, the text of the web page displayed in the enlarged view window may improve readability at the resolution displayed in that window. It is displayed in a font bitmap having a shape and pixel arrangement for those selected character outlines. Here, it is assumed that such a font bitmap is optimized for display at a display resolution.

  In many such embodiments, the resolution optimized font bitmap used is typically used for web browsing, eg, 10 pixels / em or less, or 8 pixels / em or less. It is smaller than the font bitmap. This is because in many such embodiments, the magnified view is magnified in relation to the overview, but it is actually at a much smaller scale than normally displayed on desktop and notebook computers. Is displayed.

  In some such resolution-optimized font bitmap embodiments, the web page layout defines the layout of the font bitmap used in the enlarged view window, and the web page layout displayed in the overview window. Display includes a reduced, low pixel resolution representation of the font bitmap included in the layout. The low resolution representation displayed in the overview does not have the character shape and the pixel arrangement selected to improve readability at the low resolution used in the overview window.

  In some such resolution-optimized font bitmap embodiments, the font bitmap used to show characters in the enlarged view window is displayed in the enlarged view window of the font bitmap. An anti-aliased font bitmap in which anti-aliasing is performed for a given resolution.

  In some such resolution-optimized font bitmap embodiments, the text displayed in the enlarged view window is composed of letters of the Latin alphabet, with the majority of lowercase letters having an advance width of 4 pixels or less. Displayed in the displayed font.

  In some such resolution-optimized font bitmap embodiments, the overview window and the enlarged view window are individually addressable and have screens with pixels configured in a predetermined arrangement with respect to different color sub-pixels. Displayed above. An anti-aliased font bitmap used to show characters in the enlarged view window assigns a luminance value to each predetermined sub-pixel of a screen pixel having a predetermined arrangement of sub-pixels of different colors It is an optimization bitmap.

  This color assignment is: (1) a coverage value indicating the percentage of a given subpixel covered by the character shape represented by the font bitmap; and (2) at least for some subpixels of the font bitmap. From a predetermined coverage value, which is performed to prevent color imbalance resulting from a difference between a coverage value of a given subpixel and a coverage value of a set of neighboring subpixels of one or more different colors. A color balance distribution for the ratio of the coverage value of a given sub-pixel to neighboring sub-pixel coverage values including sub-pixels of different colors, and (3) one or more for at least some of the sub-pixels of the font bitmap Coverage of a given subpixel from coverage values from subpixels near To Tsu di value, one or more color balance distribution is a function related. The color balance distribution is substantially limited to the distribution of some of the subpixel coverage values that cause color imbalance within all pixels that are part.

  In some embodiments relating to various aspects of the present invention, the digital content includes both images and text strings, and the layout of the web page is displayed on each of the images and on a predetermined line. Laying out the image and text at a virtual pixel resolution that uses the virtual layout pixel size for the image and text to assign horizontal and vertical positions in the layout to each part of the text string .

  In such virtual layout embodiments, the layout scale factor determines the layout of the web page layout displayed in the enlarged view window relative to the corresponding position in the layout performed at the virtual pixel resolution. It has a non-single value that allows it to be displayed at a pixel resolution that is scaled down.

  In many such virtual layout embodiments, the image and text in the enlarged view window correspond to the position of the image and text in the virtual web page layout as reduced by the layout magnification. Displayed in pixel coordinates. The image and text are drawn in the enlarged view window with a reduced pixel size corresponding to the pixel size used for the image and text in the virtual layout, as reduced by the layout magnification. .

  In many such virtual layout embodiments, the text string image in the enlarged view window is scaled to the size assigned to the text string in the virtual layout. Consists of a series of font bitmaps with reduced pixel size. The character shape and pixel arrangement represented by the font bitmap used to display one of the characters in the enlarged view window is the resolution that is displayed in the enlarged view window of the font bitmap. Along with the pixel boundaries of the bitmap, it has been selected to increase the percentage of the array of character-shaped edges.

  In many embodiments relating to features of the present invention related to the limitation on the width in which a web page column is displayed, it is also used with a resolution optimized font bitmap of the type described above.

  In some embodiments relating to various aspects of the above-described invention, the web page content is displayed on the screen of a client browser computer that relies on a remote proxy server to perform a portion of the browsing process, such as the layout of the web page. Is displayed. In many embodiments, the client computer is a cellular phone, a personal digital assistant (PDA), or other small computer that can benefit from having layout processing performed by a large computer.

  In such client proxy embodiments, the client includes a program that allows the user to view the digital content. The client computer includes a viewing program corresponding to user input requesting a predetermined web page by requesting the web page from a proxy server on the computer network. The proxy server communicates user requests to one or more servers that store or dynamically form the requested web page over the computer network. One or more servers send the requested web page to the proxy server. The proxy server receives the downloaded web page as digital content. The proxy server executes the layout of the web page. The proxy server downloads the web page layout to the client computer. Thereafter, the browsing program executed on the client computer generates a divided view by displaying the downloaded web page layout in the overview window and the enlarged view window.

  The present invention also provides a computer device embedded in a program and / or hardware to perform the functions described above in the summary of the invention (i.e., means for solving the problems), and It relates to a program recorded in a machine readable form for causing a computing device to perform such a function.

  Other features of the invention not summarized above will become apparent in the "Detailed Description of Some Preferred Embodiments" below (ie, the best mode for carrying out the invention). The

  These and other features of the present invention will become more apparent upon reading the following description of the best embodiment in conjunction with the accompanying drawings.

  FIG. 1 illustrates several features of the present invention to improve the display of web browsing and / or other types of computer generated content, particularly on systems having low resolution display devices. Figure 6 is a high level drawing showing basic processing and data display that may be used.

  The digital content 100 including one or more bitmap images 102 and text 104 illustrated at the top of FIG. 1 is displayed in a sub-pixel optimized reduced format 106 displayed at the bottom of the drawing. ing.

  In one embodiment of the invention, the difference processing comprised in step 108 is used to subpixel optimize the display of the bitmap image 102, rather than being used to subpixel optimize the display of text content. Used to do. Step 108 uses a subpixel optimization routine that is particularly suitable for generating a subpixel optimized image from a color bitmap. Process 108 also reduces the bitmap for display on a screen having a resolution lower than the resolution at which most web content is currently displayed.

  Text 104 included in digital content 100 is processed for display on a low resolution subpixel addressable screen by using steps 110 and 112. Step 110 replaces the font used for normal text display with a font optimized for low resolution display on a subpixel optimized screen. Step 112 was generated by a subpixel optimization routine that is particularly suitable for representing high-resolution images of the same color shape, such as mathematically defined outlines commonly used to define font shapes, for example. Use font bitmaps from alternate fonts.

  One use of the present invention is in portable, low-resolution web browser content that displays images and / or text expressed in a markup language downloaded from the Internet. .

  To date, many so-called markup languages exist. One of the oldest and most supported languages is SGML (Standard General Markup Language). SGML is a text-based language that can be used to “mark up” data with descriptive “metadata” that provides information about the data. As an example, markup metadata can be used to indicate the intended purpose of the data or the location within the visual presentation of the document in which the data is placed. It can also be used to indicate links to other types of data, such as images, that are inserted at predetermined locations in text or in documents written in a markup language. Some markup languages commonly used today, such as HTML and XML, are derived from SGML.

  In the best embodiment of the present invention, the digital content 100 referred to in FIG. 1 is a standard web content including images and / or text expressed in a markup language such as HTML. It may be. This standard web content 100, possibly representing the home page of a website, is various devices and methods described below for display on the portable low resolution browser device 200 illustrated in FIGS. It is possible to download at. Prior to display on the browser device 200, the digital content 100 can be scaled and / or sub-pixel optimized by various methods and processes as described below, for example, to improve legibility. There is sex.

  FIG. 2 illustrates a networked computer environment implemented in accordance with one embodiment of the present invention. The thin client browser 200 program may be a handheld computer device capable of capturing and displaying text and / or graphics on a small display screen, such as a liquid crystal display (LCD) screen, or other Implemented in small computing devices. This browser allows the user to request digital information from a remote source (eg, the Internet) and display it on the screen.

  In the embodiment of the present invention illustrated in FIG. 2, the user requests the capture and display of digital content including images and / or text by operating the control buttons of the thin client browser 200. . The requested digital content may be a specific web page accessible via the Internet. The thin client browser 200 then executes the content request 202 via the proxy server 210 that is physically remote on the network 138, which can be, for example, a LAN, WAN, or the Internet.

  Proxy server 210 performs proxy process 216 for the request for digital content by generating request 214 corresponding to physically remote web server 220 having digital content 100 requested by the user. The download 220 of the digital content 100 over the network 138 to the proxy server 210 causes the server 220 to respond to the proxy server request 214.

  Next, proxy process 216 within proxy server 210 uses computational resources to scale digital content 110 and sub-pixel optimize, including the execution of functions 108 and 110 illustrated in FIG. . Scaling and sub-pixel optimization are features of the present invention that produce improved legibility of images such as text and / or graphics on small display devices. These will be discussed in more detail in the next section.

  The proxy server 210 completes the download 212 of the sub-pixel optimized content to the browser 200 here. At this point, the user can view the content on the browser 200 screen.

  In the embodiment of the present invention illustrated in FIG. 2, the text portion of the digital content includes a string of one or more characters and associated designations of font family, font size, and other font attributes. It is downloaded to the browser in a format that includes it. By displaying a string using an image composed of individual subpixel optimized font bitmaps, the thin client browser performs the function 112 illustrated in FIG. If such a string in a specified font size and font family does not have a bitmap for any character, the thin client requests one or more such bitmaps from the font server 230. In various embodiments of the present invention illustrated in FIGS. 2-8, the proxy server can provide such a font bitmap, and the thin client can increase the size of the browser software. However, it is also possible to have those bitmaps as a standard part of the software. In other embodiments, the font can be an outline font. One advantage of font bitmaps is that some font vendors tend to want to be able to distribute bitmaps of those fonts more freely than the outlines of those fonts. That is the point.

  One alternative embodiment relating to the present invention is illustrated in FIG. In this embodiment, the proxy server 210 and the web server 220 in FIG. 2 are replaced with one remote server 220A. The thin client browser 200 executes the request 202A for the digital content 100 to the remote server 220A on the network 138. For example, the network 138 may be the Internet or a LAN, and the digital content 100 may be a specific web page. The remote server 220A has the requested digital content 100 and executes a proxy process 216A that responds to the request 202A. The proxy process 216A may be any process executed on a server that dynamically scales and / or subpixel optimizes web content for display on a client browser. Proxy processing 216A is performed on the stored digital content 100 and dynamically converts the digital content 100 into the format 106 illustrated in FIG. 1 by performing steps 108 and 110 of FIG. The remote server 220A completes the scaling 212 and / or subpixel optimized content download 212 to the thin client browser 200.

  Another alternative embodiment according to the present invention is illustrated in FIG. Similar to FIG. 3, the request of the thin client is directly requested to the remote server (here, the server 220 </ b> B) on the network 138.

  In this embodiment, the remote server 220B requests in both the standard format 100 used on a standard browser computer and the scaled and / or sub-pixel optimized content 100A. Digital content. Conversion from standard digital content 100 to a scaled and / or sub-pixel optimized format is performed in advance, thereby eliminating the need for proxy processing to dynamically convert it. The thin client provides information to the server indicating that it should receive a scaled and / or sub-pixel optimized version of the requested content. The remote server 220B completes the download 212 of the scaled / reduced and / or sub-pixel optimized content 100 for the thin client browser 200.

  A further alternative embodiment is illustrated in FIG. The browser 200 </ b> A is a full-scale browser that further includes enlargement / reduction and / or sub-pixel optimization processing 510. The browser 200A makes a request 202B for the digital content 100 to the remote server 220C on the network 138. The server 220C completes the download 212A of the digital content 100 requested to the browser 200A. Enlargement / reduction of digital content 100 and / or conversion to a sub-pixel optimized format is handled by process 510 executed in browser 200A.

  FIG. 6 illustrates a single computer system 600 capable of scaling digital content 100 and / or subpixel optimization. In this best embodiment, the digital content has been previously generated or read in the computer system 600. The computer system 600 includes browser processing 620 that includes scaling the sub-process 640 and / or sub-pixel optimization. Here, the user requests the computer system 600 to display the digital content 100 using an attached input device (for example, a keyboard or a mouse). The browser process 620 captures the requested digital content 100 from one of the computer system's storage elements (eg, electrical memory or disk storage). Next, when the browser process 620 captures the digital content 100, the browser process 620 transmits the digital content 100 to a sub-process 640 that performs enlargement / reduction and / or sub-pixel optimization. When the conversion is completed, the converted content is displayed on the display screen of the computer system 600. This embodiment according to the invention operates without the need for a network or a remote server.

  FIG. 7 illustrates an alternative embodiment for a single computer system. In this embodiment, the computer system 700 includes digital content 100, such as pre-formed or read (eg, content of a specific web page), proxy processing 740, and browser processing 720. Yes. The proxy process 740 executes a scaling / subtraction and / or sub-pixel optimization program 760. The browser process transmits a user request for displaying the digital content 100 to the proxy process 740. Next, the proxy process 740 captures the digital content 100 from the storage element of the computer system 700. When the digital content 100 is captured, the program 760 is sent to a scaled and / or sub-pixel optimized format that is then sent to the browser process 740 for display on the display device of the computer system 700. The digital content 100 is converted.

  FIG. 8 illustrates a second alternative embodiment for a single computer system. Here, the computer system 800 includes web content 810 that is scaled and / or sub-pixel optimized. Browser processing 820 processes a request to display content 100A from a user, retrieves it from a storage element of computer system 800, and displays it on the screen of computer system 800.

  In the embodiment of the present invention shown above, the scaling of the image from the resolution of the source image to the screen resolution addressed in subpixels is partially addressed in the source image and subpixels. Is determined by each resolution of the displayed screen.

  In some embodiments of the present invention, the scaling determination between the resolution of the source image and the resolution displayed on the subpixel addressable display screen can be specified by the user of the browser device. In this embodiment, the browser user selects a magnification from a plurality of magnifications by transmitting the magnification to the process of reducing the image read from the storage device. Next, the process of reducing the image read from the storage device reduces the image and / or optimizes the sub-pixels at horizontal and vertical magnifications proportional to the selected magnification.

  As with most other users entering into a browser device, such a scale selection may be a physical or GUI button, menu item, dialog box, or other well-known user interface device on the browser device. This can be done by using

  In some such embodiments, the user of the browser device may re-enlarge / decrease the digital content after the image has been previously captured from the storage device and displayed in a sub-pixel optimized format at the first magnification. The second magnification may be selected from a plurality of magnifications according to reduction and sub-pixel optimization and re-display.

  In such embodiments, the scaling factor used in the first scaling / subpixel optimized display may result from a default or desired scaling factor, and It may be the result of a magnification previously selected by the user of the browser device. The user of the browser device may select a magnification from a plurality of magnifications for redisplaying the digital content by a method of operating the input device of the browser device. By operating the input device of the browser device, an image is displayed according to the second selected magnification.

  Such second enlargement / reduction may occur as a result of processing executed either in the browser device or in a physically remote server. The user of the browser device may continue to select a magnification from a plurality of magnifications for subsequent redisplay.

  With integer multiple reduction, it is easiest to reduce a digital image by an integer multiple that fits an integer number of pixels in the source image to a given pixel in the resulting reduced image. For example, the scale from 640 × 480 resolution to 320 × 240 resolution is a reduction of 2 magnification. In some embodiments according to the present invention, the user can select a magnification from a plurality of reduction magnifications including non-integer reduction magnifications. Examples of non-integer scaling factors are scaling and / or sub-pixel optimizing a 480 × 360 pixel portion of a 640 × 480 resolution source image for display on a 320 × 240 resolution display screen. This is an example of a reduction ratio of 3/2.

  In this specification, when referring to the resolution of a web page, it usually means the resolution at which the browser program that implements the layout is required to lay out the content of the web page. As shown below, some web pages have multiple elements that require a specific minimum layout width, and if the web page is required to be displayed with a width shorter than the minimum layout width, Even if the width is long, the elements are laid out with the minimum width necessary for them.

  For example, computer graphic display devices such as cathode ray tubes (CRTs) and liquid crystal display (LCD) screens almost always use RGB models for color spaces. However, the present invention is applicable to other color models such as the CMYK color model. In the RGB model, the additive three primary colors, red, green, and blue are mixed to form a desired color so that it can be recognized by the human eye.

  Most portable computing devices or imaging devices have an LCD screen that uses the RGB model. Such LCD screens are composed of a rectangular array called pixels consisting of thousands of grid elements, which form an image when recognized as a whole, and each pixel has many colors from the RBG color space. One of the values can be displayed. LCD screens are characterized by the number of horizontal and vertical pixels they have.

  Similarly, each pixel is made up of three individually addressable subcomponents, referred to herein as subpixels. Most commonly, subpixels are red, green, and blue rectangular elements. In the most common embodiment, the three subpixels of red, green and blue are each assigned a value of luminosity so that the subpixels are mixed and the entire pixel looks the desired color. Similarly, all the pixels on the LCD screen are mixed so that the desired image is visible.

  The subpixels are said to be individually addressable, because the color values assigned to individual pixels have separate red, green, and blue color components or luminance values. This is because the pixel is displayed by red, green, and blue sub-pixels. Therefore, it is possible to control the brightness of each sub-pixel separately by controlling the value of the brightness value of the associated color component in the color value assigned to the pixel.

  In LCD devices and other “subpixel addressable” display devices, such as color LEDs (including screens using organic light emitting diodes (OLEDs)) and gas plasma display devices, each subpixel is Has a fixed known position on the display. For example, many display devices, such as almost all cathode ray tubes (CRTs), are not subpixel addressable. For example, each pixel of a CRT has a separate luminance value for each of the red, green, and blue color components, but within each such pixel of the element that produces light associated with those different color values. The exact physical arrangement of the is usually unknown. This is because it is proportional to a function of the screen's individual emission pattern, the resolution of the horizontal and vertical scans, and the current exact state of the voltage that controls the exact position at which the individual pixels are drawn on the screen.

  FIG. 9 illustrates a 12 × 12 portion of an LCD screen 900 comprised of a plurality of pixel rows (R1-R12) and pixel columns (C1-C12). Each intersection of a row and a column constitutes a pixel element. An actual LCD screen can have any number of matrices. However, frequently seen are 320 × 240, 640 × 480, 800 × 600, 1024 × 768, and 1280 × 1024 grids.

  Pixels R1-C1 are illustrated within a dashed line circle 910. The pixel R1-C1 itself is composed of three pixel subcomponents (hereinafter referred to as subpixel elements). An enlarged view of pixels R1-C1 is shown as enlarged pixel 920 at the bottom of FIG. Subpixel element 902 is displayed as red, subpixel element 904 is displayed as green, and subpixel element 906 is displayed as blue. The individual sub-pixel elements 902, 904, 906 are approximately one third the width of the entire pixel and the height is equal to the height of the entire pixel.

  As shown in the LCD screen 900, when such pixels are arranged in a grid, vertical color stripes appear on the LCD screen 900. This well-known pixel arrangement is sometimes referred to as a vertical RGB stripe. In other known arrangements, the pixel elements are laid out in a perpendicular direction that results in horizontal stripes (in this case, the horizontal stripe screen is converted to a vertical stripe screen by rotating the screen 90 °).

  In general use, the luminosities for the three sub-pixel elements of a pixel are set so that the pixel is perceived with the desired hue, saturation, and brightness by the human eye. The RGB subpixel elements are used simultaneously to form a single color pixel to represent one sample of the displayed image.

  One feature of the present invention is, for example, a display having a matrix pixel ratio of 320 × 240 or 240 × 320 (in this case, it can be rotated 90 degrees to have a resolution of 320 × 240). It relates to improving the readability of downloaded web content, other digital content including text and images on low resolution screens. Many of the embodiments of the present invention discussed and illustrated in detail map images and text from a virtual layout resolution of 640 × 480 to a screen having a 320 × 240 pixel resolution. However, the present invention can be used with other resolution screens. To name just a few, it can be used to display content roughly laid out to see 1024x768 resolution on a 512x384 resolution screen, and 800x600 pixels on a 400x300 screen. It can be used to display content that is roughly laid out for viewing. In other embodiments, the present invention is used in relatively low resolution displays having a horizontal pixel dimension and / or a pixel dimension other than a vertical pixel dimension common on personal computer screens. The

  In general, when referring to a low-resolution screen, it means a screen that has a lower resolution than a screen on which a predetermined digital content or a predetermined layout of digital content is normally displayed. It also means that such a small screen includes part of a large screen having such a low resolution, for example a window on a larger screen.

  In FIG. 10, the image content 105 and / or the text content 107 shows a part of the sub-pixel optimized display of FIG. The image illustrated in FIG. 1 is a grayscale magnification of actual color values associated with sub-pixel optimized display of both text and images. The portion of the image content 105 within the rectangular portion 1000 of FIG. 10 is enlarged at 1020 to make it easier to see the individual pixels. Correspondingly, a part of the text content 107 included in the rectangular portion 1040 is enlargedly illustrated at 1060.

  It is important to note that the pixels illustrated at 1020 and 1060 represent the entire pixel, which means that the software used to generate images 1020 and 1060 is not This is because the gray scale corresponding to the RGB color values associated with each is simply expressed. Sub-pixel magnified displays 1020A and 1060A are intended to represent the brightness of each of the three sub-pixels associated with each pixel in magnified displays 1020 and 1040, respectively. 1020B is an enlarged display whose scale and position correspond to the enlarged displays 1020A and 1020. There, the pixel grid of the image is displayed with relatively thick lines, and the three sub-pixel segments within each pixel are displayed with relatively thin lines. This composite grid is superimposed on top of the original high resolution color bitmap image 102 of FIG. 1 from which the pixilation patterns illustrated in the magnified displays 1020 and 1020A are derived. In the particular image shown, the resolution of the color bitmap 102 is about twice as high as the total pixel resolution in the image 105 shown at the bottom of FIG. 10 in both the vertical and horizontal directions.

  The magnified display 1060B determines how a sub-pixel optimized font image is generated by determining a portion of each sub-pixel area that is subject to the application of a high-resolution font outline of one or more included characters. It shows how.

  By sub-pixel optimized representation of images and text at sub-pixel resolution, as can be seen by comparing the sub-pixel resolution magnified representations 1020A and 1060A with the corresponding full-pixel magnified representations 1020 and 1060, respectively. The display provides a more appropriate spatial resolution.

  FIG. 11 depicts the quality of readability provided by an embodiment of the present invention when standard web content is displayed on a 320 × 240 color display. The bitmap 1100 illustrated at the top of the figure is grayscale and an enlarged display of all pixels for an actual bitmap generated from a standard 640 × 480 layout of a portion of the web page “princeline.com”. It is. This image corresponds to a part of the web page included in the rectangular portion 1130 illustrated in the full screen 320 × 240 image of the web page illustrated at the bottom of FIG. The bitmap 1120 shown in the middle of the figure is gray scale, and is an enlarged display of all pixels of the color bitmap of the same portion of the 320 × 240 image. The 200B illustrated at the bottom of FIG. 11 represents a handheld computing device that functions as a thin client browser of the type described with respect to FIG. A representation of a 320 × 240 sub-pixel optimized bitmap representing a 640 × 480 layout of a web page is illustrated on the browser screen. Like the enlarged display 1020 of FIG. 10, the bitmap 1130 illustrated at the bottom of FIG. 11 illustrates individual pixels at grayscale levels corresponding to the average luminance of all pixels. When viewed on a 320 × 240 screen with vertical subpixel stripes in the horizontal direction, as shown in magnified views 1020A and 1060A in FIG. 10, the actual image is rather high resolution. Seems to have.

  Any well-known algorithm that obtains a sub-pixel optimized image of a color bitmap can be used for purposes related to many features of the present invention. In one embodiment of the present invention, the luminance assigned to each of the predetermined subpixels of the predetermined color is fully or partially within a rectangular window in the source image associated with the predetermined subpixel. It is determined by the average intensity of a given color value at each pixel of the existing source image. The source image window has a size and position associated with the source image that corresponds to the area of all pixels in the reduced image that is centered about the position of the predetermined subpixel in the reduced image. . The average intensity assigned to the subpixel is calculated by multiplying the intensity of each source image pixel that covers the source image window, in whole or in part, by the percentage of the area of the window covered by each such source image pixel. .

FIG. 12 illustrates the mapping of the higher resolution source image 102 to the subpixel grid of the reduced resolution display device. This figure illustrates a subpixel grid 1210 overlaid on a portion of the original higher resolution color bitmap 102 illustrated in FIG. Circle 1220 surrounds the area of the grid corresponding to one pixel in the desired lower resolution display device. The position and size of the grid pattern is determined by the relationship between the higher resolution source bitmap image and the pixel grid of the resulting subpixel optimized image. The specific grid pattern 1210 illustrated in FIG. 12 scales from the pixel resolution of the color bitmap image 102 to a display screen resolution having about half as many pixels of the source image in both the horizontal and vertical directions. Is expressed. As an example of this enlargement / reduction, there is an example in which an image having a pixilation suitable for display on a 640 × 480 display is reduced so as to be displayed in proportion on a 320 × 240 display screen. Thus, each thick line segment of the grid pattern 1210 extends to 4 pixels of the color bitmap image 102. Dashed circle 1220 surrounds one such thick line segment that includes the source pixels of the four higher resolution sources. FIG. 13 illustrates nine thick line segments centered on circle 1220 of FIG. It is an enlarged view of (that is, all nine pixels). The pixels in the circle 1300 represent one pixel of the desired display device. As FIG. 13 reveals, each of the thick line sections of the grid pattern 1210 surrounds four higher resolution source images. FIG. 13 also shows three color sub-pixels, each pixel of the desired display device including red, green, and blue sub-pixels labeled “R”, “G”, and “B”, respectively. It shows that it is configured.

  FIGS. 14, 15 and 16 illustrate the position of the rectangular window in the source image where the luminance of the red, green and blue color subpixels is determined, respectively, on the desired display device. Each such source image window region corresponds to the region of all pixels in the reduced image, centered on the portion of the source image corresponding to the subpixel used to calculate the luminance.

  The rectangular portion 1400 of FIG. 14 encloses the area of the source image window that is used to calculate the luminance of the red subpixel of the lower resolution display device. Similarly, FIGS. 15 and 16 enclose source image windows corresponding to the green and blue subpixels of the desired display device, respectively.

  As described above, the luminance assigned to a subpixel of a predetermined color is determined by the following function or an approximation of the function. This luminance is the intensity of the color of the pixel at each pixel of the source image that is in whole or in part in the corresponding source image window of the subpixel, and the percentage of the area of the window covered by each such source image pixel. To match the multiplication of.

  In FIGS. 17, 18, and 19, the brightness of the red, green, and blue sub-pixels is entirely or partially contained within the source image window centered around the portion of the source image that corresponds to the given sub-pixel. This is a function relating to the brightness of each color of a source image pixel. This is illustrated in FIG. 17 for a red (R) subpixel centered on the portion of the source image whose window region 1700 corresponds to that subpixel. The window 1800 in FIG. 18 illustrates the same content for the green (G) subpixel, and the window region 1900 in FIG. 19 illustrates the same content for the blue (B) subpixel.

  As a result of the movement between the source image windows for each subpixel, the color value obtained for each subpixel as a whole corresponds to the position of each subpixel rather than to the position of that pixel. It represents the corresponding color of the sub-pixel at the position of the corresponding source image. As a result, using different source image windows for different subpixels of a given pixel improves the spatial resolution of the resulting image.

  In the embodiment of the invention illustrated in FIGS. 17, 18 and 19, which pixels are included in the sub-pixel source image window and the percentage of the window that such pixels cover is covered by such source image. It is determined based on the ratio of horizontal and vertical scan lines. In FIG. 17, the color value of the red sub-pixel is determined as a result of multiplying the percentage of horizontal and vertical coverage lines 1720 covered by individual source image pixels by the red color value of each such pixel. The This is the case for each color value, ie, the green (G) subpixel of the scaled image of FIG. 18, and the horizontal and vertical coverage lines 1820 and 1840, respectively, and the scaled image of FIG. The same is true for each of the blue (B) subpixels and horizontal and vertical coverage lines 1920 and 1940, respectively. It should be noted that the horizontal coverage lines 1720, 1820, 1920 are intended to represent the vertical position above and below the midpoint in the vertical direction of the corresponding rectangular area. For this reason, the coverage line does not completely coincide with the position in the source image that represents the boundary between vertical pixels. Similarly, the vertical coverage line 1740 is intended to represent the horizontal positions immediately to the left and right of the midpoint in the horizontal direction of the rectangular area 1700.

  The coverage line defined above is a continuous function (e.g., 5 bit resolution or it) that determines the extent to which a region of the original image associated with a given subpixel is covered with a given color or shape. Represents an embodiment relating to one aspect of the present invention relating to the use of any moderately high resolution equivalent to a continuous function, such as those having a bit resolution greater than. In a continuous coverage function, the coverage value is not a sampling, but rather determines the boundary location where a given coverage starts and ends in one or more dimensions, and one or more such boundaries Or a mathematical function that calculates coverage as a function of length or area between such boundaries and the boundaries of the source image window associated with a given subpixel.

  In the embodiments of the present invention illustrated in FIGS. 17, 18, 19 and FIGS. 30, 31, 32, this continuous coverage function calculation is performed for each of one or more source image pixels in the window. Accelerating by evaluating the area of each source image pixel present in the source image window area corresponding to a given subpixel by determining one or more scanline portions in the source image window covered by Is done. The percentage of the length of the entire scan line of the window covered by a given pixel is multiplied by the color value of the subpixel at that pixel. The result of such a calculation is summed over all the pixels covering the scan line of any window to produce the subpixel color values. In this way, a “line coverage” type continuous coverage function can be used to determine the luminance of the sub-pixels when generating an enlarged / reduced image of the color bitmap.

  FIGS. 20, 21, and 22 illustrate the use of a single horizontal coverage line and a single vertical coverage line in the source image window 2000 associated with a red (R) subpixel in a lower resolution display screen. It is shown. In FIG. 21, the coverage value associated with the horizontal scan line 2020 includes the red value of the pixel covered by the bracket 2120 and the horizontal scan line 2020 of FIG. 20 covered by the bracket 2120. Of the pixel covered by the middle bracket 2140 and the half of the horizontal scan line 2020 covered by the bracket 2140) + (The multiplication value of the red value of the pixel covered by the bracket 2160 and the 1/6 portion of the horizontal scan line 2020 covered by the bracket 2160).

  Similarly, the coverage value associated with the vertical scan line 2040 is (the red value of the pixel covered by the bracket 2220 and the half of the vertical scan line 2040 of FIG. (Multiplier value)) + (the red value of the pixel covered by the bracket 2240 and the half value of the vertical scan line 2040 covered by the bracket 2240).

  The total coverage value for the red subpixel is the sum of one half of the coverage value calculated for the horizontal scan line and one half of the coverage value calculated for the vertical scan line.

  Similarly, FIGS. 23, 24, and 25 show a single horizontal coverage line within a source image window 2300 associated with a green (G) subpixel on a lower resolution display screen and a single vertical. 26 illustrates the use of coverage lines, and FIGS. 26, 27, and 28 illustrate a single horizontal coverage within a source image window 2600 associated with a blue (B) subpixel in a lower resolution display screen. The use of lines and a single vertical coverage line is illustrated.

  FIG. 29 is a highly simplified algorithm 2900 for obtaining a scaled, sub-pixel optimized image from a source bitmap image using line coverage of the type described above with respect to FIGS. 17-28. Pseudo-code notation.

  The algorithm performs a loop 2901 for each pixel row of the output image (ie, the scaled / sub-pixel optimized image).

  This loop executes an inner loop 2902 for each pixel in the current row. For each such pixel, loop 2902 performs loop 2904 and function 2914.

  Loop 2904 is performed for each subpixel of the current pixel of loop 2902. The loop 2904 comprises an inner loop 2906 that is executed for each of the current sub-pixel scan lines, such as the scan lines illustrated in FIGS.

  The loop 2906 includes a function 2908 and a loop 2910. Function 2908 calculates each intersection of the scan line and the pixel boundary. Typically, such intersection calculations, and other calculations in this algorithm, are performed with limited precision, such as 6-8 bit precision, to reduce the storage capacity and computational requirements associated with such operations. The

  The loop 2910 then performs a function 2912 on each end of the scan line, the end of the scan line, and the pixel boundary, or each portion of the scan line that occurs at both pixel boundaries. Function 2912 calculates the number of subpixel scanlines by multiplying the percentage of scanlines covered by the current portion of loop 2910 by the color component value of the pixels covering the portion corresponding to the current subpixel color. Divide by and add to the coverage value associated with the current subpixel of loop 2904.

  When loop 2904 calculates a subpixel luminance value for each subpixel of the current pixel, function 2914 calculates a composite RBG value having red, green, and blue component values equal to those calculated subpixel luminance values. Sets the color value of the current pixel as well as the provided color.

  In other embodiments according to the present invention, color values of different lengths can be used, for example, 24-bit, 16-bit, and 12-bit color values. This system can be used with a limited color palette, but works optimally with true colors having at least a 4-bit variation for each of the red, green, and blue sub-pixels. Typically, a 16-bit color that assigns 5 bits to red and blue and 6 bits to green (because the eyes are sensitive to green) provides a better visual effect.

  While the embodiments of the present invention described in FIGS. 17-28 above utilize a single horizontal scan line and a single vertical scan line, other embodiments relating to this feature of the present invention have more. These scan lines and / or scan lines in directions other than the horizontal and vertical directions may be used.

  FIG. 30 illustrates two horizontal coverage lines and two lines in the source image window 3020 that can be used to evaluate the color coverage value associated with the red (R) subpixel of the subpixel optimized image. Illustrates the use of vertical coverage lines.

  FIG. 31 illustrates the use of two diagonal coverage lines in the source image window 3120 associated with the green (G) subpixel of the subpixel optimized image.

  FIG. 32 shows two diagonal coverage lines, one horizontal coverage line, and one vertical coverage line in the source image window 3220 associated with the blue (B) subpixel of the subpixel optimized image. The use of is illustrated.

  Needless to say, the arrangements of the coverage lines illustrated in FIGS. 30 to 31 can be used for red, green, and blue sub-pixels.

  FIGS. 33-38 show that the line coverage method for calculating the luminance value of the subpixel has a wider range of differences between the size of the source image and the size of the resulting subpixel optimized image. It shows that it can be applied to enlargement / reduction. For example, even if compared to many sampling techniques, this is true because the line coverage method measures line coverage at a much higher resolution. This means that the coverage of pixels that are only partially present in the subpixel source pixel window is relatively successful, as often occurs when using non-integer ratio scaling.

  In an embodiment relating to this aspect of the invention, 7-bit resolution is used for the calculation of the line coverage value and produces satisfactory results. Although it is possible to use higher or lower resolutions, the line coverage resolution is typically an array of 16 (4 × 4) to 256 (16 × 16) points, with a source image window of subpixels. It is desirable that the resolution be higher than the resolution of 2 to 4 bits / dimension, which is used in the technique of measuring the coverage by the sampling coverage within.

  FIG. 33 shows the various in the source image window associated with the blue (B) subpixel for mapping from the source image resolution to the target pixel optimized image with half of the horizontal and vertical pixels. Brackets are used to illustrate the coverage of two horizontal coverage lines by source image pixels. FIG. 34 illustrates the same content for two vertical coverage lines used with such a subpixel luminance calculation method. Figures 33 and 34 illustrate the integer ratio between the number of pixels in the source and reduced images.

  FIGS. 35 and 36 show horizontal and vertical scans by pixels of the same source image for a magnification in which the reduced sub-pixel optimized image has only about 40% horizontal and vertical pixels of the source image. Each line coverage is illustrated.

  FIGS. 37 and 38 illustrate the same content for a magnification where the sub-pixel optimized image has horizontal and vertical sub-pixels of approximately 66.66% of the source image.

  It can be seen that the scanline coverage technique illustrated in FIGS. 33-38 provides an accurate estimate of the percentage of each source image window covered by each differently scaled source image with relatively little computing power. .

  39 and 40 illustrate diagrams relating to “region” type continuous coverage functions. In some embodiments of the present invention, the percentage of the source image window of a given subpixel covered with each associated source image pixel is not the line coverage approximation described above, but rather the source of the subpixel. It is calculated by the actual calculation of the area of that portion of each such source image located within the image window. For each such source pixel, the component color value of the pixel corresponding to the color of the current subpixel is determined. Next, the luminance value of each sub-pixel is calculated by summing the multiple of the coverage value ratio and the sub-pixel color value for each source image pixel appearing in that source image window.

  FIG. 39 illustrates a source image window area 3900 associated with a blue (B) subpixel. Source pixel 3920 is included within source image window 3900, as is a portion of eight other source pixels. The percentage of the source image window 3900 covered by the source pixels 3920 is calculated by obtaining the ratio of the shaded area 4020 area of FIG. 40 to the total source image window 4000 area. Similarly, the percentage of the source image window 4000 covered by other source pixels contained therein is within the source image window for the entire area of the source image window, as indicated by the different shaded areas of the window 4000. Calculated by obtaining the percentage of those regions present.

  FIG. 41 provides a highly simplified pseudocode notation for an algorithm 4100 that can be used to perform an area coverage function of the type discussed with respect to FIGS.

  The algorithm includes a loop 4102 that is executed for each pixel row in the generated sub-pixel optimized image. For each such row, loop 4102 performs an inner loop 4104 for each pixel in that row.

  This inner loop 4104 includes a loop 4106 and a function 4116. Loop 4106 is performed for each subpixel in the current pixel of loop 4104. The inner loop 4106 includes a function 4108 and a loop 4110. Function 4108 determines which pixels of the source image are in the source image window associated with the subpixel, as described above. When this is done, loop 4110 is executed for each source image pixel.

  The loop 4110 includes a function 4112 and a function 4114. Function 4112 calculates the percentage of the corresponding source image window area of the subpixel covered by the current source image pixel of loop 4110. Step 4114 sets the luminance value calculated for the current subpixel of loop 4106 to a multiple of the percentage of the source image window area of the subpixel covered by the current source image pixel and the color of the current subpixel. Is multiplied by the color component value of the source image pixel corresponding to.

When loop 4106 is executed for each subpixel in the current pixel,
Function 4116 sets the color value of the current pixel in the same manner as the color having RGB color component values that match the red, green, and blue subpixel luminance values calculated in loop 4106.

  42-53 relate to features of the present invention relating to a two-color subpixel optimized image.

  A “two-color” image is an image in which individual pixel colors change between two different color values. In general, these two different color values are black and white, and the source pixel and sub-pixel image pixels have values limited to black and white, and in some cases between black and white The value is limited to the gray scale. However, in some embodiments, two different color values can represent any uniform foreground and background colors and their intermediate colors. In many cases, the display of text is colored with colors including foreground and background colors, so in many cases a two-color image is used to represent the text. However, to represent other two-color shapes, two-color bitmaps, portions of colored multi-color bitmaps, or multi-color bitmaps represented by two-color outputs, such as grayscale images, It is also possible to use a two-color image. For example, it is possible to process a multicolor source image as a grayscale image by simply processing each of the pixels having a grayscale value corresponding to the average luminance of each of the three color components.

  The advantage of using such a two-color subpixel optimized output image can often provide a higher spatial resolution than the multicolor subpixel optimized output image. It is in that point. Such higher resolution is acceptable when the two colors are black and white, grayscale values, or opaque and transparent, which means that each color of each such two-color pair is red, green, blue. This is because each subpixel can represent both the foreground and background of such a two-color pair in the same way as the other because it has equivalent components. Except for the need to perform color balance, as shown below, the brightness of each subpixel is a function of the extent to which the portion of the source image that matches its own area in the output image is covered with the foreground or background color. As determined. By using a smaller source window (ie, a source window that matches the size of the subpixel rather than the size of the pixel) in this way, a more accurate spatial resolution of the source image can be achieved.

  If the foreground and background colors are not black and white, the foreground and background colors have red, green, and blue values of relatively equal brightness, and the average brightness of the different foreground and background colors as much as possible , The resolution generated by the two-color subpixel optimized image is the best. In fact, in some embodiments relating to features of the invention relating to a two-color subpixel optimized image, the output of 2 is achieved by transitioning to the corresponding grayscale color to improve the spatial resolution of the output image. One or both of the colors are changed from the two colors of the corresponding input.

  The degree to which the subpixels of the two-color subpixel optimized output image display the foreground color may be represented by an alpha value or an opacity value. Such an alpha value indicates the degree to which the luminance of the sub-pixel should correspond to the color component of the sub-pixel in the foreground or background color. An alpha value of “1” means that the color component value of the subpixel is equal to the corresponding color component in the foreground color. An alpha value of “0” means that the color component value of the subpixel is equal to the corresponding color component in the background color. An intermediate alpha value means that the color component value of the subpixel is a weighted blend value of the corresponding color component in both the foreground and background colors. When the sub-pixel optimized bitmap is expressed in terms of alpha values, the alpha values can be used to represent a predetermined pattern of two-color images that use different foreground and background colors. In font display, bitmap patterns of a given character font shape of a given size are often displayed with different foreground and background colors, so this is generally used to represent the font shape. used.

  In some embodiments relating to features of the present invention relating to two-color sub-pixel optimization of bitmap images, scaled sub-pixel optimized images of bitmap images are: As a function of c), it is generated by associating foreground or background two-color coverage values with each subpixel of the scaled image. (A) the ratio of the foreground or background color for each source image pixel in the source image window corresponding to the region of the subpixel; (b) the percentage of that window covered by each such source image pixel; A color balance function that distributes the sub-pixel coverage values to reduce color imbalance. If a two-color output image is generated for a grayscale input image or a multicolor input image, it is calculated for each sub-pixel as a function of the total pixel brightness of the source image pixels covering the source image window. Coverage value can be obtained. In some embodiments, the degree to which a given luminance value associated with a given subpixel's source image window is distributed to other subpixels is a function of the degree to which the luminance value causes color imbalance. It is.

  FIGS. 42 to 44 relate to a method for determining the luminance of each sub-pixel of a grayscale two-color image of a bitmap image. In FIG. 42, a rectangular portion 4200 encloses a window of the source image that is associated with the red (R) subpixel of the enlarged / reduced image. The luminance associated with such a red (R) sub-pixel is multiplied by the percentage of the source image window covered by each such source image pixel, respectively, to the total of one or more source image pixels covering the source image window 4200. This is a function related to pixel luminance. Any known method for calculating or estimating the percentage of such coverage can be used.

  In the embodiment illustrated in FIG. 42, the source image window 4200 associates two horizontal scan lines 4210 and 4220 and two vertical scan lines with it. 43 and 44 illustrate coverage lines corresponding to source image windows 4300 and 4400 for green and blue subpixels, respectively. As previously described, a mathematical function is performed that determines the boundary locations where a given coverage starts and stops along each scan line in order to estimate the extent to which the source image window area is covered by the source pixels. Coverage is calculated as a function of the length between one or more such borders, or the length between such borders and the border of the source image window associated with a given subpixel. This can be performed in a manner similar to that described above in FIG.

  Color imbalance can occur when computing a two-color subpixel optimized image. This means that if the two-color method consists of foreground and background colors, and if the source image has an intermediate color, it consists only of colors in the spectrum between the foreground and background colors, This is because a perceived output image is to be generated. However, the coverage value of the individual red, green, and blue subpixels of a pixel is determined by the percentage of foreground color in each such subpixel, which often means that the color of the individual output pixel is In the absence of such color balance, it means that the desired two-color spectrum (usually gray scale) is not relevant.

  For example, in a grayscale image, the source image changes from entirely white to totally black at a location corresponding to the boundary between the red and green subpixels in the subpixel optimized output image. The corresponding pixel in the output image has a red sub-pixel coverage value that tends to switch the sub-pixel to an overall on state and a green color that tends to switch such a sub-pixel to an overall off state. And a blue sub-pixel coverage value. In this example, the output image is supposed to be a grayscale image, which results in a visible red color for the pixel.

  45-47 are used in the prior art to color balance subpixel coverage values calculated based on font outline rasterization to color balance coverage values generated from a two-color bitmap. It illustrates how certain types of conventional linear color balance methods can be used.

  FIG. 45 illustrates a set of grayscale source image pixels under the RGB grid 4600. The grid 4600 has four pixel regions surrounded by thick line segments. Each such pixel region is associated with every pixel in the subpixel addressable screen on which the output image is displayed. Each pixel region is further divided into three regions associated with subpixels of the associated pixel on the subpixel addressable screen. The region 4610 associated with the subpixel is associated with the red (R) subpixel, and the region 4612 associated with the subpixel is associated with the green (G) subpixel and associated with the subpixel. Region 4614 is associated with the blue (B) subpixel. Similarly, regions 4616-4632 associated with subpixels are associated with respective display screen subpixels.

  Regions 4614-4630 associated with sub-pixels are wholly or partially covered by source image pixels having non-white luminance values, which in this example correspond to varying degrees of black foreground color. The sum of the black luminance values of the source image pixels in each of the regions 4614 to 4630 associated with the subpixels (corresponding to a value obtained by subtracting white or a normal luminance value) is shown in the RGB grid 4700 of FIG. To the corresponding subpixel region. The height of the shaded area in each subpixel area 4744-4760 is determined by the sum of the non-white luminance values of the corresponding subpixel areas 4614-4630.

  The bottom portion of FIG. 46 can be used to distribute coverage values associated with a given subpixel, eg, subpixel 4750, to five subpixels centered on that given subpixel 4750. The use of a center-weighted symmetrical color filter is illustrated. Three-ninths of the coverage value of subpixel 4750 is distributed to subpixel 4750 itself. Two-ninths of the coverage value of subpixel 4750 is distributed to subpixels 4748 and 4752 located immediately to the left and right of subpixel 4750, respectively. To complete the color distribution of subpixel 4750, one-nineth of the coverage value is subpixel 4746, which is the second subpixel on the left side of subpixel 4750, and the second subpixel on the right side, respectively. Distributed to 4754.

  In general, color balance assigns color values that are in the vicinity of neighboring subpixels. In many embodiments, such as the embodiment shown in FIG. 46, the neighboring subpixel where such an assignment is performed is only one pixel away from the subpixel to which the luminance is assigned. However, in this example, the neighboring subpixel is defined as being within a distance corresponding to two pixels from the subpixel to which the luminance is assigned.

  FIG. 47 shows a case where the center-weighted symmetric color balance filter of FIG. 46 is linearly applied to the coverage values calculated for each of the subpixels 4744 to 4760 shown in the upper half of FIG. The results are illustrated.

  In FIG. 47, the coverage value associated with each subpixel 4744-4760 illustrated at the top of FIG. 47 is for its own subpixel and, as illustrated in FIG. The two subpixels on the left and right sides are distributed using a color balance filter in which coverage values are distributed at the same ratio. The central grid 4802 of FIG. 47 illustrates the magnitude of the contribution that such distribution is made to each of the subpixels 4740 to 4762. The distribution associated with each of the predetermined subpixels 4744 through 4760 is centered on a vertical column located directly below each subpixel.

  The RGB sub-pixel grid pattern 4804 illustrated at the bottom of FIG. 47 sums all contributions made by the distribution of all coverage values illustrated in the center panel 4802 to sub-pixels 4740 through 4740. The luminance values calculated for each of 4762 are illustrated. To complete this method, the luminance values of the red, green, and blue subpixels of each pixel in grid 4804 are used as three color component values that identify the color of each such subpixel.

  The color imbalance of images scaled with this linear method is reduced, but this is performed at the cost of a substantial reduction in spatial resolution. This relates to the values in the bottom RGB grid pattern 4804 of FIG. 47 representing the subpixel luminance values in the subpixel optimized output image and to the source image pixels in the region corresponding to the subpixels of the output image. It can be understood by comparing the values in the RGB grid pattern 4800 located at the top of FIG. 47 representing the luminance or foreground coverage. As can be seen from FIG. 47, the spatial resolution of the output image is unclear compared to the spatial resolution of the source image.

  The present invention provides similar color balance for sub-pixel optimized output images, but often includes innovations that provide color balance with less blur for the output image. This is done by using a non-linear color balance filtering method. A method for applying the non-linear filtering is illustrated in FIGS.

  The RGB grid pattern 4900 in FIG. 48 is a copy of the RGB grid pattern 4700 in FIG. Also in this case, the total luminance or the foreground color coverage, that is, the value of the source image pixel corresponding to the related sub-pixel is expressed by the height of the hatched area.

  The first stage of the non-linear color balance method is illustrated in FIG. As described above, the RGB grid pattern 4900 is divided into four by thick line sections, and each of the four divided parts, that is, parentheses 4902, 4904, 4906, 4908 is enlarged / reduced or output. Associated with all pixels of the rendered image. Each of the pixel regions 4902, 4904, 4906, 4908 is further divided into sub-pixel source image windows corresponding to the sub-pixels in the generated output image. For each pixel, it is determined which of the corresponding source image windows of the subpixel has the lowest subpixel brightness or foreground coverage value. A luminance (or alpha) value equal to the minimum luminance / coverage value is the luminance calculated for each subpixel of the output image pixels 4912, 4914, 4916, 4918 of the RGB grid pattern 4910 illustrated in the lower half of FIG. / Added to the alpha value.

  In the upper half of FIG. 48, the dashed line 4920 indicates that the minimum luminance value / alpha value of the pixel area 4902 is zero, indicating that the first two subpixel source image windows for that pixel are This is because the luminance value is “zero”. Thus, the step illustrated in FIG. 48 sets the luminance / alpha value to zero for the red, green, and blue sub-pixel regions of pixel 4912. Similarly, the minimum luminance value of pixel area 4904 is determined by the value of red subpixel source image window 4922 for pixel 4904. This minimum luminance value is mapped to the corresponding pixel area 4914 shown in the lower half of the figure. Similarly, the minimum luminance values of pixel regions 4906 and 4908 are mapped to pixel regions 4916 and 4918 illustrated in the lower half of FIG. The partially calculated luminance / alpha value that occurs after completion of this step is represented by the bottommost RGB grid pattern 4910 in FIG.

  The second stage of the non-linear color balance method is illustrated in FIG. In this example for the second stage, a portion of the luminance value / alpha value of each sub-pixel that exceeds the minimum luminance value / alpha value of the associated pixel utilizes a color balance distribution filter of the type described above with respect to FIGS. By doing so, it is mapped to the RGB grid pattern 4910.

  The top pixel grid 5000 of FIG. 49 shows the luminance value / alpha value of each subpixel remaining after the minimum subpixel luminance value / alpha value (indicated by the low density slash) for the corresponding pixel is removed. 48 corresponds to the topmost pixel grid 4900 of FIG. 48 (and has the same subpixels 4740 to 4762), except that the portion of high density (shown with diagonal lines) is represented.

  In FIG. 49, only the sub-pixel luminance value / coverage value excess portion illustrated at the top of FIG. 49 having a high density diagonal line is obtained using a color balance filter of the type illustrated in FIG. 49 except that it is distributed, the sub-pixel grid 5002 located in the center of FIG. 49 corresponds to the similarly shaped pixel grid 4802 located in the center of FIG.

  As can be seen from this part of the figure, the excess luminance value / alpha value for each sub-pixel is calculated by using the same ratio filter shown in FIG. And to the right two sub-pixels. The portion given the number 5004 near the bottom of FIG. 49 illustrates the sum of such excess luminance / alpha values distributed to each of the sub-pixels 4740 to 4762 in this example for the non-linear method. ing.

  As shown at the bottom of FIG. 49, the luminance value / calculated for each sub-pixel to produce a sum of luminance / alpha values used for each sub-pixel in the output image. The sum of the alpha value excess is added to the minimum luminance value / alpha value previously added to that sub-pixel by the step illustrated in FIG. To complete the non-linear color balance process, the summed luminance / alpha value for each subpixel of the RGB grid pattern 4910 determines the red, green, and blue color component values of the pixel with which it is associated. Used for. The red, green, and blue color values of each individual pixel in such a display may not be equal, but the red, green, and blue colors in any neighborhood of subpixels that are adjacent by about five pixel rows The sum of values should be substantially equal or balanced.

  A comparison of the results achieved through the use of linear and non-linear color balance filtering methods is illustrated in FIGS.

  FIG. 50 illustrates the original filtered source pixel luminance / coverage values as originally mapped to the RGB grid pattern 4700 of FIG. 46 and the RGB grid pattern 4900 of FIG.

  FIG. 51 shows the result of the non-linear filtering method as illustrated at the bottom of FIG.

  FIG. 52 shows the result of the linear filtering method as illustrated in the RGB grid pattern 4804 of FIG.

  As FIG. 51 shows, the output of the non-linear color balance method is much more similar to the original spatial distribution for the luminance / alpha values of FIG. 50 than the result of the linear method shown in FIG. The luminance values generated by the nonlinear method of FIG. 51 are significantly less blurred, thereby providing a higher and more visible spatial resolution than the output generated by the linear method. This is the part of the sub-pixel luminance / coverage values where the ratio linear method is trying to perform a color balance distribution that has the detrimental consequence of blurring spatial resolution and requires such a distribution to prevent color imbalance. It is because it is trying to execute only for. This is the sum of the subpixel luminance / coverage values distributed to each subpixel using a non-linear method, as indicated by numeral 5004 in FIG. 49, and for each subpixel at the bottom of FIG. This can be understood by comparing the corresponding sum distributed to each sub-pixel using a linear method, as shown for the entire cross-hatched region shown.

  FIG. 53 illustrates one embodiment of a method for generating a sub-pixel optimized two-color output bitmap from a bitmap source image using a non-linear color balance method in a highly simplified pseudo-code description. Is.

  The algorithm 5300 in the figure includes a loop 5301 executed for each pixel row of the image. This loop executes two sub-loops 5302 and 5322 for each pixel row.

  Loop 5302 is executed for each pixel in the current row of loop 5391. For each such pixel, loop 5302 performs loop 5304, function 5314, and loop 5316.

  Loop 5304 is executed for each subpixel in the current pixel of loop 5302. For each such subpixel, loop 5304 performs function 5306 and loop 5308.

  Function 5306 determines whether any pixel of the source image is located within the window of the source image corresponding to the sub-pixel region in the scaled image. This can be performed by any well-known coverage calculation and estimation function, including those described above with respect to FIGS.

  A loop 5308 comprised of functions 5310 and 5312 is performed for each source image pixel that is wholly or partially contained within the source image window of the current subpixel. Function 5310 calculates the percentage of the area of the source image window that is covered by the area of the source image pixel. Function 5312 adds a multiple of the percentage of the window area covered by the source image pixel to the average foreground intensity of the source image pixel to the luminance / coverage value calculated for the current subpixel.

  If the two-color image is a grayscale image, the foreground intensity (in many cases referred to above as luminance, or luminance / coverage value) is either white or black. Depending on whether it is, it can correspond either to what is commonly referred to as luminance for each of the entire source image pixels, or to the inverse of what is commonly referred to as luminance. If the source image is a multicolor image, the color component of each source image pixel is used to determine the usable luminance value for the source image pixel when calculating the foreground color intensity for the purpose of function 5312. Average luminance values can be used.

  A loop 5304 can be used to determine the brightness / coverage value of the type illustrated in FIG.

  When loop 5304 is performed for each subpixel at the current pixel, function 5314 is the smallest subpixel calculated for the current pixel, as illustrated in the upper half of FIG. The luminance value / coverage value is obtained.

  When this is done, a loop 5316 consisting of functions 5318 and 5320 is executed for each sub-pixel in the current pixel.

  As shown in the lower half of FIG. 48, function 5318 calculates the luminance / alpha value being calculated for the sub-pixel as the minimum sub-pixel luminance value determined for that pixel by function 5314. / Set to coverage value.

  As shown in FIG. 49, the function 5320 calculates a portion of the subpixel luminance coverage value that exceeds the minimum subpixel luminance value / coverage value of the pixel using the color balance distribution filter. Distribute the luminance / alpha values calculated for the subpixels in the row and the neighboring subpixels.

  In one embodiment of the present invention, if the sum of such distributions performed for a given subpixel exceeds the maximum allowed luminance value / output value, then the subpixel luminance value / alpha value is Limited to its maximum value. Although such clipping can cause some color imbalance, the inventor has found that the resulting imbalance is hardly noticeable.

Luminance / alpha values are calculated for each sub-pixel in the row, and when loop 5302 is complete, loop 5322 performs function 5324 for each pixel in the row. This function converts the pixel color value to a composite RGB value with red, green, and blue component values corresponding to the luminance / alpha values calculated for each of the red, green, and blue subpixels of the pixel. FIG. 54 shows the range in which the subpixel optimized image generated from the source image is generated by the multicolor subpixel optimization process or the two-color subpixel optimization process. 1 illustrates one feature of the present invention that allows a display user to dynamically perform a tradeoff. The two-color subpixel optimization in this example can produce only a grayscale output image, while the output image produced by the multicolor subpixel optimization produces a reduced color image with reasonable color accuracy. It can be expressed. However, in some cases, such grayscale output images have a more accurate spatial resolution than output images generated by multicolor subpixel optimization, especially when the source image has sharp edges. When it has a white part and a black part provided, it has a color imbalance that is difficult to perceive.

  Functions 5410 and 5430 that use a two-color subpixel optimization method, such as the example described above with respect to FIGS. 42-53, to generate a scaled and subpixel optimized grayscale bitmap 5440. By utilizing this, the color bitmap image 5400 can be enlarged / reduced and sub-pixel optimized. A color bitmap by a process 5420 that uses a multicolor subpixel optimization method, such as the example described above with respect to FIGS. 17-41, to generate a scaled and subpixel optimized color bitmap 5450. It is also possible to scale 5400 and sub-pixel optimize.

  According to the embodiment of the present invention illustrated in FIG. 54, a user of a display device can use a function 5460 to achieve a blend of a color bitmap 5450 and a grayscale bitmap 5440, for example a pointing device, It is possible to operate the control device of the display device such as a keyboard or other input device. For example, this can be executed by operating a slide bar. Function 5480 accepts grayscale bitmap 5440 and color bitmap 5450 and color / grayscale tradeoff information selected by the user, and sets each color value as a function of user selected color / grayscale tradeoff 5460. Weighting mixes the corresponding pixel color values from the grayscale bitmap 5440 and the color bitmap 5450.

  In some embodiments of the type illustrated in FIG. 54, if the user selects a trade-off value at either limit of the color / grayscale spectrum, this process causes the bit corresponding to the selected limit. Only by calculating the map 5440 or 5450, the arithmetic processing can be reduced.

  The advantage with this feature of the invention is that color balance and / or positional accuracy is paramount, or if it simply changes the trade-off choice to obtain a more easily readable display In addition, if it is most important or accurate in color, the user of the display device can support it.

  Not all features of the present invention require sub-pixel optimized text, and many of them do use prior art methods to generate font shapes for sub-pixel optimized fonts. I can do it. However, some features of the invention relate to an improved method for generating a sub-pixel optimized font bitmap.

  FIGS. 55-97 relate to features of the present invention related to the formation and use of sub-pixel optimized font bitmaps.

  FIG. 55 illustrates a font outline 5500, which in this example illustrates the outline of a capital letter “B” in a Times Roman font. This outline is displayed on a sub-pixel grid 5502 formed by using a plurality of individual pixels 5504 each including red, blue, and green sub-pixels 5506, 5508, and 5510, respectively.

  The font outline illustrated in FIG. 55 is an outline that can be used for display at a relatively normal text size, and the inventor's method of sub-pixel optimized character font shape can be applied to a wide range of applications. FIG. 11 shows that the present invention is not limited to the small screen display of the type shown at the bottom of FIG. However, if this feature of the present invention is applied to small screen displays and / or display of fonts with very small pixel sizes, for example, in some embodiments at a size of 10 pixels / em or less, And in other embodiments, it is desirable to optimize the font used to be displayed in a small size with a size of 8 pixels / em or less.

  FIG. 56 illustrates a font optimized for such a small display on a subpixel addressable screen. FIG. 57 shows the same bitmap in double size. Unfortunately, the bitmap prints illustrated in FIGS. 56 and 57 display the average brightness of all pixels, and the higher that is possible when such a bitmap is displayed on a subpixel addressable display. The resolution cannot be obtained.

  The fonts illustrated in FIGS. 56 and 57 are generated by a hinting process that changes the selected individual font outline boundaries to the middle of the horizontal and vertical dimensions between pixel boundaries, subpixel boundaries, and subpixel boundaries. ing. Such high resolution hinting is used to achieve optimal readability on subpixel displays. When a character is displayed in such a small font size, the font designer uses various hinting values to optimize each pixel's sub-pixel until the user is relatively satisfied that the character is as clear as possible. This is done by letting you look at the quantization bitmap. As those familiar with font hinting understand, fonts have a hint that directs the placement of individual parts of the font outline beyond the rendering of all sizes of the font, and the character font shape at a particular pixel size. You can have special tips that apply to The fonts illustrated in FIGS. 56 and 57 have been hinted to optimize their display at 8 pixels / em, and some of them have specific hints that apply only at such small sizes. Have.

  In fact, most fonts in the 320 × 240 pixel resolution screen shot illustrated in the figure of this application are 8 pixel / em fonts that are specially hinted to display at that size. While relatively high levels of readability are possible, these fonts allow a relatively large amount of web text to fit on a small screen. These fonts allow the majority of the lowercase letters of the Latin alphabet to be displayed in 4 pixel columns or less, including any space separating adjacent characters. With these fonts, the majority of capital letters can be displayed in 5 pixel columns or less.

  The readability of such small fonts is greatly improved by using either sub-pixel optimization or anti-aliasing, which is related to the extent to which the character shape covers a given pixel by these techniques. This is because the information can be expressed by a method more than binary display at all pixel levels. In fact, sub-pixel optimization is a type of anti-aliasing, which, like traditional anti-aliasing, means that pixels that are partially covered by the font shape due to sub-pixel optimization are proportional to the function of such coverage. This is because it has color values to

  58 and 59 illustrate that the sub-pixel optimized bitmap generated by the present invention can be represented as a font outline and / or a font bitmap. The font outline description 5802 preferably relates to the shape of one or more characters of a given font, along with hinting information designed to optimally position the boundaries of the character outline with one or more different font sizes. Includes a mathematical geometric description. These font outlines are designed to be optimally rendered on a subpixel addressable display, eg, as discussed herein, and / or optimized for display on a subpixel addressable display. It may have a hinting.

  A font renderer 5806 can be used to generate a subpixel optimized bitmap 5804 from such an outline, as described below.

  In some embodiments of the invention, as illustrated in FIG. 58, a computer 5808 and / or an application running on the computer is accessed from a font server 5812 via a computer network 5814. Display text using a font bitmap or font outline. In other embodiments, as illustrated in FIG. 59, a computer 5900 and / or an application 5902 running on that computer is required to render the text stored therein. A font bitmap 5804. Such computers and / or applications can only hold font bitmaps, or they can store scaleable font outlines 5802 and render font bitmaps 5804 at different sizes from time to time. I can do it.

  The advantage of saving only the font bitmap is that the need to save the font outline and font renderer on the computer 5900 is eliminated. This also eliminates the need for operations related to font rendering. In addition, many font vendors further want font bitmaps to be relatively freely available on the Internet, and then font outlines.

  The advantage of saving font outlines is that if the user is interested in rendering fonts at various sizes, the font renderer is more efficient than saving font bitmaps for all different size characters. Saving the necessary code and saving the font outline description that can be enlarged / reduced are actually more efficient.

  As illustrated in FIG. 58, the advantage of receiving fonts from a font server is that the client computer 5808 does not need to store a large font library, for example, as depicted in FIG. By downloading such fonts from time to time, it is possible to represent text in any of a number of different fonts and sizes. The client computer 5808 preferably caches a reasonable number of character font bitmaps so that it does not need to communicate over the network 5814 each time it tries to display a character string.

  FIG. 60 is a highly simplified pseudocode description of an algorithm 600 used in some embodiments relating to features of the present invention relating to generation of a subpixel optimized font bitmap. This algorithm uses a non-linear color balance of the type described above with respect to FIGS. Such a sub-pixel optimization algorithm is particularly optimal when used to display text characters, because the placement of text outline boundaries with full pixel boundaries has been rasterized for the use of hinting. This is because it is very common in font shapes.

  The algorithm 6000 of FIG. 60 includes a loop 6002 that is executed for each pixel row in rasterizing individual character font shapes at a predetermined pixel resolution. The loop 6002 includes three sub-loops 6004, 6008, and 6020 that are sequentially executed for each pixel row.

  Loop 6004 is executed for each subpixel in the pixel row for which the current iteration of loop 6002 is being performed. For each such subpixel, the loop 6004 performs a function 6006 that determines the coverage value for each such subpixel as a function of the percentage of the subpixel area covered by the character font shape from which the image is being generated. .

  61-90 are used to discuss methods that can be used to determine the coverage value for each sub-pixel in step 6006 of FIG.

  As shown in FIGS. 61, 62, 63, the area in the character font shape image for which such coverage value is calculated for a given pixel 5504 is the red, green, blue subpixel 5506, 5508 and 5510 correspond to image areas respectively displayed. This is different from the case of a multi-color image that is typically sub-pixel optimized, as shown in FIGS. 14-16 above, where the source image window corresponding to each sub-pixel is typically larger. The source image window used in the method of FIG. 60 has the same size as the area of the source image window used for the two-color bitmap described above with respect to FIGS.

  Such higher resolution source image windows can be used because the shape of the character font described by most font outline descriptions is related to the foreground color (mostly represented by an alpha value of “1”). A two-color image with a region covered by a font outline that is considered to be and a background color (mostly represented by an alpha value of “0”) and all other parts of the image Because.

  The calculation of the coverage value in function 6006 of FIG. 60 is an arbitrary that can rasterize a character font outline in relation to a pixel array having the same spatial resolution as the sub-pixel resolution of grid 5502 illustrated in FIG. Can be implemented using the prior art methods.

  FIGS. 64 through 67 illustrate some of the conventional methods used to calculate the percentage of units in the rasterized grid covered by the font shape 6402. In this prior art, the rasterization unit 6400 typically corresponds to all pixels in the output image. In the method of FIG. 60, it corresponds to a subpixel of the output image.

  FIG. 64 illustrates one method for determining the coverage value of a rasterization unit 6400 that uses a mathematical technique to accurately calculate the area of the unit covered by the font shape 6402. This has been relatively rarely used because it is relatively computationally expensive.

  A substantially more efficient use of the computer is described in FIG. 65, which uses a piecewise linear approximation 6504 for the boundary of the font shape to render the rasterization unit covered by the font shape 6402. A ratio of 6400 is calculated.

  FIG. 66 illustrates how a computer can be used more effectively, while producing substantially lower institutional results. In this way, the percentage of coverage values of the rasterization unit 6400 is determined by determining how much the set of sample points 6600 fits within the font shape 6402.

  FIG. 67 illustrates a method for determining a coverage value that provides a more accurate result for a relatively low degree of computation, the same as the method of FIG. It determines the coverage of the rasterization unit as a function of the average percentage of the number of scanlines 6700 and 6702 covered by the font shape 6402.

  FIGS. 68-87 illustrate a computer-efficient method of calculating the rasterization unit coverage value, which is illustrated in FIG. 66 for the same amount of computation, for example. In general, it produces better results than sampling methods.

  One embodiment of this method is described in more detail in a US patent application filed in the name of one of the inventors of the present invention, Sampo J. Carsila. This US patent application is Ser. No. 09 / 363,513. The United States patent application was filed on July 29, 1999 and is entitled “Systems For Rapidly Performing Scan Conversion With Anti-Aliasing Up Up Fonts And Other Graphic Elements”. This application was issued on August 20, 2002 as US Pat. No. 6,437,793. This application also has the disclosure of the invention published in the PCT application “PCT / US00 / 21559”. This application and the patents obtained from that application are incorporated herein by reference in their entirety.

  In the method of FIGS. 68-87, the coverage value for the rasterization unit is determined by the ratio of two scan lines covered by the font outline shape 6402, ie one of the horizontal scan lines 6804 or the vertical scan lines 6802. Is done. The scan line whose coverage value is used as the coverage value for the rasterization unit is a scan line with a more intermediate coverage value. For example, in embodiments where horizontal and vertical scan line coverage is calculated in the range of values from 0 to 126, the selected scan line is the scan line whose value is closest to 63, indicating 50% coverage. is there.

  In FIGS. 68-71, it is the percentage of coverage of the vertical scan line 6802 that is used to represent the percentage of coverage of the rasterization unit 6400. In FIGS. 72-75, it is the horizontal scanline 6804 that has the closest value in the middle, so it can be used to represent the percentage of actual coverage for all rasterized units or at valley time. Have a proportion.

  In all of the remainder of FIGS. 76-87, the coverage value of a scanline with a coverage value closer to the middle is usually very close to the actual coverage value for the entire rasterization unit, which is usually the entire rasterization unit. It can be seen from the actual coverage value for that that it never changes more than 25%.

  FIGS. 88-90 show the contribution of coverage values for individual scanlines to estimated coverage values for their associated rasterization units as a function of their line coverage values having more intermediate coverage values. Fig. 5 shows another combination of scan lines that can be used according to the weighting method. In such a way, the coverage values calculated for all rasterization units are multiplied by the sum of the coverage values for each scanline and the middle of that coverage line and divided by the middle sum of each scanline. Can be set equal to the thing. In this calculation, the middle of the scanline is calculated by subtracting the absolute value of the difference between the coverage value of the middle percentage of the scanline and the coverage value of the middle percentage of the scanline and the actual percentage of the scanline. equal.

  FIG. 91 illustrates a virtual font outline 9102 that is mapped to each of the red, green, and blue subpixels 5506, 5508, and 5510 in row 9100 of pixel 5504.

  FIG. 92 illustrates a corresponding coverage value 9202 that has been calculated for each of the subpixels in row 9100.

  FIG. 93 illustrates how coverage values determined for individual subpixels can be distributed using a linear color balance method. The linear color balance is the same as described above with respect to FIG. 46, except that it is adapted for subpixel optimized display of font outlines rather than subpixel optimized display of bitmap source images.

  Referring now briefly to FIG. 60, when step 6006 of FIG. 60 calculates or estimates the coverage value for each subpixel in a row, loop 6008 is in that row, as shown in FIG. Performed for each pixel. This loop color balances the coverage values calculated for a row of subpixels. It does not use a linear color balance routine of the kind illustrated in FIG. 93 and described above with respect to FIGS. Instead, it achieves a more highly perceptible spatial resolution by using a non-linear color balance technique similar to that described above with respect to FIGS.

  Loop 6008 performs two functions 6010 and 6012 and loop 6014 for each such pixel.

  Function 6010 determines that the subpixel of the current pixel has the minimum coverage value calculated for that subpixel. Step 6012 then adds this minimum coverage value to the temporary alpha or opacity value being calculated for each subpixel of the current pixel. This is similar to the function illustrated in FIG.

  Next, loop 6014 performs functions 6016 and 6018 for each subpixel of the current pixel. Function 6016 determines, for the current subpixel of loop 6014, an excess of the calculated coverage value that exceeds the minimum coverage value determined for the pixel of which the subpixel is a part. Next, function 6018 computes this excess value for the current subpixel in the current pixel row, the two subpixels to the left, and the subpixel being calculated for the two subpixels to the right. Distribute to alpha value. This function corresponds to the function described above with respect to FIG.

  94 and 95 illustrate two different color balance distribution filters used in one embodiment of the present invention. In this embodiment, the asymmetric center-weighted color balance filter illustrated in FIG. 94 is used to distribute the coverage values associated with the red and green subpixels. On the other hand, the asymmetric color balance filter illustrated in FIG. 95 is used to distribute the coverage value associated with the blue sub-pixel. Therefore, this embodiment of the present invention differs from the process described above with respect to FIG. 49 in that it uses a distribution filter that differs in shape from one for other colors. .

  One of the inventors of this application is that the human eye is more likely to perceive green than blue, so color balance coverage values associated with different color sub-pixels should use such different distribution filters. I found In different embodiments of the invention relating to non-linear color balance (including non-linear color balance for two-color images), the same color that can use different color balance filters for each different sub-pixel color A balance filter can be used for all such colors, and either a symmetric color balance filter or an asymmetric color balance filter can be used.

  The particular color balance filter illustrated in FIGS. 94 and 95 is designed for use with coverage values calculated on a scale of 0-126. A given calculated coverage value having a value between 0 and 126 has five distributions on the right side of the table of FIGS. 94 and 95, with the associated coverage value on the left side of the table being closest to its own coverage value. Associated with a set of distribution values consisting of values. For example, if the coverage value calculated for a given red or blue subpixel is 126, the alpha value calculated for the two left and two right subpixels of the given subpixel "1" is added, and "3" is added to the alpha value calculated for the left sub-pixel and the right sub-pixel of the given sub-pixel and calculated for the given sub-pixel itself. “4” is added to the current alpha value. In this particular embodiment, the alpha value is calculated on a scale of 0-12.

  The relative size of the color balance distribution illustrated in the last row of FIGS. 94 and 95 more accurately reflects the desired distribution ratio. This is because the larger value distributed in each of these last rows allows a larger numerical resolution than is seen in the rows located above each of the last rows.

  Of course, in some other embodiments that use higher numerical accuracy to describe the coverage or luminance values that are balanced, the balance distribution used in a wide range of luminance is It has a ratio between the contributions to the different subpixels, rather close to those ratios reflected in the last row of 94 and 95. This is particularly true for the general type of filter illustrated in FIGS. 94 and / or 95, as described above with respect to FIGS. 48-52, for two-color sub-pixel optimization of an image. This is the case when used for color balance. This calculates the brightness of the output image at a lower resolution in such a two-color subpixel optimization for a bitmap image than that used in a subpixel optimized bitmap. This is because it is generally undesirable.

  When step 6018 is performed for each subpixel of each pixel in a row by loop 6008 of FIG. 60, each such pixel has a separate alpha value calculated for each of its three subpixels. Each such alpha value has one of 13 opacity levels. This means that each pixel can have one (ie 13 squared) of the different 2197 possible combined alpha values. In other embodiments of the present invention, alpha values with higher or lower resolution can be used.

  In many embodiments of the invention, particularly on computers with limited computing power, or to reduce the line or storage capacity required to store or download font bitmaps. In an embodiment designed to perform in the desired system, mapping from a relatively large color space that is a combination of 2197 different subpixel alpha values that can be taken after such color balance to a smaller color space. It is preferable.

  The embodiment of the present invention in FIG. 60 performs such a mapping in loop 6020. When loop 6008 is executed for each pixel in the current row, loop 6020 performs a further function 6022 for each such pixel. Function 6022 takes the three alpha values calculated for each of the pixel's sub-pixels and derives them from each of the 2197 possible color values defined by the possible combinations of the three alpha values of the pixel. , 122 is used as an input value for a lookup table that maps to one value. In this embodiment, the color space is reduced to such a small number of colors. As a result, a machine having a 256-value color space still has sub-pixel optimized font display while still having more than half of the color space so limited for other uses. Each of the 122 values selected for use in can be displayed. By using such a small color palette for displaying font bitmaps, the number of bits required to store such font bitmaps is reduced and more efficient to download. In other embodiments related to this aspect of the invention, the source and destination color spaces used in such mapping may have different sizes.

  FIG. 96 illustrates a method 9600 that is used to generate such a color mapping in one embodiment of the invention. Of course, other types of mappings can be used in other embodiments. In some embodiments, no such mapping to a smaller color space is required.

  The method of FIG. 96 uses multiple characters from one or more fonts in which a bitmap is displayed using a limited color space calculated by the nonlinear color balance subpixel optimization algorithm described above with respect to FIGS. Starting at step 9602. When this is done, the histogram holds the number of times each possible 2196 different composite pixel alpha value is calculated for any pixel. Although this histogram is useful, it means that most of the three color alpha values calculated for the pixels in the subpixel optimization bitmap are over the entire possible color space of 2196 such three color alpha values. This is because they tend to concentrate on various small areas. This concentration is likely to be more pronounced with respect to non-linear color balance because it substantially reduces the amount of luminance distribution due to color balance.

  Next, function 9604 generates a limited color palette having 122 colors in this example by executing functions 9606 and 9608. Function 9606 is the 13 gray scales that can be taken for all pixel alpha values if each subpixel can have one of 13 alpha levels as part of the palette being formed. Select a value. Next, function 9608 selects the 109 other most frequently occurring colors in the histogram previously calculated by step 9602.

  When a limited color palette is selected, a loop 9610 is performed for each of the 2196 possible total pixel alpha values. For each such possible alpha value, condition 9612 tests to see if the input color matches exactly one of the 122 colors. If there is a match, function 9614 associates the input color with its same output color in the constructed lookup table. If condition 9612 is not met, loop 9618 and function 9628 are performed on the current input color of loop 9610.

  Loop 9618 is performed for each of the 122 output colors in the palette. That is, the difference (ri-ro) between the red alpha value of the mapped input color and the current output color of loop 9618 is the green alpha value of the current input color and the green output alpha value for the current output color. And a condition 9620 for performing a test in order to confirm whether or not the same sign as the difference (gi-go). The condition 9620 indicates that the difference between the red alpha value and the green alpha value for the current output color (ro-go) is the difference between the red alpha value and the green alpha value for the input color (ri-gi). Test to see if it is less than (add possible value X to give some margin). If these two conditions are met, aimed at preventing a relatively significant difference between the input color and the output color to which the input color is mapped, functions 9622-9626 are executed.

  A function 9622 calculates the distance from the input color to the output color. Function 9624 tests to see if that distance is the shortest distance ever to the input color in the current loop 9618. If the test of function 9624 is satisfied, step 9626 saves the current output color of loop 9618 as the closest acceptable palette color. After loop 9618 has been performed for each of the 122 output colors of the restricted palette, step 9628 associates the current input color of loop 9610 with the closest allowable palette color calculated in loop 9618. .

  When loop 9610 is executed for each possible input color, each of those input colors is mapped to one of the 122 output colors.

  In the particular color mapping scheme illustrated in FIG. 96, the color value of the non-grayscale pixels generated by color balance is one of 109 most frequently occurring non-grayscale color values selected in step 9608. If not mapped to one, it is mapped to a grayscale color value. In general, this yields at least as good results as conventional anti-aliasing, which displays all bitmaps with grayscale alpha values.

  FIG. 97 illustrates an algorithm 9700 used to display a font bitmap of the type generated by the method of FIGS. 60 and 96 on a subpixel addressable screen.

  A function 9704 and a loop 9702 composed of loops 9706 and 9714 are executed for each character string to be displayed.

  Function 9704 samples a set of points in a bitmap rectangle where the string is drawn to determine an average background color value for the string. In other embodiments, the background color is determined separately for each character or for each pixel of each character, but in the illustrated embodiment, it saves computation. Therefore, the background color is determined only once for each character string.

  Once the background color for the string has been determined, loop 9706 performs sub-loop 9708 and function 9712 for each of the 122 total pixel alpha values described above with respect to FIG.

Loop 9708 performs function 9710 for each of the three subpixel colors. This function 9710 calculates a luminance value for the current subpixel color as a function for the current total pixel alpha value component corresponding to the current subpixel color.
Function 9710 is determined by function 9704 to a subpixel alpha value multiplied by the luminance value being calculated by the luminance for the corresponding color of the current subpixel in the foreground color of the drawn string. Set equal to the subtraction of the alpha value of the current subpixel multiplied by the luminance for the corresponding color of the current subpixel in the background color.

  When this loop is performed for each of the three subpixels, function 9712 calculates the current total pixel alpha value of loop 9706 to be the total pixel composed of the three subpixel luminances calculated in loop 9708. Maps to the color value of.

  Next, loop 9714 performs function 9716 and loop 9718 for each character of the current string displayed on the subpixel addressable display.

  Function 9716 accesses the font bitmap for the current character. Next, loop 9718 performs functions 9720 and 9722 for each pixel of the bitmap. Function 9720 determines the color value mapped to the current full pixel alpha value by loop 9706, shown for the current pixel in the font bitmap of the character. When this color value is determined, function 9722 sets the corresponding pixel in the subpixel addressable display to the color value of all pixels.

  When the loop 9718 is executed for each pixel of each character in the string, the string is fully displayed with the sub-pixel optimization method.

  FIGS. 98-101 illustrate how the techniques for image / font scaling and sub-pixel optimization work. FIGS. 98 and 100 illustrate an overview of two different web pages laid out and displayed at 640 × 480 pixels using a common browser program. FIGS. 99 and 101 illustrate the same web page after the images and text have been scaled in the manner described above to fit on a 320 × 240 display. Unfortunately, 320x240 pixel images are printed with grayscale values determined by the average luminance of all pixels, so the actual clarity added by the subpixel resolution is shown in these images. Not.

  102-113 illustrate in more detail the interaction between a proxy server and a thin client computer in one embodiment of the present invention.

  FIG. 102 is a highly schematic block diagram of a system that includes a proxy server 210 and thin client 200 of the type described above with respect to FIG.

  The proxy server 210 has a browser 10200 that includes a program 10202 for executing the standard functions of a complete web browser. This program is changed because the browser acts as a proxy for thin clients. When the proxy browser receives an HTML description 10204 of the requested web page over the network, it generates a two-dimensional layout 10206 of the web page.

  FIG. 103 illustrates a portion of the HTML description of the web page whose display is illustrated in FIGS. The numeral 10300 illustrated in FIG. 103 illustrates the text portion in the HTML illustrated in the left column of the web page illustrated in FIGS. The number 10302 points to an image tag that identifies the bitmap used to display the word “Section” illustrated in the same column.

  When the proxy browser code receives a web page download, it attempts to create a web page layout 10206 at the virtual layout resolution. The virtual layout resolution often corresponds to the width of the virtual screen size, which is the size of the window that the proxy browser considers to display all or part of a web page. The window that the browser considers as displaying a web page will be referred to as a virtual screen 10208.

  FIG. 104 illustrates the layout 10206 of the web page illustrated in FIGS. 98 and 99, which illustrates the mapping of the virtual screen screen to that layout in the black thick line rectangle 10208. FIG. Given the location of the virtual screen shown in FIG. 104 relative to the layout 10206, 10220 illustrates a virtual screen image displayed on the thin client.

  Many of today's web pages have elements larger than the 640 × 480 virtual screen resolution used in the described system. The layout has either the minimum pixel width required to lay out the objects of the web page or the virtual screen width, whichever is greater. For example, it is common today that many web pages are laid out with a resolution of the smallest possible width of 800 pixels. In this case, the virtual screen has a smaller pixel width than the layout. This is the case of the example shown in FIG. Given the virtual layout width, the height of the virtual layout is the height required to display all of the web page.

  The view window 10210 illustrated in FIG. 102 represents a portion of a virtual screen that is actually displayed on the thin client screen. In the views shown in FIGS. 99, 101, and 104, the view window corresponds to a virtual screen.

  However, if the user zooms in on a portion of the virtual screen, the zoom magnification control 10216 illustrated in FIG. 102 changes and the view window is mapped to a subset of the virtual screen.

  The view window moves corresponding to the layout by the scroll control 10218 shown in FIG. If the view window is moved to include a portion of the layout that is not currently located on the virtual screen, instructions are sent to the browser software to scroll the virtual screen. The event queue 10220 stores events, ie user input received on the thin client and uploaded by the browser to the proxy server for the corresponding action. Events that occur on the thin client's screen are mapped to the corresponding location on the virtual screen via the view window, and then placed in the proxy browser's event queue, so the proxy browser is as if it were video output It responds to such input as if it were received at the appropriate location on the virtual screen that it considers drawing directly on the device.

  Each time the browser program 10202 of FIG. 102 considers that it is drawing an object on the virtual screen, the corresponding reduced scaled position in the download display list 10212 is displayed. Modified to create an object.

  This display list is downloaded via the network 10222 to the client computer that stores the display list, as indicated by the numeral 10212A. A reduced image 10214 referenced by this display list is also downloaded. A program 10218 located on the thin client displays a character string, an image, and other elements included in the display list on the thin client screen 10221. When a user clicks on the thin client screen, the thin client operating system 10222 sets such clicks and click positions on the thin client screen in the event queue 10224. Each such event not related to a program handled locally on the thin client is uploaded to the proxy server's event queue 10220 as described above.

  FIGS. 105A-110 show a high level of programs and data structures on browsers and thin client computers designed to control these interactions so that thin clients can browse web pages through a proxy. This is a simplified pseudo code description.

  105A and 105B are highly simplified pseudo-code notations for a portion of the browser code 10202 illustrated in FIG. 102 that is used to facilitate acting as a proxy for thin clients.

  In the particular embodiment illustrated in these figures, a large web browser designed for normal use has been patched so that it functions as a proxy. Of course, in other embodiments of the present invention, the functions necessary to operate the browser as a proxy can be more closely and finely integrated into the browser code. In yet another embodiment, code in the operating system or code in a function that interrupts an operating system call can be used to run a standard web browsing program as a proxy for thin clients.

  In the embodiment illustrated in FIG. 105A, when the proxy browser receives a request from a thin client for a web page, steps 10502 and 10504 communicate the request to the server computer indicated in the URL of the request. .

  If the browser receives an indication from its own code that it has finished drawing or redrawing the virtual screen 10208 described above with respect to FIG. 102, the functions 10506 and 10510 perform the screen capture and download routines illustrated in FIGS. 106A-106C. call.

  106A-106C are highly simplified pseudocode descriptions for the screen capture and download routine 10600. FIG.

  If this routine is called by function 10510 of FIG. 105A described above, its step 10602 is a routine for rendering each element in the layout of a web page that fits entirely or partially within the virtual screen. Calls the browser to redraw the screen, called by. The routine of FIGS. 106A-106B records the information contained in each of these drawing calls and uses it to generate the download display list 10212 illustrated in FIG.

  When the browser calls the measurement string routine 10606 of FIG. 106A, functions 10608 through 10618 are executed by this routine. Such a call is made by the browser to determine the size of the text that is to be laid out on the virtual screen. Although not shown, these same functions 10608-10618 make calls to the browser to measure the size of the string even though the screen capture and download routines shown in FIGS. 106A-106B are not running. It is executed at any time. Thus, all virtual layouts of the web page use the font matrix provided by functions 1068-10618.

  Function 10608 maps the font specified in the measurement string call to a font having a different font family and a different font size. This font substitution is controlled by three things indicated by numerals 10610 to 10616.

  Item 10610 attempts to select a size for the replacement font as a function of the requested font size in a call to the measurement string routine and to select a display scale factor.

  The display magnification is the ratio of resolution along a predetermined dimension for the portion of the virtual screen 1028 corresponding to the view window and resolution along the same dimension that the view window is displayed on the thin client. In some cases, display magnification has different components to represent different scale ratios used along the horizontal and vertical directions, but in many cases display magnification is used for both horizontal and vertical resolutions. Consists of a single scale ratio.

  In the particular embodiment described, when a web page layout is performed, the layout is performed for non-zoom display magnification. The non-zoom display magnification is a magnification at which the view window 10210 in FIG. 102 is the same size as the virtual screen 10208. As a result, the non-zoom display magnification is equal to the ratio of the resolution of the virtual screen to the resolution of the screen on which the virtual screen is displayed. If the user zooms in on a part of the layout after the web page is displayed, the view window 10210 only corresponds to the lower part of the virtual screen, and the display magnification is the fact that the layout elements are displayed in a larger size. Change from its non-zoom value to reflect. The current embodiment lays out content that uses a non-zoom display magnification to optimize the layout of the text for the display of the entire virtual screen, which is why the user generally lays out the layout. This is because we are assuming that you want to see. In other embodiments, the virtual layout can be optimized for display magnifications other than non-zoom display magnification.

  In the embodiment illustrated in FIG. 102, this magnification is stored in the zoom / magnification control 10216. If the virtual screen has a resolution of 640x480, the view window is equal to the size of the virtual screen, and the view window is displayed over the entire 320x240 display, the magnification is "2", which means that the browser Means that the element is rendered on the thin client's screen at half-pixel resolution, assuming that the element is rendered on that virtual screen.

  Item 10612 in FIG. 106A replaces all font sizes that are smaller and higher when displayed on a thin client screen in a font family that is narrower and higher than the average pixel size of the font selected by item 10610 alone. When reducing from a 640 × 480 virtual screen to a 320 × 240 display screen, in contrast to the bitmap format, this can have most or all of the web page text expressed in characters. This replacement is performed, in part, when the subpixel addressable display used with this embodiment of the present invention is 3 times horizontal compared to the subpixel resolution in the vertical direction. This is because it has sub-pixel resolution in the direction. For this reason, reducing the character width has less negative impact on readability than reducing the character height. Thus, in order to display the maximum amount of text that is relatively easily readable on such a subpixel addressable display screen, this replacement effectively reduces the character width by a factor that exceeds the display magnification. The character height is effectively reduced at a magnification less than the display magnification. For example, the fonts of the small screen display shown in FIGS. 56, 57, 99, 101, 168, 169, 172, 173, and 174 are all replaced by fonts enlarged / reduced in such a manner.

  The fonts in these figures have a pixel size of 8 pixels / em. Most lowercase letters in this font fit within an advance width of 4 pixel columns or less. A width of 4 pixels or less includes the spacing that occurs between the shapes of characters having such width, if any. In these special fonts, more than 80% of the lowercase Roman letters fit within such advance width. These characters have a height “x” of 4 pixel rows or more, which is generally much higher than the width. In general, such a relatively narrow font can display more text within a given area at a given readability level than a wider font.

  The matter represented by the numbers 10614 and 10616 tests to see if a flag is set to limit the minimum font size, but this test is less than a predetermined pixel size and displays thin clients. Indicates that the font displayed above should not exist. Generally, this flag is set to prevent display of text that is too small to be read. When actually displayed on a display with a virtual screen size, it is not set when the user wants to see a more accurate reduced representation of how web page text is normally displayed. . Such demands are particularly likely to occur when the display magnification is large, i.e. making such a minimum restriction on the text size dramatically changes the appearance of the layout of the web page.

  In most cases, if the system limits the minimum font size, steps 10614 and 10616 prevent the replacement font size from becoming less than the minimum pixel size. In the current embodiment of the invention, this minimum pixel size is 8 pixels / em. The developer of this embodiment developed a hint font for displaying a sub-pixel at 7 pixels / em and thought that such a font was relatively easy to read. However, feedback from other users that such a small font is difficult to read. Accepted from.

  Limiting the minimum font size often substantially changes the relative font size at which the various sizes of fonts on the web page are actually displayed.

  In some embodiments of the invention, all web text is displayed in one font size. In most web pages, this method actually works very well on most web pages, since actually large fonts are represented by bitmaps.

  When the function 10608 determines which font family and font size the font for which the measurement string routine was called replaces, the font and font size replaced after the measurement is magnified by the non-zoom display magnification. Given the size of each character in the character string at, function 10618 returns to character string measurement of the character string from which the routine was called.

  By returning to this value, the browser layout engine actually displays the characters according to the unzoomed display magnification, ie the ratio of the virtual screen resolution to the actual resolution at which the virtual screen is displayed on the thin client screen. Lay out a web page that uses a font matrix for the characters to be scaled in relation to the pixel size to be scaled. This means that the virtual screen is laid out using a font matrix that is different from the actual font matrix displayed as a result of the layout.

  When the screen capture and download routine receives a call to the string rendering routine 10620 of FIG. 106A, the routine executes functions 10621 and 10624.

  The function 10621 converts the screen position where the drawing of the character string starts to the corresponding position on the thin client screen where the character string is finally displayed. This conversion takes into account the mapping between the view window 10210 and the virtual screen 10208 illustrated in FIG. This mapping reflects both the current zoom setting saved by control 10216 and the current scroll setting saved by scroll control 10218 illustrated in FIG.

  Function 10622 was requested to display the current string with the replaced font family and size associated with the string, from a previous call to the measurement string routine described above for numbers 10606 to 10618. Test to see if any other attributes are different from the current values for those font attributes. The current value for each such font attribute is determined by the last value for each such attribute defined by the font command already recorded in the download display list 10212 illustrated in FIG. If such a difference is found, function 10623 saves the font command at the current end of the display list that changes such font attributes to those suitable for displaying the current string.

  The function 10624 stores the screen position calculated and converted by the step 10622 at the end of the download display list 10212 illustrated in FIG. This is done by setting a character string command in the display list that includes the converted start position of the character string and its character, as will be described below with respect to FIG.

  When the screen capture and download routine receives a call to the rectangle drawing routine 10626, the routine executes functions 10628 through 10634. In order to draw horizontal and vertical lines that can be used not only to generate areas of a web page with different background colors, but also to underline text, or as boundaries between different parts of a web page layout, The browser calls the rectangle drawing command.

  Function 10628 converts the geometric value included in the call to a corresponding geometric value in which the corresponding rectangle is rendered on the thin client display. This includes converting the starting screen position of the rectangle, the width of the rectangle, and the height of the rectangle.

  Function 10630 tests to see if the color of the rectangle is different from the current (ie, last) rectangle color in the display list. If so, function 10632 adds a background color command to the end of the display list that changes the current background color to the color specified in the current call to the rectangle drawing routine.

  Next, the function 10634 saves the rectangle and the converted screen position, width, and height of the rectangle together with the rectangle command at the end of the download display list.

  If the screen capture and download routine receives a call to the bitmap rendering routine 10636 illustrated in FIG. 106B, this routine causes functions 10638 through 10670 to be performed. To display graphics, font graphics, banner advertisements, and images associated with hot zones and other graphical user interface (GUI) bitmaps of the page, the browser calls a bitmap image routine.

    In some embodiments, only the first screen for a given animation is captured and recorded in the download display list to reduce the line capacity required to display the web page. In other embodiments, it is not necessary to apply such constraints, especially for more advanced line capacity links.

  In the embodiment of the invention described above with respect to FIGS. 106A-106C, the bitmap drawing associated with a particular graphical user interface (GUI) is ignored, which is what the thin client program does for such control. This is because the sub-pixel optimized and reduced bitmap is saved.

  Step 10638 tests to see if the URL of the image for which the bitmap rendering routine was invoked already exists in a download image list (not shown) that includes each of the images referenced in the download display list. . If not, the requested bitmap has not yet been processed for the current download and there is no need to perform functions 10642 through 10661 against it.

  Function 10642 tests to see if the bitmap is a color bitmap. In the case of a color bitmap, functions 10644 through 10654 are executed by function 10642. Function 10644 scans the color image against one or more individual regions of sufficient size to justify separate processing, each containing only colors from a single two-color spectrum. A two-color spectrum corresponds to a set of colors in a line in the RGB color cube (ie, a color cube defined by red, green, and blue values is distributed in each of the three major dimensions).

  Functions 10648 and 10650 are performed by function 10646 for each of the two color portions of the image that appear to be large enough to justify the individual processing. Function 10648 uses the most extreme end of the two-color spectrum as its foreground and background colors in the current part of the image and determines how much it will reduce that part of the image. A two-color subpixel optimization of the type described above with respect to FIGS. 42-53 is performed using the current display magnification. The subpixel optimization performed in steps 10654 and 10658 described in the following paragraphs is the ratio of the resolution of the image in the proxy browser virtual layout to the resolution displayed on the thin client screen. The image is reduced by the display magnification which is.

  After this subpixel optimization is performed, function 10650 determines whether the foreground color is too chromatically unbalanced. That is, it is close to pure red, green, or blue. In this case, the purity of such a color reduces the accuracy with which it can display the spatial resolution of the color image. In this case, it is possible to replace the foreground color with a corresponding color that more closely resembles the grayscale value, thus allowing more accurate spatial resolution.

  In some embodiments of the present invention, such foreground color substitution is not used because it disturbs the color balance of the color image. In general, it is better not to use such foreground color replacement unless the foreground color appears in a significant portion of the overall color image. Such color replacement works best when a small font or other shape is represented by an optimized bitmap image. In other embodiments of the present invention, the background color associated with a two-color image can be changed. However, it is often not advisable to change the background color of an image on a web page.

  For each non-bicolor portion of the current image, by function 10652, step 10654 causes the multi-color subpixel optimization of the type described above with respect to FIGS. Execute the conversion.

  If the bitmap for which the bitmap drawing routine was called is a grayscale bitmap, then by function 10656, step 10658 causes the current display magnification to use black and white as the foreground and background colors, Perform two-color subpixel optimization.

  Function 10661 saves the reduced, sub-pixel optimized bitmap with the unique image ID, its URL, and the scaled width and height at the end of the image list.

  Regardless of whether the image from which the bitmap rendering routine was called was previously in the image list, by the time the program proceeds to function 10664, it will be present in the list, the ID number, and the converted width And a height is assigned. At this time, the function 10664 converts the screen position at which the bitmap rendering routine is called for the image into a position applicable to the thin client screen, and then the image ID of the image, the converted screen position. And store the image position command of the type illustrated in FIG. 108 with its converted width and height at the end of the download display list.

  In some embodiments of the invention, all bitmap images are subpixel optimized using a multicolor subpixel optimization routine. In other embodiments, only the grayscale bitmap is subjected to an optional two-color subpixel optimization process.

  In some embodiments of the present invention, vector images can be processed by performing sub-pixel optimization on the shape defined by the vector description. In some such embodiments, subpixel optimization is performed on the proxy, while in other embodiments it is performed on the thin client. One of the advantages of vector or geometrically defined drawing is the simplicity of being able to represent images by their description. Thus, if line capacity for thin clients is the primary limitation, it is possible to download vector descriptions for the images and then have the thin clients render them using sub-pixel optimization. .

  In some embodiments, it is possible to perform image recognition on an image and then download the recognized image to the thin client in symbolic notation. For example, it is common to display large text in bitmaps on many web pages. Optical character recognition can be performed on such bitmaps, and the corresponding characters and their fonts, or approximations of those fonts, to reduce the line capacity required to describe the page, Downloaded symbolically to thin clients.

  If the screen capture and download routine receives a call to a routine that creates a control object, such as a radio button, check box, text field, or button from a browser, for example, by the control generation routine 10666 illustrated in FIG. , Functions 10667 to 10670 are executed. Function 10667 translates the screen location that the browser requested control of rendering to the location where it was rendered on the thin client's screen. As illustrated in FIG. 108, function 10668 sets a control generation command corresponding to a download display list that includes a corresponding text label, and function 10670 generates a corresponding browser-side portion of the control object.

  In this embodiment of the present invention, the functionality of the control object displayed on the thin client screen is shared between the proxy and the thin client. For example, state information such as whether a check box is checked and which set of radio buttons is pressed is stored in the thin client. This avoids the need to communicate from the thin client to the proxy each time the user enters information into the control object. In general, a client needs to send such information to a proxy for relay to such a server because the information stored for that control is sent to the remote server computer that originally generated the web page. This is limited to the case where the user clicks on any button indicating that this is the case.

  In other embodiments of the present invention that have higher line capacity linked to thin clients, have more or substantially all the functionality associated with individual control objects running on the proxy Thus, it is desirable to simplify the thin client code.

  If the screen capture and download routine determines completion of the screen redraw requested by function 10602 of FIG. 106A, function 10672 of FIG. 106C causes function 10474 to execute the download display list routine 10700 illustrated in FIG. call.

  As shown in FIG. 107, the download display list routine includes a function 10702 that sets all elements in the download display list, all or part of which is newly displayed on the new thin client screen in the download stream. Yes. This typically includes any element in the browser's virtual screen that occurs within the current view window. However, as described below, in the case of scrolling where a significant portion of the previous bitmap can be reused on the thin client screen, the reusable portion of the current bitmap on the thin client screen. Only those elements that occur at least partially in the portion of the view window that do not exist in the download stream are set in the download stream.

  In many embodiments of the present invention, the function of FIGS. 106A-106C that generates a download display list does not enter an element on the download display list if it does not fit within the view window. In other embodiments, this filtering is performed in function 10702.

  In many embodiments of the present invention, the downloaded element is clipped to download only those portions of the element that actually fit within the thin client screen. This has the advantage of reducing the number of bits required for download, but it complicates the operation.

  When all elements on the download display list displayed on the thin client screen are set in the download stream, the function 10704 is set in the download stream, as indicated by the numeric item 10818 in FIG. Bitmaps of all images with corresponding image position commands are placed at the end of the download stream. Prior to setting such bitmaps at the end of the download stream, some embodiments of the present invention perform lossy compression on them. In some embodiments, the algorithm used is that the difference in green color value is more perceptible than the difference in red color value, and the difference in red color value is An algorithm that uses a metric that takes into account the fact that it is more perceptible compared to the difference and aggregates the color values in the image into clusters of colors that have visually unperceivable differences in RGB color values. .

  Function 10705 then uses a lossless compression algorithm to compress the download stream containing the images previously compressed by the lossy algorithm. For this purpose, standard prior art lossless compression algorithms can be used.

  FIG. 108 is a schematic diagram regarding such a download display stream. In some embodiments, such a stream is actually represented using a markup language.

  A font command 10812 illustrated in FIG. 108 represents a font command recorded in the display list by the function 10623 in FIG. 106A.

  A character string command 10814 in FIG. 108 represents a command recorded in the download display list in step 10624 in FIG. 106A.

  A background color command 10806 in FIG. 108 represents a background color command input by the function 10632 illustrated in FIG. 106A.

  A rectangle command 10808 in FIG. 108 represents the rectangle information saved by the function 10634 in FIG. 106A.

  The image position command 10810 illustrated in FIG. 108 represents the image position command recorded by the function 10664 of FIG. 106B.

  A control command 10816 in FIG. 108 represents a control command set in the download display list by the function 10668 in FIG. 106C.

  Referring back to FIG. 107, once all elements for the download stream have been selected and are ready to be sent, function 10706 opens a socket connection between the browser computer and the thin client, and then the function 10708 transmits the display list information of the download stream to the thin client. The thin client then displays the information as described in detail below with respect to FIGS. 109A-109C.

  Referring now again to FIG. 106C, upon completion of the call to the download display list routine in function 10474, function 10676 clears the display list, and for the next screen downloaded to the thin client, a new one is displayed. Display lists can be generated.

  Referring again to FIG. 105A, the completion of the screen capture and download routine called by function 10510 illustrated in FIG. 105 is as just described.

  If the browser proxy code receives a query from another part of the browser code regarding the state of the one or more control objects displayed on the thin client screen, as illustrated in FIG. Thus, functions 10516 and 10518 are executed. Function 10516 sends a query to the thin client regarding the state of the one or more control objects. If such state information is received from the thin client, function 10518 returns it to the program that made the request for such state information.

  As described above with respect to functions 10666 to 10670 in FIG. 106C, this embodiment of the present invention allows the user to make a selection before making a selection with such status information sent to the website of the web page on which such control is displayed. Therefore, in order to reduce the communication line capacity, for example, the client actually draws and saves state information regarding individual control objects such as radio buttons, check boxes, and text input fields. In general, when the user clicks the submit button, the associated click event is sent to the proxy computer, which converts the screen coordinates to the corresponding coordinates on the virtual layout screen, and then has the virtual screen resolution. The browser code for responding to the click event is set in the browser's event queue as if the click event had been generated on the screen that the browser assumed to be displaying. When this is done, the browser code conventionally requests the state for all control objects of the current web page, thereby notifying the web server from which the current web page is derived that the information is returning. It is such a request that causes the operation of functions 10514 to 10518.

  When the browser proxy code receives a scroll or move command from the thin client, function 10520 causes functions 10522 through 10534 of FIG. 105A to be executed.

  Function 10522 moves the view window 10210 illustrated in FIG. 102 relative to the browser layout 10206 in response to scrolling or moving. The function 10526 then tests to see if any significant portion of the view window that was present in the view window before the move is still present in the view window after the move. If a significant portion is present in the view window, this means that a substantial portion of the bitmap currently displayed on the thin browser screen can be reused in the display after performing the requested scroll or move. It means that it is possible. In this case, function 10528 sets the scroll command 10804 illustrated near the top of the download stream in FIG. 108 to the starting position of the new display list generated for the scrolled screen. Such a scroll command includes an XY shift value indicating which portion of the previous screen bitmap of the thin client screen is to be reused.

  In FIG. 108, a clear command 10802 and a scroll command 10804 are shown at the start position of the download stream, and thus both commands can be explained. In this embodiment, only one of the two commands, the clear command or the scroll command, starts a predetermined download stream, and the clear command is used when the thin client screen is entirely redrawn. And the scroll command is used when the portion of the thin client's previous bitmap is shifted for reuse on a new screen.

  Downloaded to a thin client in a scroll that involves a relatively small change in position by reusing a substantial portion of the screen display, previously downloaded and rendered, enabling scroll commands The amount of data can be substantially reduced. For example, when a browser and a thin client communicate via a relatively low-speed digital cellular phone link, which is common at the time of filing of this application, the line capacity between the browser and the thin client is particularly limited. In certain circumstances, this can substantially improve the rate at which scrolled screens can be displayed on thin clients.

  If the moved view window resulting from the scroll or move command includes a portion of the layout of the web page that does not currently exist on the virtual screen 10206, schematically illustrated in FIG. 102, the function 10530 of FIG. 10532 and 10534 are executed. Function 10532 scrolls the browser virtual screen so that all of the view windows are contained within the browser virtual screen, and then function 10534 redraws from the browser for the newly moved virtual screen. Request. Upon completion of this drawing, functions 10506 and 10510 capture and draw the newly drawn elements as described above with respect to FIGS. 106A-106C.

  When the browser proxy receives a zoom command from the thin client, functions 10538 through 10552 are executed by function 10536 in FIG. 105A.

  A function 10538 changes the display magnification according to the zoom change.

  Function 10540 scales the view window relative to the browser virtual window according to the selected zoom.

  Function 10542 checks to see if the scaled view window includes a portion of the web page layout that is not currently included in the virtual screen. When including the layout portion of the web page, function 10542 causes function 10544 to scroll the virtual screen or change the resolution to fit the scale view window within the virtual screen.

By scrolling the virtual screen, if the new view window can fit into the virtual screen, there is no need to lay out the web page again, displaying different locations within the virtual screen, and / or different magnifications With the virtual screen display, the zoom can be used to display the same layout that existed before the zoom. However, in the embodiment illustrated in FIG. 105A, if the zoom is a zoom out that enlarges the view window relative to the virtual screen size, the proxy browser may allow any part of the view window displayed on the client. A new view window that can be fully adapted to fit within the virtual screen so that any input supplied to it can be processed as if it occurred at the corresponding location on the proxy browser's virtual screen. It is necessary to display a web page with a large virtual screen size. In the described embodiment, if the new virtual screen resolution is larger than the layout resolution used in the previous layout, this causes the web page to be displayed in the new layout, causing line breaks at different positions. For example, an embodiment designed to support zoomed display rather than a proxy browser being patched, described above with respect to FIG. 115 where the client zooms directly in relation to downloading the entire layout. In other embodiments related to the present invention, such as the embodiment, extreme zoom-out does not require web page re-layout.

  Finally, function 10552 calls screen redraw. As a result, the screen capture and download routine acquires a redraw of the current view window at the new zoom magnification and downloads the corresponding display information to the thin client, so that the web page can be displayed with the new zoom setting.

  As shown in FIG. 105B, when the browser proxy receives the virtual resolution change command from the thin client, the functions 10556 to 10560 are executed by the function 10554. Function 10556 changes the virtual screen resolution of the browser stored in the virtual resolution control 10214 illustrated in FIG. 102 to the requested resolution. Next, step 10560 requests a screen redraw. This is the display magnification corresponding to the ratio of the number of pixels the view window has on the virtual screen to the number of pixels the view window has on the thin client screen when the browser lays out the current web page again at the new virtual screen resolution. This is because the entire virtual screen is redrawn.

  Such a change in virtual resolution changes the size at which the layout is performed relative to the size of the images and text in such a layout. Unless the user makes a change in the relative size of the view window in relation to a virtual screen that cancels such a change in size, such a change in the relative layout size will cause the screen to change. Change the size of the image and text displayed above. In the absence of such compensatory changes in the relative view window size, reducing the virtual resolution increases the size at which images and text are displayed on the screen, allowing more text lines to be larger text. There is a tendency to make the text lines shorter in relation to the size of the fonts displayed on them to fit on the screen at a time in size. Therefore, changes in the virtual layout size can be used to provide a certain type of zoom function to the display of the web page.

  The inventor is very good at using a 640x480 virtual screen when displaying a web page on a 320x240 screen, which is a typical PDA size, including reducing the layout by a factor of 2 I found out that it can provide easy readability. However, in order to allow the user to see how the web page will look when displayed against a larger resolution display, even if readability is sacrificed, for example PDA size The present invention can be used to display a web page at a further reduced scale, such as when displaying a 800 × 600 virtual screen resolution on a 320 × 240 screen. Needless to say, the layout of the page is quite different from the layout originally intended to display at such resolutions due to the relative increase in the resulting font size, but the minimum font size is limited. , Even text with a larger virtual resolution is displayed in a readable font, as described above with respect to function 10614 of FIG. 106A.

  If the browser proxy code receives other user input from the thin client in connection with a click on the thin client screen, as indicated by function 10562 of FIG. 105B, the function 10564 is displayed on the virtual screen. In response to a click on the corresponding position of the, it converts the position of the thin client screen, as if most of the browser code actually clicked on the virtual screen it considered to be laying out Function 10566 relays this event to the browser's event queue so that the browser can respond to the event.

  Whether this is a text link or an image link, the browser on the proxy responds to input from the thin client user to select the web link on the web page displayed on the thin client Is the method. For example, if a user clicks on a link displayed on a thin client screen, it supports a browser on a proxy that behaves as if the user clicked on the same link on the virtual screen that it considers displayed Click to be relayed. The proxy browser then responds by issuing an HTTP request over the Internet in response to the link. If a web page corresponding to the link is received, the browser will attempt to lay it out on the virtual screen and display it, so that functions 10506 and 10510 in FIG. 105A are placed in that portion of the layout corresponding to the view window. Capture the contained information and let the thin client download it for display on its screen. As a result, thin client users can browse the web in much the same way as regular browse computer users.

  FIGS. 109A-109C illustrate a thin client designed to help work in conjunction with a proxy browser so that a thin client computer user can use its screen to browse the World Wide Web (WWW). A highly simplified pseudo code notation for code 10900 on a computer.

  The function 10902 of FIG. 109A corresponds to the reception of all or the first part of the download stream sent to the thin client by the function 10708 of FIG. The function 10902 performs such a response by starting to respond to individual commands of the type illustrated in FIG. 108 that are included in the stream in the order in which they are received. The function 10902 starts as soon as one or more commands are received so that the drawing work for a new screen does not need to be delayed until the download stream is completely received. Correspondence to various different commands included in the download stream is indicated by functions to which numerals 10904 to 10956 are assigned in FIGS. 109A to 109B.

  When the thin client reads a clear command in the download stream, as indicated by functions 10904 and 10906, the bitmap displayed on the screen is set, for example, by setting it to a generally white value. Cleared.

  When the thin client reads a scroll command in the download stream, functions 10910 and 10912 are executed by function 10908. Function 10910 copies a portion of the thin client screen bitmap that will be reused after the scroll specified in the scroll command to the new position on the screen indicated by the XY shift value included in the command. To do. Function 10912 then clears the remaining portion of the screen.

  When the thin client reads a background color command in the download stream, functions 10914 and 10916 set the current rectangular background color variable to the color specified by the command. Thus, until the background color value is changed, all rectangles drawn by the thin client corresponding to the rectangle command have the specific color value.

  When the thin client reads a rectangle command in the download stream, functions 10918 and 10920 use the current background color and draw a rectangle with the screen position, width, and height specified in the command. .

  If the thin client reads an image position command, functions 10922 and 10923 do nothing at this point. This is because the bitmap required to render the image referenced in such an image position command is usually not received in such a case. In another embodiment, the browser associates a rectangle drawing command with the image so that a portion of the browser screen associated with the image has a rectangle drawn on the image that indicates the display position of the bitmap image. It will be.

  When the thin client reads a font command, functions 10924 and 10926 set all the font attributes listed in the font command to the values listed for those attributes in the command.

  Different font attributes can be used in different embodiments of the invention. At least the font family, font size, and font foreground color are preferably supported in the font attributes.

  When the thin client reads a character string command in the download stream, functions 10930 to 10940 are executed by the function 10928.

  Function 10930 determines whether the thin client has a bitmap in the font bitmap cache for each character of the current string in the current size and font family specified by the current font attribute value. To test. If it does not have this bitmap, functions 10932 through 10936 are executed.

  Function 10932 sends an HTTP request over the client's Internet connection to font server 134 described above with respect to FIG. If the requested font is received from the font server, functions 10934 and 10936 set it in the thin client's font bitmap cache.

  It should be noted that in some embodiments of the present invention, the font bitmap is permanently saved as part of the thin client browser software so that the functions 10930 and 10936 need not be used. is there. In other embodiments, as illustrated in the font bitmap in the example illustrated in FIG. 109A, the subpixel optimized font outline is either permanently stored or required by the thin client. As required.

  If the thin client has all of the font bitmaps needed to render the current string, functions 10938 and 10940 use the current font attribute value, including the foreground color, to specify the string for the specified screen. Draw on the screen at the position. In the current embodiment, the font bitmap is represented as an alpha value bitmap of the type described above with respect to FIGS. When executed in this manner, the background color is extracted from a part of the screen bitmap on which the character string is drawn.

  In some embodiments, to reduce computation, the color value of the portion of the screen on which the string is drawn is sampled at a relatively small number of points and the average of these sampled color values. Is used as the background color for the entire character string display, as described above with respect to FIG.

  In the described embodiment, all of the strings contained in the download stream are single line text strings, many of which are continuous text wrapping across line boundaries by the proxy browser layout engine. Arising from. As a result, in this embodiment, the thin client need not perform any wrapping of such text.

  The function 10940 draws a bitmap image of a character string by composing a plurality of different font bitmaps corresponding to the characters of the character string. Usually, in such a configuration, each different character is represented by a different separate font bitmap.

  The fonts used in such a configuration with different font sizes (eg, different font sizes caused by changes in display magnification) are the shape of each character selected to improve readability at each such font size and It has a pixel array. In most embodiments, such improved readability is achieved by selecting the character shape and position relative to the font bitmap to improve the character shape alignment using bitmap pixilation. To realize. Such shapes and pixel arrays are particularly important when dealing with font bitmaps of 10 pixels / em or less, but are more important at 8 pixels / em or less. This is because of the coarser pixilation of the font bitmap, it becomes harder to read as the font bitmap becomes smaller, and therefore the shape of the character that the font bitmap has been chosen to conform to such pixilation And having an arrangement becomes even more important.

  In many embodiments of the present invention, the font bitmap used by the smaller scale step 10940 is generated by a non-linear color balance of the type described above in which only the color imbalance occurring within the pixel is distributed. A sub-pixel optimized bitmap. When such subpixel optimizations are combined with well-formed and placed outlines to fit well with their coarse bitmap pixilation, the resulting bitmaps rendered will have their small pixel size Is extremely easy to read. In other embodiments of the present invention, non-subpixel optimized anti-aliased fonts are used that also have outlines that are formed and positioned to improve readability at such small pixel sizes.

  Referring now to FIG. 109B, when the thin client reads a control command from a download stream, functions 10944 through 10948 are executed by the function 10942. The function 10944 performs a test to confirm whether data corresponding to the control ID specified in the current control command or a program object has already been generated. If not, function 10946 creates the type of data or program object specified in the control command and associates it with the control ID specified in the command.

  Step 10948 then renders a sub-pixel optimized bitmap of the specified type of control object on the thin client screen at the location specified in the control command. Step 10948 then renders the text associated with the control on the control object's bitmap using the sub-pixel optimized font. Step 10948 then associates a hot zone having a display screen location corresponding to the control bitmap with a data or program object representing the control on the thin client.

  When the thin client reads an image command from the download stream, functions 10950 to 10956 are executed by the function 10950.

  The function 10952 scans the current display stream each time an image position command having the same image ID as the current image command is generated. For each such image position command, function 10952 causes function 1095 to render a bitmap at the position specified by the image position command on the thin client screen. As with all thin client drawing functions, any portion of the image that does not fit on the thin client screen will be cropped in such a drawing operation.

  Next, function 10956 redraws all other items in the display list that occur at the same position as any of these drawn bitmaps. This is typically required when a web page sets the text at the top of the image, so any string that should appear in the same position as the bitmap image is rendered by these images. It is desirable to redraw afterwards. In one embodiment of the invention, the thin client simply redraws all non-image elements in the display list of the download stream that occur after the first image position command in the list.

  As shown in FIG. 110, when the user clicks on a hot zone associated with a text entry field, the functions 10958 and 10960 in FIG. .

  Function 10962 displays a pop-up user keyboard 11102 and a text edit field 11104 on the thin client screen, as shown in FIG. The loop 10964 is then executed until the user presses the “Enter” key on the pop-up keyboard. During this loop, each time the user types a text character, function 10966 causes function 10968 to set the subpixel-optimized text bitmap for the character to the current cursor position on the text editing line of the pop-up keyboard and the cursor bit. Move the map to the position after the newly drawn character. The function 10970 then adds the typed character to the temporary text edit string associated with the pop-up keyboard program.

  When the user presses the “Enter” key on the pop-up keyboard, functions 10972 through 10974 are executed by the function 10972. Function 10974 saves the value of the temporary text edit string associated with the pop-up keyboard in the text edit control that causes the pop-up keyboard to be triggered. Function 10976 uses the subpixel optimized bitmap to render the text edit string characters in the bitmap of the text input field 11000 of the control object on the thin client screen, as shown in FIG. .

  Next, function 10978 removes the pop-up keyboard from the thin client screen by drawing on the pop-up keyboard a bitmap displayed on the screen before the pop-up keyboard is drawn.

  FIG. 113 illustrates that the pop-up keyboard routine can be used for purposes other than entering text into the text entry field. Although not displayed in the pseudo code of FIGS. 109A-109C, the pop-up keyboard can also be used to enter the URL of the web page that the user wants to see displayed on the thin client.

  FIG. 114 is substantially identical to FIG. 113 except that it illustrates an embodiment of the present invention having a button bar or toolbar at the top of a graphical user interface (GUI). This button bar has at its left end a “Back” button and a “Forward” button of the type commonly found in web browsers. It is also a button that is generally found in web browsers, but buttons labeled "R", "B", and "H" corresponding to the refresh button, bookmark button, and history button, respectively. Have. The button bar also has a URL text input field in which the pop-up keyboard shown in FIG. 114 appears when clicked. If the pop-up keyboard is not displayed, this text entry field displays the URL of the current web page displayed on the thin client screen. In an embodiment of the present invention, the user can select to display or hide such a toolbar by pressing a hardware button. In this embodiment, even if such a toolbar is not displayed, the user can click a hardware button to invoke some of the more common web browsing functions, such as back commands and forward commands. Can be used.

  In other embodiments of the present invention, such a graphical user interface (GUI) toolbar also includes zooming of the display of the web page and / or relative layout size (ie, virtual layout resolution). It is preferable to have buttons or menus that allow the user to access other functions.

  Referring now again to FIG. 109B, if the user clicks on a button or menu item controlled hot zone, functions 10981 and 10982 are executed by function 10980.

  Function 10981 changes the appearance of the button or menu item appropriately. In the case of a button, the bitmap associated with the button is redrawn to indicate that the button is pressed. In the case of a menu item, the submenu is displayed or the display of the menu item is removed depending on whether or not the final selection has been made.

  In the case of a menu item, if the final selection is made, or if the button is pressed and released, the function 10982 is the button control ID or menu item control ID and it is selected. The browser responds by operating the corresponding button or menu item control object on the browser as if it were pressed.

  If the user clicks on a hot zone associated with another type of thin client control, function 10983 changes the appearance of the control's bitmap on the thin client's display accordingly. For example, in the case of a check box, a check is displayed in or removed from the control display on the screen. Next, step 10985 saves the corresponding state change corresponding to the control object. As described above, in the described embodiment, the state of such control objects is not communicated to the browser until the browser requests such information in order to reduce communication requirements.

  When a user clicks on a client program or other part of the thin client screen that is not associated with the computer's control interface, functions 10986 and 10987 are associated with the click in addition to the screen location where the click occurred. Event to be sent to the proxy browser. As described above with respect to functions 10562 through 10556 of FIG. 105B, the browser converts the position of such clicks to a corresponding position on the virtual screen, and the screen that the browser considers to be rendering at the resolution of the virtual screen. Above, respond to such clicks as if they had occurred. In some embodiments, to further reduce communication requirements, only if the thin client has a rationale that other clicks match the user input that the proxy browser is supposed to support The thin client reports these other clicks to the browser.

  Referring back to FIG. 109C, if the thin client receives a query from a proxy browser requesting the state of one or more control objects, function 10988 causes function 10998 to return the corresponding control on the thin client. Querying the status, function 10909 sends such status information to the proxy browser. As described above with respect to function 10518 of FIG. 105A, the proxy browser then requested it as if it was part of the current state of the corresponding control object associated with the virtual screen. Return the requested information to the browser part.

  If the thin client user enters a command to scroll the screen, functions 10991 and 10992 upload the scroll command to the proxy. This causes the functions 10520-10534 described above with respect to FIG. 105A to generate and download a new download stream to display the current web page at the newly scrolled position.

  When the user enters a command to change the zoom (ie, scale) of the image displayed on the thin client, functions 10993 and 10994 upload the corresponding zoom command to the proxy. Thereby, the functions 10536 and 10552 described above with respect to FIG. 105A result in a new download stream downloaded to the thin client to display the current web page with the new zoom setting.

  If the user enters a command to change the virtual resolution for the display of the thin client, i.e., changes the resolution at which the virtual screen on the proxy browser is laid out, functions 10995 and 10996 will return the selected virtual resolution. Upload to proxy. This causes functions 10554 to 10560 described above with respect to FIG. 105B to change the virtual screen resolution of the proxy browser, re-lay out the virtual screen at the new resolution, and send the corresponding download stream to the thin client. A portion of the virtual screen corresponding to the window can be displayed on the thin client screen with the current zoom setting. Since changes in virtual resolution tend to change the relative size of layout elements, such as text and images, relative to the width of the virtual layout, in this document virtual resolution is relative. Sometimes referred to as layout size.

  If the user enters other commands associated with the thin client control graphical user interface (GUI), as illustrated at the bottom of FIG. 109C, this is indicated by the ellipsis 10999 by function 10997. An appropriate response to is executed. Such other functions include bookmark selection, access to bookmarked web pages, Back and Forward functions, and any other functions that can be part of the browser user interface. Use of physical buttons or other physical input to thin client computers, eg selection of graphic objects such as buttons, menu items or dialog box controls, or any other well known graphical user interface Such requirements can be selected by (GUI) technology.

  115-117 relate to an alternative method that allows thin clients to browse the web through a proxy server. In this embodiment, as illustrated in FIG. 117, the overall layout 10206 of the web page generated by the proxy computer is downloaded and cached by the thin client. As will be described later, this allows the thin client to scroll through the layout substantially faster, but this will attempt to download the entire layout and all images for each web page that the thin client sees. This may increase the total number of bits downloaded.

  FIG. 115 illustrates a portion of proxy browser code 11500 that can be used in such a page layout caching scheme.

  In this embodiment of the invention, when the proxy browser receives a request for a web page from a thin client, functions 11504 through 11524 are executed by function 11502.

  In this particular web caching embodiment, as illustrated by the pseudo code associated with function 11502 in FIG. 115, the thin client can specify the desired virtual resolution, zoom settings, and view window position. A web page can be requested with desired view settings for the page. This is done so that each time a page is requested, the user automatically selects such a web page at the desired virtual resolution, zoom setting, and view window position without having to enter such settings separately. This is to allow the user to associate such view settings with a bookmark that includes a specific URL or URL pathname portion so that it can be viewed. For example, this allows the user to view a normally accessed web page with an automatically zoomed display at the desired location on the page.

  When a request for a web page is received from the thin client, function 11504 on the proxy browser requests the web page from the server identified in the URL requested from the thin client. When a web page is received from the server, functions 11506 through 11516 are executed by function 11506.

  Function 11507 causes the browser's layout engine on the proxy to lay out the received web page with the virtual screen resolution associated with the view settings specified in the request from the thin client for the web page. This layout is performed using scaled string measurements for the substituted fonts in a manner similar to that described above with respect to functions 10606 through 10618 of FIG. 106A. The magnification used is determined by the view settings specified in the request for the current page.

  Function 11508 identifies the virtual screen location in relation to the resulting layout, including potential view windows in view settings for the current request. Thus, for example, if the view settings require viewing the rightmost 640 × 480 portion of the layout, and the layout is forced to have an 800 pixel width, the virtual screen position will be approximately pixels in the layout. The column 160 is expanded to the pixel column 800.

  As schematically illustrated in FIG. 117, function 11518 causes function 11520 to scale each received image 11702 associated with the laid-out web page and subpixel optimization. To do.

  When all images referenced in the web page have been received, scaled, and sub-pixel optimized, function 11522 causes function 11523 to create a display list for the layout, the display list, And compress all relevant sub-pixel optimized, scaled images. Next, function 11524 sends the compressed data to the thin client in a download stream that includes a web page layout followed by a reduced sub-pixel optimized image.

  In other embodiments relating to this aspect of the invention, a portion of the layout can be downloaded to the thin client before all or even a portion of the image is downloaded to the proxy server. Thereby, the thin client can start displaying the text of the web page more quickly. Many web pages contain a description of the size of all of their images, so in many cases it is possible to perform proper layout of web pages before the images are received by software that performs such layout. I can do it. If not, the page must be laid out again to verify the size of the entire image, after which these new layouts can be sent to the client.

  If the user receives a request from a thin client to rescale / re-scale and sub-pixel optimize one or more previously downloaded images at different scales, functions 11526-11532 re-scale those images. / Reduce and sub-pixel optimize, compress and download to thin client. This allows the user to view the web page with different sub-pixel optimized sizes when the user wants to view the downloaded web page layout with different zoom settings.

  When a screen input event is received from the thin client, functions 11536 through 11542 are executed by function 11534.

  Function 11536 tests to see if the page layout coordinates associated with the command match the portion 10206A of the web page layout 10206 currently mapped to the proxy browser virtual screen 10208 shown in FIG. I do. If not, function 11538 scrolls the virtual screen to a new portion 10206B of the layout that includes the layout coordinates associated with the command.

  Function 11540 calculates virtual screen coordinates corresponding to the page layout coordinates of the received screen event. The function 11542 then displays the input screen event along with the virtual screen coordinates so that it can respond to an event such as a link click, as if the user clicked on the corresponding virtual screen coordinates in the virtual screen itself. Set in the browser event queue.

  FIG. 116 is a highly simplified pseudo code description for a portion of thin client code that can be used to support the page layout cache scheme illustrated in FIGS. 115 and 117.

  When the thin client starts to receive the download stream including the display list of the page layout, the functions 11604 and 11606 are executed by the function 11602.

  Function 11604 sets the mapping of the view window (eg, view window 10210A illustrated in FIG. 117) in relation to the page layout, and then calculates the current client-side display magnification based on the mapping.

  Function 11620 displays any portion of the downloaded page layout that fits within the current view window using the current client-side magnification. This process includes functions 11622-11630.

  A function 11622 displays each character string element generated in the current view window using a font size which is a function related to the current client-side magnification. If that is done, when the pixel size at which such characters are displayed changes, if necessary, to the relative size of the characters, which may result from non-uniform effects due to font hinting Make adjustments for any non-uniform changes in To provide WYSIWYG communication between text display on a computer screen and the appearance of the text when printed at higher resolution, for example, spacing between characters The function 11622 does this by using techniques that compensate for discontinuities and imbalances, such as changes in. If a font bitmap with a size different from the previously displayed size is required, access the font bitmap for those different size characters from the storage on the thin client and from the network font server. Or rasterizing at the required size from the font outline.

  If the display generated by function 11620 is at a different magnification than the magnification at which bitmap image 10818 was reduced, functions 11626 through 11630 are executed. They require the proxy server to re-scale / re-scale all images that are wholly or partly in the view window with the new magnification and to sub-pixel optimize. The bitmap of the same image is then rescaled locally from the previously reduced subpixel optimized image 10818 stored and displayed on the thin client to provide a temporary display for such images. The Then, if the requested image and the higher resolution bitmap associated with the web page are received by the thin client from the proxy server, they are scaled again from the original size, and they are displayed on the display screen as appropriate. It is drawn at the correct position.

  In some embodiments, if the user changes the zoom of the display, the bitmap of any image corresponding to the portion of the page on the screen at the new magnification will be downloaded with the appropriate sub-pixel optimized image. Until then, it is displayed in a quick but coarse representation of the image generated on the thin client to provide the user with a temporary display of such an image. If the new magnification is an integer multiple of the previously downloaded bitmap magnification, it is relatively easy to generate such a quick display. If it is not an integer multiple of the bitmap magnification, a temporary display can be generated in any of a number of ways. For example, they can be sub-pixel optimized images generated by one of the methods described above with respect to FIGS. The basic procedure described in these methods can be applied even when the source image has a lower resolution than the output image. Furthermore, a computationally simple method occupies more space than an appropriately scaled image intended to be rendered as an image with an integer conversion ratio and relative to a previous image. The previous image is displayed as an image that has been cut out at a size that is the same or smaller than the appropriate scale, or an enlarged / reduced image.

  When the user generates screen input to be sent to the proxy browser, functions 11632 through 11636 convert the thin client screen coordinates of the input into corresponding page layout coordinates. Then, the screen input and the corresponding page layout coordinates are uploaded to the proxy browser. The proxy browser then responds to such input using functions 11534 through 11542 described above with respect to FIG. This allows the proxy browser to respond to such screen input as if the user had clicked on the corresponding portion of the web page on the proxy browser's virtual screen.

  A thin client is currently displayed on the screen, eg, as described above with respect to FIGS. 115-117, so that the user can scroll and / or zoom more quickly in relation to the content of the web page. A caching scheme that can store more parts than web page parts can be used. This is especially true when the thin client has a relatively low line capacity relative to the proxy server.

  To enable relatively quick browsing of the web, even when using a line capacity as low as that associated with the current digital mobile phone communication speeds generally available at the time of this application Embodiments relating to such caching schemes described herein can be implemented. This is because all content except images contained in most web pages can usually be compressed to fit 3000 bytes or less. Therefore, at the current digital mobile phone communication speed, the entire text portion of most web pages can be downloaded in seconds, and the first portion of text can be rendered in a shorter time. Needless to say, it may take more time to download the image, but the first image on the web page, other than the large image, can generally be displayed within a few seconds. This delay can be greatly reduced with a higher-speed communication line.

  118-120 illustrate features relating to the present invention that can be used in a virtual web browsing environment, but are particularly useful when browsing the web on a small screen. This includes, for example, use on small screen devices such as thin client computers discussed above. These features of the present invention include zooming in or out on selected portions of the web page so that it can be operated quickly on a thin client computer by using the layout caching scheme described herein. Can be executed.

  FIG. 118 shows a diagram of a standard web page that is laid out at a virtual resolution of 640 × 480 and then reduced and subpixel optimized for display on a 320 × 240 screen. Most people with good vision can read such content at a distance that usually uses a handheld computer. However, when displayed in a larger size, the content of most web pages can be made more easily readable. Since most web pages are laid out in multiple columns, it is often desirable to be able to quickly zoom the display to the top of the column portion that the user wants to begin reading. The user can do this by dragging the pointing device 11902 beyond the desired text column to a vertical position that the user wants to display near the top of the display screen in a zoomed view. . When the display is in a mode that performs this type of zooming, the horizontal linear drag of the type illustrated in FIG. 119 allows the web page layout of the dragged web page layout to be matched to the screen width. Increase / decrease width. In the example illustrated in FIG. 119, this user input zooms the display as illustrated in FIG.

  The user interface also allows the user to drag a selection box around the area in the web page layout shown on the screen so that the area selected on the web page will fit on the screen. Preferably, the system zooms the display of the web page.

  Also, in such a drag, it is desirable that the user be able to drag the pointing device beyond the boundary associated with a given edge of the screen, and if this is done, corresponding drag As a starting point, the user performs a zoom to fit the width, height, or area in a web page layout that is too large or improperly positioned to fit entirely within the screen Such a drag that scrolls the portion of the web page displayed on the screen to make it selectable will select the portion of the layout that is too large to fit on the screen at the magnification displayed during the drag. In order to reduce the size at which text and images are displayed, the magnification is changed.

  The fact that the selected width, height, and area in the layout are scaled to fit the screen means that its maximum dimension is two-thirds of the screen and a fully corresponding dimension. It means that it is enlarged / reduced so as to be distributed in the range between. In general, with such scaling, it is desirable for the selected length and area to have a maximum dimension distributed from 80-90% of the screen to a fully corresponding dimension range.

  121 to 128 illustrate a feature of the present invention called zoom click. This feature makes it easier and more accurate for the user in screens used at low resolutions, very small screens, or pointing devices that cannot be accurately placed in relation to the desired screen position. You can select items. This is particularly useful in environments such as mobile phone-sized screens, touch screen devices that use a finger as a pointing device, and / or a moving car where it is difficult to accurately set the pointing device. This is the case when handling touch screen devices.

  In zoom click, when the user clicks down at a predetermined position on the screen, the portion of the screen clicked by the user is displayed at an enlarged scale. The user is then free to navigate with the pressed pointing device in this enlarged display until the pointing device is located at the desired location. At this point, the user can finish pressing the pointing device and release it, so that the current position in the release of the pressing is for the purpose of responding to a conventional graphical user interface (GUI) click. Are handled as the selected position.

  Zoom clicks can be used to express double clicks differently. One of the simplest methods is to simply record a quick second click and release immediately after the zoom click and move closer to the same position as the zoom click so as to convert the zoom click into a double click.

  Other embodiments relating to zoom clicks operate differently, but according to the preferred embodiment, pointer movement in an enlarged view during a down click in a zoom click is a normal pointer in such an enlarged / reduced view. Occurs at the same rate as navigation. This means that, for example, in zoom clicks that double the display resolution, the user has a pointing resolution that is twice that of other cases. In addition, when the user moves the pointer while maintaining the down click in the zoom click mode, and at the same time reaches the edge of the screen or near the edge, the user can navigate the entire page in this mode. Thus, it is desirable that the image scrolls.

  In the example of FIGS. 121-128, a folded phone / computer 120C is shown. In this example, it is assumed that the mobile phone has 320 × 240 full pixel resolution and color subpixel addressability. Of course, other resolutions can be used in other embodiments of the invention. For ease of use, the cell phone is assumed to have a touch-sensitive screen that can be operated with the user's finger. In other embodiments, other types of pointing devices can be used with zoom clicks, such as a touchpad, a pointer with an eraser, a mouse, a navigation button, an ultrasonic visual touch screen pointer position detector.

  FIG. 121 illustrates a mobile phone having the same image as the web page of “priceline.com” illustrated in FIGS. 11 and 110.

  FIG. 122 illustrates what happens when in zoom click mode, the user attempts to press the user's finger 12102 on the screen to select the same text entry field 11000 as discussed with respect to FIG. It is shown. When the user touches his / her finger on the touch screen, it is often difficult to estimate in advance the exact position where the cursor will be placed. The zoom click is useful for this because the user can see the position of the cursor 170 resulting from the finger touching the touch screen before any selection is made. It also illustrates a larger display scale screen that facilitates the user to position the cursor 12204 over a desired link or control, such as the desired text entry field 1100 illustrated in FIG. 123, for example. Yes.

  As shown in FIG. 124, when the user releases his / her finger from the touch screen having the cursor at the position shown in FIG. 123, the same as when the pop-up keyboard appears in FIG. Pop-up keyboard 11102 appears.

  As shown in FIG. 125, in zoom click mode, when the user attempts to touch a desired character (“b” in this example) on the pop-up keyboard 11102, an image of the touched portion of the keyboard Is greatly expanded. In the example illustrated in FIG. 125, when the user first depresses the touch screen, the cursor 12204 is not accurately positioned at the desired position. The user can easily correct this problem in the zoom click mode by dragging the finger 12202 to position the cursor 11102 at the position illustrated in FIG. Thereafter, when the user lifts his / her finger from the screen, the screen image returns to the normal scale. This causes the entire pop-up keyboard 11102 to be displayed, regardless of where the next character is located in the keyboard. Can be selected quickly.

  As shown in FIG. 127, the selected character “b” in FIG. 126 is displayed as if it was entered in the text entry field 11104 of the pop-up keyboard.

  When the user continues to select characters by the processing illustrated in FIGS. 125 to 127, the user can select the “Enter” key and input the entire character string of the text. Thus, as shown in FIG. 128, a desired text is input at a desired position on the web page.

  In many embodiments of the present invention, the zoom used with zoom clicks enlarges a bitmap that was previously displayed in whole or in part on the screen by an integer multiple, such as 2x or 3x. including. Thus, even when using a processor with a relatively low processing capability, such zooming can be performed substantially instantaneously, so zoom clicks provide a very fast user interface.

  FIGS. 129-137 illustrate features of the invention that allow a web browser user to select a portion of text from a web page that will be reflowed or re-layed across line boundaries at substantially greater magnification. ing. Such text reflow is particularly useful on displays with small screens, because text reflow causes the selected web text to be displayed in a larger font and at the same time all lines of such text are This is because it can fit within the screen. This allows you to read such lines quickly without having to repeatedly scroll back and forth horizontally to read those consecutive lines of text.

  Regardless of how high the resolution of a small screen display is, the human eye can only see what it displays at a relatively large resolution unless the display is kept relatively close. I can do it. This feature of the present invention allows web text to be displayed as a lump over a line that fits within the width of the display at a relatively large magnification. For example, it allows a user of a handheld computer with a 4 inch diagonal screen to display text at a scale large enough for a group of people standing 5 or 6 feet apart. Similarly, it allows the user to see text on these displays without holding the cell phone or watch size display close to the face. It can also be used with a normal sized computer display screen to display web text to people who are relatively far away from the screen or to people with poor vision.

  FIG. 129 shows a highly simplified pseudo-code description for a program 12900 that can be used on a client computer to redisplay web text in accordance with this aspect of the invention.

  Of course, the features of the present invention are not limited to use with client computers. In fact, as seen in systems of the type described below with respect to FIGS. 140-141, use this feature of the present invention with modifications when viewing visual output generated by an application other than a web browser. I can do it.

  Many web pages are designed to lay out text in different columns, i.e., lay out text in different horizontal positions in relation to such layout. The web page can show such different desired horizontal permutations in a number of different ways, including the use of tables and frames. In a preferred embodiment, such text reflow is used with a web browser that has the option of displaying the text of a web page in such a multi-column layout to reflect such instructions regarding different desired horizontal substitutions. The If the user selects a region of such a multi-column web page layout for text reflow in a single column at a new scale factor, functions 12904 through 12904 are performed.

  In some embodiments of the present invention, the user drags a pointing device, such as the stylus 11902 illustrated in FIG. 130, beyond the width of the web page that the user wishes to reflow at a larger scale. To make that choice. The currently discussed method allows text to be reflowed across line boundaries, displaying selected columns of text in a larger font, and simultaneously fitting the entire line of text into the screen. Except this is similar to what has already been discussed with respect to FIG.

  Function 12904 in FIG. 129 selects all strings in the current web page layout that are substantially within the selected layout region and corresponding underlined portions (ie, labeling text as links). To do.

  FIG. 131 illustrates the top of the web page layout 10206A illustrated in FIG. 130, similar to the view shown in the lower half of FIG. 131, a broken-line rectangle 13102 represents a portion of the layout of the web page corresponding to the column selected by the user in FIG.

  In some embodiments of the present invention, if a significant portion, eg, two-thirds or three-quarters of its length, fits within the region selected by the user, the string is It is considered to exist within the selected area. For example, in FIG. 130, the user was trying to select text in the right portion of the screen displayed in the figure. However, in the example of FIG. 130, the user cannot accurately select the intended column width by dragging the stylus. Nevertheless, function 12904 selects substantially all the strings that exist in the selected region, so text reflow is performed as if the user had correctly selected the intended column. .

  FIG. 132 illustrates the first part of the string in the layout of the web page illustrated in FIG. 131 that is contained within the selected region. In this figure, the underlined portion indicates the portion of text corresponding to the link.

  When all of the strings in the selected region have been selected, function 12906 may indicate that one or more of them are part of the same paragraph, due to accessibility or other characteristics in the layout. Label any group within a string of characters. This is illustrated in FIG. 132 by paragraph brackets (parentheses marked with P) 13202.

  As illustrated in FIG. 132, this method may not detect all groupings of characters that are paragraphs, but detects many of them without having to reference the HTML corresponding to the text. . In the described embodiment of the invention, such HTML is stored on a proxy server, which requires that such access requires a delay associated with communication between the client computer and the proxy server. Means. In other embodiments, particularly where the client computer resides in a full browser or in an embodiment with a high access line capacity connection to a proxy computer, how the selected string is Access to HTML code can be used to more accurately determine what should be grouped in a paragraph. In other embodiments, the layout information downloaded to the proxy server can include any information regarding paragraph boundaries contained within the HTML of the web page. In some embodiments, such text reflow can be performed on a proxy server rather than on a thin client that displays such reflowed text so that the proxy server can display a web page. You can directly access any information about the grouping of paragraphs contained in the.

  When the selected strings are grouped into paragraphs, the function 12908 goes beyond the width of the display screen (or display window if the image is displayed in a smaller area than the entire display screen). Reflow the text of each paragram using the selected magnification.

  This text reflow process is illustrated in FIG. 133 in which the character string at the top of FIG. 132 is laid out at a new magnification. In the example shown, the text of FIG. 130 has been selected to reflow at twice the original size within the same screen. The thin client user interface preferably allows the user to select a plurality of different magnifications used with the selected text reflow function, ranging from moderate font sizes to extremely large font sizes.

  In FIG. 133, the underlined portion is used to display something different from the underlined portion of FIG. In FIG. 133, the text on each line resulting from the normal layout string in FIG. 132 is illustrated with a continuous underline. The spacing of the underlined portion between the text portions on the same line in FIG. 133 resulting from the different character strings in FIG. 132 is exaggerated so that the differences can be seen more easily. .

  In FIG. 133, all of the individual strings resulting from the original layout illustrated in FIG. 132 that are clustered across line boundaries are continued from the portion on one line to the next line. This is indicated by the arrows.

  FIG. 134 is a schematic diagram showing how the selected string of the original layout illustrated in FIGS. 130 and 131 looks when it is reflowed on the thin client screen at approximately twice the size. It is. As can be seen with reference to FIG. 134, such text reflow makes the web text much easier to see from a distance. By reflowing the same text at 4 times or 6 times instead of 2 times, it becomes possible to display the same web content to a user located far away from the display screen.

  FIGS. 135-137 illustrate other ways in which the user can select the portion of text to be reflowed.

  FIG. 135 illustrates a portion of a web page having a central column of text into which another portion of text or multiple portions intrude.

  FIG. 136 illustrates how the user obtained a reduced view of the layout of the entire web page. In many practical embodiments, text greeking is used to indicate portions of text that are too small to be displayed in such a reduced view as individual characters. Such reduced views can be quickly generated on a thin client computer, as described above with respect to FIGS. 115-117, where the entire web page layout is stored on the thin client itself.

  In FIG. 136, the user has selected a mode in which the user can define such an area on the enlarged web page view by clicking on the display of the web page at a corner in the polygonal area. When this is done, the selected region is used by function 12904 illustrated in FIG. 129 to select which text is reflowed.

  FIG. 137 illustrates how the selected text will appear when the selected text is reflowed and displayed.

  FIGS. 138 through 139 describe the font server 230 described above with respect to FIG. 2 in more detail.

  In FIG. 138, there are a plurality of client browsers 200, and each client browser accesses content from one or a plurality of servers 220 via the same proxy server 210, and fonts from the same font server 230 are displayed. FIG. 138 corresponds to FIG.

  This is the same proxy server where the software sold, licensed or distributed for use with each thin client browser is programmed to retrieve fonts that the client does not have from the same font server 230 This is because it is programmed to execute the web request via 210. Of course, in other embodiments relating to this feature of the present invention, thin clients may use any of a number of common proxy servers based on their geographical location or factors such as Internet service providers. Programmed to choose which proxy server to use. In order to select any one of a plurality of general font servers that request and receive fonts, the thin client can consider the same contents.

  FIG. 139 is a highly simplified pseudo code description of a program 13900 that can be used on a font server of the type illustrated in FIG. In addition to a normal browser computer, this font server can be used to supply fonts to a computer that executes an application other than a web browser.

  When the font server receives an HTTP request for a character of the specified font from the computer, steps 13904 to 13922 are executed by the function 13902.

  The particular embodiment of the font server code illustrated in FIG. 139 is designed for use with a protocol that identifies each character designed for a specified font at a specified size in a separate HTTP request. Yes. It specifies the desired font, font size, and characters as part of the URL path name. Of course, in other embodiments, an HTTP request can specify multiple characters, multiple fonts, and / or a font other than a URL pathname, by a font server. .

  In systems that require each character font shape separately, the HTTP protocol 1.1 or later protocol is preferably used, which opens a separate connection to handle each such request, Alternatively, it is possible to process a plurality of HTTP requests from a predetermined client computer by a server by using such a protocol without the need for closing.

  In the embodiment of the present invention illustrated in FIG. 139, if the font server determines that it currently stores a font bitmap that matches the URL pathname specified in the request, then by function 13904, Function 13906 sends the font in the HTTP response to the network address for which the URL request was made, and then function 13908 directs the associated account to process. The downloaded font may be either a font bitmap or a font outline description.

  Such account instructions are not used in all embodiments of the invention. In some of the embodiments in which it is used, the commanded account is the account associated with the computer to which the font is sent. In other embodiments, the instructions are made to a group account associated with a web page that includes such a font specification. In yet another embodiment, the instructions are made to an account associated with a proxy server of the type described above, or to a user of such a proxy server service.

  If the requested font does not exist in the font server storage and the requested font is a bitmap with the font server's corresponding outline font, functions 13912 through 13922 are executed by function 13910.

  The function 13912 generates a bitmap having attributes such as the size and executable conversion indicated by the font path name of the HTTP request. This function includes determining whether the pathname of the requested font indicated that a subpixel optimized version of the font was desired. Where indicated, the functions 13914 and 13916 preferably generate a sub-pixel optimized version of the font using the non-linear color balance method described above with respect to FIGS.

  When the font bitmap is generated, function 13918 sends the bitmap in the HTTP response to the requesting address over the network. Function 13920 caches the font bitmap at the address corresponding to the path name specified in the request. Function 13922 commands the account associated with the process in the embodiment in which the command is executed, as previously discussed with respect to function 13910.

  FIG. 140 illustrates the present invention so that thin client computer 200 can display digital content corresponding to text and images generated as screen output by one or more applications running on remote computer 1400. It illustrates that certain features of can be used. Such an application may be a web browser, spreadsheet, word processor, database program, or any type of software capable of generating a virtual screen display.

  The remote computer includes a remote screen generation program 14006 that includes hooks in the dispatch table 14008 of the remote computer operating system 14004. These hooks block calls to the operating system executed by one or more applications 14002 to draw text, shapes, lines, control objects, and bitmaps on the screen at a predetermined display resolution. In some embodiments, such a drawing command causes the content to actually be displayed on the screen associated with the remote computer, while in other embodiments the remote computer does not have a screen, so A drawing command is executed on the virtual screen. In the text below, for the sake of simplicity, reference is made to the video space that these applications determine as receiving a user input and displaying a graphic output as a virtual screen on a given client computer.

  When one of the applications 14002 requests the operating system to draw a display element, one of the hooks in the operating system dispatch table is used to make a corresponding call to the corresponding routine 14010 of the remote screen generator. The call is blocked by one. In a manner similar to that described above with respect to FIGS. 102 and 106A-106C, this produces a download display list 10212A that is substantially similar to the display list 10212 described above with respect to FIG. 102 and subsequent figures. The In many embodiments for such systems, a bitmap rendering routine call generates a reduced and sub-pixel optimized image of the type described above for download and provides the measurement string routine and string rendering routine. Calls include sub-pixel optimized fonts or other anti-aliased font substitutions and font matrices, including fonts as small as 8 pixels / em or less.

  The zoom, scroll, and virtual layout controls 1412 corresponding to the controls 10214 to 10218 illustrated in FIG. 102 control the mapping of the thin client's view window to the virtual screen and are therefore rendered on the virtual screen by the application. The display magnification that is drawn and arranged in the download display list 10212A is controlled. This preferably includes sub-pixel optimization of the image bitmap and font substitution of the type described above with respect to FIGS. 106A-106C.

  When a download display list is generated for a given virtual screen, download to a corresponding client computer that will later draw the download display list on that screen in a manner similar to that described above with respect to FIGS. 109A-109C. The display list is compressed and downloaded.

  In some embodiments of the invention, each drawing on the virtual screen has a corresponding drawing command downloaded to the thin client. This can be used to increase the speed at which minor changes to the thin client screen can be performed in response to corresponding changes to the virtual screen.

  In the embodiment illustrated in FIG. 140, user input associated with a screen location is uploaded from the thin client to the remote computer, and the screen location reflects the mapping between the thin client's view window and the virtual screen. To convert the screen coordinates. At the time this is done, such an event. These transformed screen coordinates are used to set the event queue of the remote computer operating system, so that the associated application 14002 will have its input as if it were entered on the corresponding virtual screen of the remote computer. Corresponds to the event.

  For example, LapLink (location: ZIP code 98011, Washington, Washington, USA, North Creek Parkway 18912 Suite 100), LapLink, or Symantec (location: ZIP code 95014, Cupertino Steven Creek Boulevard, California, USA 20330) pcAnyware, etc. Many of the techniques used by screen sharing applications can be used with embodiments of the present invention of the type illustrated in FIG. Indeed, if the remote computer in the figure has its own screen, the embodiment illustrated in FIG. 140 can be used to perform screen sharing between the client computer and the remote computer.

  Of course, in embodiments where the client computer has sufficient computing power, the client and the remote computer can operate in a peer-to-peer manner. The remote computer may be a dedicated application server computer or any type of computer such as a desktop computer, a laptop computer, or a personal computer including a tablet computer.

  FIG. 141 illustrates an operating system generated by one or more applications 14002 such that screen displays generated by such applications are reduced and / or sub-pixel optimized in accordance with features of the invention described above. 140 illustrates an embodiment of the present invention that is somewhat similar to the embodiment illustrated in FIG. 140 in that it uses a hook in the dispatch table 14008 of the computer operating system 14004 to block calls to ing. The embodiment of FIG. 141 differs from the client server embodiment illustrated in FIG. 140 in that it is designed to operate on a single computer system 14100, as illustrated in FIG. It is. A low-resolution screen, but with enough power to launch a desktop application to run such programs without modification, as if their output were displayed on a virtual screen with high resolution You can use the programming of this system on a computer. It can also be used to draw a reduced version of the application's output screen on a portion of a larger screen.

  In this embodiment, the application 14002 is reported that their outputs are operating on the screen at a higher virtual resolution than the resolution displayed on the screen 10220A. When such an application makes a call to the operating system to render an element on the screen, a portion 14010A of the subpixel optimized screen generator program 14006A scaled by hooks set in the OS dispatch table 14008. The corresponding drawing routine in is started. This replacement rendering routine renders elements corresponding to the virtual screen display list 10206B, including the creation of reduced and / or sub-pixel optimized bitmaps as described above. It also allows any part of those screen elements drawn on the part of the virtual screen that fits within the current view window 10210C to call the drawing command in the operating system, or to place them directly on that screen. By drawing, it is immediately displayed on the display screen 10220A of the computer 14100 in a reduced and / or sub-pixel optimized manner.

  When an application program invokes the operating system for a measurement string command, that command is similarly blocked, so that the call is replaced with a font that has been replaced in the manner described above with respect to functions 10608 through 10618 in FIG. 106A. Returns the font matrix for the size.

  Screen event input to the computer screen is extracted from the operating system event queue, and the screen coordinates at which such events are generated on the screen map the view window to that virtual screen to control such transformations. Is sent to the event position scaler which translates to the corresponding position in the virtual screen layout represented by the display list. When the event coordinates are properly transformed, the event is returned to the operating system event queue, and the application responds to the event as if it had been entered on the virtual screen by the operating system.

  According to embodiments of the present invention of the type illustrated in FIG. 141, computer users can subpixel optimize selected text reflows on a screen generated by a standard computer application 14002, These functions can be performed even if not designed to support scaling, zooming, and performing functions.

  In another embodiment of the present invention not shown, the computer operating system is modified to include the type of functionality illustrated in the scaled subpixel optimized screen generator 14006A illustrated in FIG. I can do it. Furthermore, in other embodiments of the present invention, the application program 14002 including the browser program can be modified to directly support all or part of such functionality.

  FIG. 142 illustrates that a thin client computer such as, for example, the thin client computers 200A-200D illustrated in FIG. 142 can be used to access Internet content or application programs over a wireless network. Figures 102 and 140 illustrate how the embodiments of the invention illustrated in Figures 102 and 140 can be used.

  In this figure, computers 200A to 200D correspond to the thin client computer 200 shown in FIGS. The computer 200A is a handheld computer. The thin client computer 200B is a mobile phone. The thin client computer 200C is a wristwatch type computer. The thin client computer 200D is a head mounted computer or a head mounted display for a portable computer. Each of these thin client computers can have a subpixel addressable display.

  At the time of filing of the present application, it is now possible to produce a screen for each type of device that has a sufficiently high resolution for use with most features of the present invention. For example, it is currently possible to produce a 320 × 240 color LCD display having a diagonal length of 2 inches or less. At present, it is possible to manufacture organic LED devices having higher resolution. In the near future, the cost of such small screens should decrease, and their availability and resolution should increase.

  All of the thin client computers illustrated in FIG. 142 receive the type of information described above with respect to the remote proxy server computer 210 of the type illustrated in FIG. 102 or the remote application server 14000 of the type illustrated with respect to FIG. It has a wireless transceiver that enables transmission and reception. Such a transceiver may be a wireless LAN transceiver for communicating with a wireless LAN transceiver 14204, a digital cell phone wireless transceiver for communicating with a wireless internet transceiver 14202, and both types of wireless transceivers. It is desirable for the transceiver to be designed to communicate with. In other embodiments, other types of wireless communications can be used, such as, for example, Bluetooth or infrared communications.

  The remote computers 14000AA to 14000AC illustrated in FIG. 142 correspond to the remote server computer 14000 illustrated in FIG.

  The remote application server computer 14000AA illustrated in FIG. 142 represents a laptop computer, desktop computer, server computer, or other type of computer that can be programmed to operate as a remote application server computer 14000. The subpixel-optimized application server 14000AB is designed to run applications for multiple thin client computers connected to a LAN or WAN associated with such a client, as shown in FIG. A kind of remote computer. Remote computers 14000AA and 14000AB can communicate with the thin client via private local area wireless transmitter 14204, or can communicate with the thin client via the wireless Internet, as indicated by numerals 10222 and 14202. It is.

  The subpixel optimized application server 14000AC connects directly to the Internet so that multiple thin client computers 200 can use the application over the Internet by the wireless transmission network indicated by numeral 14202. This is an application server similar to the server 14000AB except for the above.

  In FIG. 142, a proxy server 210 of the type described above with respect to FIG. 102 is shown connected to a LAN or WAN 14204. For example, this may be a proxy server that attempts to handle web browsing that the company wishes to not connect to the Internet. Of course, other proxy servers, such as, for example, a proxy server operated by a company that provides commercial proxy provisioning services, are typically directly connected to the Internet 10222 also illustrated in FIG. Connected to.

  The system illustrated in FIG. 142 allows a small computer that is practically always convenient to carry to access and display web pages and the output of most application programs. At the time of filing this application, the line capacity of a relatively inexpensive wireless LAN transceiver, such as, for example, the LAN transceiver 14204 illustrated in FIG. 142, allows such digital content to be viewed on a desk computer connected to a cable modem. The thin client of the type illustrated in FIG. 142 is fast enough to see the output of the web content or application program at a rate similar to that possible. And this is a machine that can be carried in a pocket, on the wrist, or as part of eyeglasses, and is a mechanical device that can access such media within seconds after the switch is turned on.

  Downloading the web page image usually takes a few seconds to download the entire text of most web pages on the digital cellular phone capacity generally available in the United States at the time of filing this application It takes more time. Of course, many embodiments of the present invention begin displaying text as soon as a portion of the text is received, allowing the user to begin viewing a portion of the downloaded page very quickly.

At the time of this application, a new, higher-speed digital cellular phone system capable of providing line capacity in the range of hundreds of thousands of bits / second or millions of bits / second has been developed. When such higher speed systems are commonly deployed, users of the present invention may access web pages on a desktop or laptop via a DSL or cable modem connection. Read and interact with web pages and application screens on a small portable device that can be switched on within seconds in most places where the user moves, with almost the same speed and comfort I can do it.
The user of the present invention is on a small portable device that can be used within a few seconds of switch-on in most places of travel, as if accessing on a desktop or laptop computer via DSL or cable modem Now you can read web pages and application screens and interact with each other.

  FIGS. 143-144 illustrate a handheld computer 200A that can function as a thin client to either a proxy server of the type described above with respect to FIG. 102 or a remote application server computer of the type described above with respect to FIG. Figure 2 shows two diagrams.

  In FIG. 143, the computer is shown in a vertical orientation designed for use. The native operating system on a computer is designed to render fonts and graphical user interface (GUI) elements vertically. This is the way many of the handheld computers sold at the time of filing this application were designed and built. For example, there are a number of such handheld computers on the market today that have sub-pixel addressable screens with 240 × 320 full pixel resolution. Many of these computers also have sub-pixel stripes that run horizontally when the display is positioned in the intended vertical direction.

  Unfortunately, in such a portrait orientation, most people are accustomed to using computers and do not provide the kind of aspect ratio that most web pages are designed for. Furthermore, if such a computer has horizontal subpixel stripes, such stripes provide all potential increases in subpixel resolution in the vertical direction. Unfortunately, the display of text tends to benefit substantially from an increase in horizontal resolution compared to an increase in such vertical resolution.

  For all these reasons, many embodiments of the present invention that use such vertical machines use them when they are rotated 90 degrees, as illustrated in FIG. Have a horizontal aspect ratio that is rather close to the aspect ratio for most computer screen layouts, so that their sub-pixels are most useful for displaying text Provides an increase in horizontal resolution. In many such embodiments, the thin client 200 described above is rotated to be viewed as a horizontal machine and a vertical personal digital assistant (PDA) in which reduced fonts and image bitmaps are drawn in the horizontal direction. ). In such embodiments that use sub-pixel optimization, rather than the orthogonal arrangement of sub-pixels that appears when pixels are viewed in the vertical direction designed to use such PDAs, such pixels are viewed in the horizontal direction. The web page is displayed using a sub-pixel optimized image and / or font bitmap for the spatial arrangement of sub-pixels within each pixel that appears.

  FIG. 145 illustrates some of the features of this embodiment to meet the requirement to draw basic shapes such as rectangles, ellipses, lines, and curves using subpixel optimization. It is a highly simplified pseudo code notation that shows how it can be used. Such functionality can be used in many different types of applications, operating systems, and thin client software.

  In the example of FIG. 145, the illustrated pseudo code is associated with a rectangle drawing function 14500 that can be used in place of the rectangle command 10918 described above with respect to FIG. 109A, among other uses. Such routines can be called by a call to draw a rectangle whose position, width, and / or height is defined at a higher resolution compared to the total pixel resolution of the displayed subpixel addressable screen. Called. Correspondingly, function 14502 uses a subpixel optimization routine at subpixel resolution to render a rectangular image defined at such higher resolution. For example, as described above, in many cases, a two-color optimization scheme with non-linear color filtering, except for a monochrome rectangle that tends to provide the highest viewable spatial resolution, can be any arbitrary subpixel. This can be done using an optimization scheme.

  FIG. 146 illustrates code 14602 that can be run on a server and / or proxy computer to allow sub-pixel optimized screen elements to be rendered on the client's screen by an applet downloaded from the server, and a thin client. A highly simplified pseudocode notation 14600 for code 14604 that can be run on a client computer, including a computer.

  In such an embodiment, the client function 14606 requests media from the server. By downloading media or data containing one or more applet programs that can be run on the client computer, the server responds with function 14608. In function 14610, the client computer receives the media containing the applet, and function 14612 loads and executes the applet. In function 14614, the subpixel optimized elements are rendered on the subpixel addressable screen on the client computer.

  Duplicate or generate subpixel-optimized bitmaps, render text in subpixel-optimized fonts, eg vector-defined graphics, or relatively, eg lines, rectangles and ellipses By drawing a sub-pixel optimized shape, such as a simple geometric shape, an applet can draw a sub-pixel optimized element.

  FIGS. 147 and 148 illustrate how sub-pixel optimization can be applied to rollover images and GIFF animations, respectively.

  In the subpixel optimization routine 14700 illustrated in FIG. 147, a non-rollover image 14702 that is displayed when the pointing device is not present in the portion of the screen associated with the image, so that the pointing device can be detected, As can be detected, both rollover images 14704 displayed when present on the screen portion are reduced and sub-pixel optimized by function 14706. Thus, a non-rollover image 14708 that has been enlarged / reduced and sub-pixel optimized and a rollover image 14710 that has been enlarged / reduced and sub-pixel optimized are generated. The function 14712 then displays which of these two subpixel optimized images to display based on whether the pointer is detectably present on their associated screen area. Used to select. This allows the two sub-pixel optimized images to function as a composite “rollover” graphic.

  In other embodiments relating to this aspect of the invention, a similar technique is used for two images associated with a button: an image that is displayed when the button is not pressed, and a button that is pressed. It can be applied to the image displayed in the case.

  The method 14800 illustrated in FIG. 148 is similar to the method described above with respect to FIG. It takes each of the separate images 14802 to 148906 of the GIFF animation and optimizes the subpixels in function 14808 to produce a corresponding reduced and subpixel optimized GIFF animation set. Next, function 14816 displays the sub-pixel optimized image in a manner that is substantially the same as the way the non-sub-pixel optimized GIFF animation is displayed.

  Using the sub-pixel optimization described above with respect to FIGS. 147 and 148 with other features relating to the above-described invention, including access to web pages on sub-pixel addressable screens, including screens on thin client computers. I can do it.

  FIG. 149 illustrates a method 14900 for subpixel optimization of a 3D animation. The method includes performing a set of functions 14904-14908 for each successive frame of animation.

  Function 14904 executes the 3D animation engine to create a bitmap of the current frame or a bitmap of at least those portions of the image that have changed since the last frame. This function produces such a bitmap at a higher resolution than the full pixel resolution at which a sub-pixel optimized version of such bitmap is displayed.

  The function 14906 then uses a technique such as the technique described above to reduce and sub-pixel optimize the changes made in the frame bitmap, or at least the frame bitmap from the last frame. .

  Next, function 14908 can subpixel address the reduced, subpixel optimized image of the frame bitmap, or the reduced, subpixel optimized image of at least the modified portion of the frame. Display on the screen.

  To allow the user to run games and view the images generated by such games at a higher resolution enabled by sub-pixel optimization, it is illustrated in FIG. The method is particularly useful. In a handheld device with a small screen, such a purpose can be used in the method. The method can be used for both client computers that display animated images generated on remote computers and computers that generate such animations locally.

  FIGS. 150 and 151 illustrate one method in which the method of FIG. 149 can be used in a client server game application.

  FIG. 150 illustrates a program 15000 on a game server computer used in such an embodiment. As indicated by the numbers 15002 and 15004, when the game server receives user input from one or more game client computers, it sends the input to the game engine. If such input is a screen input, the input is appropriately scaled to compensate for the difference between the user's screen resolution and the space that the game engine associates with the screen input.

  In function 15006, the game engine computer calculates a display list for the current frame or any changes associated with the current frame relative to the previous display list. The function 15008 then causes the 3D rendering program to render a frame bitmap corresponding to the display list generated for the current frame and render the necessary changes to the current frame bitmap. Such a bitmap is generated at a higher resolution than the resolution of the subpixel optimized image generated by function 15010.

  If the client generates different screen images for different clients, the function 15008 is performed separately for each of these separate views.

  Next, function 15010 reduces and subpixel optimizes the bitmap of the current frame or the bitmap of the current change to the frame. Also, if this function only reduces the bitmap of such changes, it reduces the screen position associated with those changes.

  Next, function 15012 compresses the subpixel-optimized bitmaps and, where appropriate, compresses their positions, and function 15014 compresses, scales, and subpixel optimizes. Downloaded images and any such locations for display to the client.

  FIG. 151 illustrates a program 15100 on a game client designed for use with the program of FIG.

  Function 15101 receives the downloaded images and function 15102 decompresses them. Next, function 15104 displays the scaled, subpixel-optimized animation frame bitmap, or displays a bitmap of changes to the previous animation screen image at the position applied to those changes. To do. This is done in a subpixel addressable display.

  As indicated by the numbers 15106 and 15108, when the client receives user input, it uploads the input to the game server so that any screen coordinates associated with such input are properly translated.

  In other embodiments relating to this aspect of the invention, the distribution of functionality between the game server and the game client may be different. In some embodiments, a proxy server generally similar to the proxy server described above is sub-pixel optimized to display game content originally generated on a game server different from the proxy server on the thin client. Can be used to implement In yet another embodiment, the game client itself can perform subpixel optimization.

  FIG. 152 is a highly simplified illustration of features of the present invention that allow an image with an associated transparency map to be displayed with sub-pixel optimization of both such images and the associated transparency map. Pseudo code description.

  The program 15200 illustrated in FIG. 152 scales sub-pixel optimization for foreground images, ie, images controlled by a transparent bitmap associated with the display of the top of the background or other previous bitmap. A function 15202 for generating a generated bitmap. The sub-pixel optimization used may be either a two-color sub-pixel optimization or a multi-color sub-pixel optimization, or a combination of the two. Any method known to generate a sub-pixel optimized representation of an image can be used, including the methods described above.

  Function 15204 performs sub-pixel optimization of the associated transparency map of the image. It is desirable to use two-color sub-pixel optimization, which is a high-resolution source image of a transparency map, with alpha values ranging from 0 to 1 and varying along a straight line in a three-component color space. It is because it has. Such source image alpha values correspond to grayscale colors, which means that the area of the transparent map source image corresponding to any pixel in the sub-pixel optimized output image of the map has a constant transparency value. This is because when covered, all of the subpixels of the output pixel tend to have the same alpha value. It is desirable to perform a two-color subpixel optimization of the transparency map using the nonlinear color balance described above.

  When such subpixel optimization is performed on the foreground image and its associated transparency map, function 15206 displays this combination in the subpixel optimization display. This process includes performing a loop 15208 for each pixel column, including a loop 15210 for each subpixel of each pixel column of the displayed image. Function 15210 causes functions 15212 and 15214 to be performed for each subpixel. Function 15212 sets the current alpha value to the alpha value of the corresponding subpixel of the subpixel optimized transparency map. Next, the function 15214 calculates the current sub-pixel brightness by multiplying the “current alpha value” by the “sub-pixel optimized foreground image corresponding sub-pixel brightness value”. The value obtained by multiplying “the value obtained by subtracting the alpha value of” by “the previous luminance value of the current subpixel in the background bitmap on which the transparent image is drawn” is added.

  This is because when the foreground image is rendered on the previous bitmap, the respective luminance value of each individual subpixel is derived from the corresponding subpixel value of the foreground image or the corresponding subpixel value of the previous bitmap. It means that the calculated degree is specified as a function of the corresponding subpixel alpha value of the subpixel optimized transparency map.

  In some embodiments of the present invention, the image associated with the transparency map is scaled and subpixel optimized on the server or browser computer, downloaded, and displayed by function 15206 on the client computer. Is done. In other embodiments of the present invention, such subpixel optimized transparency images are made available on recorded digital media. In yet another embodiment of the invention, the images are generated by the same computer that displays them.

  In other embodiments of the invention, alpha values contained in non-subpixel optimized transparency maps can be used to display a subpixel optimized foreground image.

  In some embodiments of the invention, lossy color compression is used to represent a group of colors that are perceptually close to one color. One-dimensional transparency value, a three-dimensional transparency (ie, opacity or alpha) value of the type described above with respect to FIGS. 70, 96 and 97, or a transparent component value as a special color dimension, similar to an RGB component value Such compression can be performed on color values. In such compression, avoiding transparency values or component color values that exhibit an alpha value of 1 or 0, or a value very close to 1 or 0, with transparency values that are far from 1 or 0, respectively. Generally desirable. This is because the eye is more sensitive to slight changes in opacity at both extremes of the transparency range compared to those changes in other parts of the transparency range.

  For all purposes where non-subpixel optimized images are used for transparency maps, subpixel optimized images with transparency maps can be used on subpixel optimized displays. This includes use in animation and web page layout.

  FIGS. 153 through 162 are highly simplified pseudocode descriptions relating to features of the present invention relating to subpixel optimization of video and / or animation. Such sub-pixel optimization can be used in web browsing content as well as other virtual content where video and animation are used.

  FIG. 153 shows a program 15300 that is used to sub-pixel optimize a displayed video using interpolation between video key frames. The program includes a function 15302 that is used when the video to be subpixel optimized is received in a compressed format. It can decompress such images and optimize sub-pixels.

  Function 15304 reduces the keyframe of the video and optimizes the subpixel. Function 15306 reduces interpolated changes between key frames, but does not perform subpixel optimization. In some embodiments relating to this aspect of the invention illustrated in FIG. 153, such interpolated changes can be sub-pixel optimized, but there is little benefit from doing so. This is because such changes appear on the screen too quickly, so their subpixel optimizations are not recognized and reduce the computational burden by avoiding those subpixel optimizations.

  Function 15308 displays the reduced video on a subpixel addressable display using interpolation between subpixel optimized key frames and non-subpixel optimized frames.

  In other embodiments of the present invention, this concept of simply sub-pixel optimizing the portion of the image present on the screen at one location long enough to be clearly recognized is used in other ways. I can do it.

  FIG. 154 illustrates a program that can be used to subpixel-optimize a video that is represented in whole or in part by a sequence of part-full-frame image elements drawn against a display frame. Show. In general, such images also include full-frame images and, if necessary, part-to-full to gradually change the screen to represent the movement of one or more objects present therein. Use a frame drawing sequence. This includes the type of animation described above with respect to FIG. It can also include various methods of video compression, including video with the general types of key frames and interpolation between frames described above with respect to FIG.

  The program of FIG. 154 includes a function 15402 that is used when a sub-pixel optimized video is received in a compressed format, where the function decompresses the video. Next, the function 15404 enlarges / reduces any frame images included in the video, optimizes the sub-pixels, and reduces them by the display magnification. Next, function 15406 scales any modified bitmaps, subpixel optimizes, and scales those image sizes and their positions by that factor.

  Functions 15407 and 15408 repeatedly display any video frame that has been scaled and subpixel optimized in the video sequence on a subpixel addressable screen. After display of such a video frame, function 15406 allows any one or more scaled subpixel optimizations for the bitmap of the frame at the scaled position associated with the modified bitmap. Display the modified bitmap.

  It can be seen that the method of FIG. 154 makes it possible to draw a video and animation optimized for subpixels in a method that reduces the amount of computation required for subpixel optimization. In the method of FIG. This is because every time a change is made to the video image, sub-pixel optimization of the entire frame is not required.

  FIGS. 155 and 156 illustrate two different ways in which a sub-pixel optimized image that moves relative to the frame can be displayed.

  FIG. 155 includes a program 15500 that displays an image with a fixed sub-pixilation so that the image moves in increments of all pixels on a sub-pixel addressable display in conjunction with a larger image. It includes a function 15502 that saves a subpixel optimized image that can be generated in any manner, including the methods described above. It includes a loop 15503 that is executed for each successive frame time. This loop consists of functions 15504 and 15506. Function 15504 calculates motion relative to the image in relation to the larger image. In this movement calculation, the position calculated for the object in each display frame is approximated to the nearest horizontal and vertical pixel positions, and the image size and direction are not changed. Function 15506 displays the image with function 15504 at the full pixel resolution position calculated for it. This method is computationally very efficient because only one subpixel-optimized bitmap of the image has to be computed, and that single image is used repeatedly across the screen. is there.

  FIG. 156 illustrates a program 15600 that displays a moving image by changing sub-pixilation. It includes a function 15602 that saves a high resolution source image of the image being moved. It also includes a loop 15603 that is executed for each successive frame time. This loop includes a function 15604 that calculates the current transformation, rotation, and / or transformation of the high resolution source image, if any, for the current frame. The loop function 15606 then generates a reduced and sub-pixel optimized bitmap of the resulting transformed, rotated, and / or transformed bitmap. This subpixel optimization takes into account the location of this transformed bitmap in relation to the subpixel array in which it is displayed at a higher resolution than the full pixel resolution. The frame loop function 15608 then displays the resulting subpixel optimized bitmap on the subpixel addressable display.

  In addition to fairies in game animation, use any of the methods described above with respect to FIG. 155 or 156 to display animated characters or other visual representations that are moved in relation to larger frames. I can do it.

  The method of FIG. 155 tends to provide a less accurate representation of the movement of visual objects, but is more computationally efficient. The method of FIG. 156 provides a more accurate visual representation, but is computationally expensive.

  In some embodiments of the invention, a combination of these two methods can be used. For example, a small subset of possible mappings between an object and a subpixel array can be stored, and when an object moves, a higher resolution with respect to its current location relative to the subpixel array on which it is displayed It is displayed in such a stored mapping way that represents the expression in detail.

  FIGS. 157 and / or 158 are used to optimize the display of DVD or HDTV video by reducing the DVD or HDTV video and subpixel optimization for display on a subpixel addressable screen. The features of the present invention are illustrated. This is particularly useful when used in conjunction with subpixel addressable screens that have a higher subpixel resolution in the horizontal direction than in the vertical direction, which is generally the case for DVD and HDTV video. This is because the aspect ratio is substantially wide rather than high.

  FIG. 159 illustrates features of the present invention that can be applied to video formats representing sub-components of video images as separate objects with different attributes. A special example in FIG. 159 includes a program 15900 for sub-pixel optimization of the display of MPEG4 video.

  The program illustrated in FIG. 159 includes a function 15902 for receiving and decompressing MPEG4 video. It includes functions 15904 and 15906 that use different subpixel optimization methods in reducing different types of objects in MPEG4 video. This function uses two-color subpixel optimization for two-color objects and multi-color subpixel optimization for multi-color objects, preferably using non-linear color balance. Function 15908 displays a combination of two-color and multi-color objects on the subpixel optimization screen and moves those subpixel optimization objects relative to the screen as indicated by the MPEG4 description. And use a method of the type already discussed with respect to FIGS. 155 and / or 156.

  Some features of the present invention are not limited to using different subpixel optimization algorithms for different object types in the MPEG4 data stream. However, by using such a different subpixel optimization algorithm, it is possible to provide a higher perceptual resolution for a two-color object, eg text, thereby providing a somewhat improved image. It has advantages.

  160 and 161 relate to a system in which a user accesses sub-pixel optimized video on a computer network.

  FIG. 160 illustrates a program 16000 used by a server computer to provide sub-pixel optimized and reduced video. Such a server may be a proxy server that accesses the video requested by the client from another server computer and then scales it down and subpixel optimizes it before downloading to the client. In other embodiments, the provision of sub-pixel optimized video is performed without going through such an intermediary proxy server.

  The program of FIG. 160 includes a function 16002 for receiving a request for a specific video from a client computer. In many embodiments, such as the embodiment illustrated in FIG. 160, this requirement also describes the horizontal and vertical subpixel resolution at which the video is subpixel optimized. In embodiments that only provide a set of clients with one fixed subpixel resolution, such information is not required as part of the request.

  Function 16004 receives the requested video content. As described above, accessing video content from a remote server, accessing video content from RAM or mass storage associated with a running computer, dynamically generating such content, or some This can be done by selecting video input from any source.

  Function 16006 reduces the received video to the subpixel resolution associated with the request of function 16002 and optimizes the subpixel. Next, function 16008 compresses the subpixel optimized video, and function 16010 downloads the compressed video to the requesting device.

  The compression algorithm used for such sub-pixel optimized images is such that the position of the color value associated with any pixel in such a sub-pixel optimized image is at a distance that is greater than or equal to the relatively limited color distance. Unless moved in the RGB color space, it is possible to have a compression algorithm with some loss without substantially reducing the increased spatial resolution possible with subpixel optimization.

  FIG. 161 depicts a system 16100 that can be used with the features of the present invention described in FIG. The system has proxy computer code 16100 and thin client computer code 16112, both of which are illustrated in FIG. 161 by highly simplified pseudo code.

  If the thin client receives a user request for a given video, function 16113 responds by sending a request for the video that includes the sub-pixel resolution at which the video is displayed on the proxy. When the proxy receives a request for such a video, according to its function 16100, the function 16103 sends a corresponding request for the video to a server capable of obtaining the video. In many embodiments, this is the server identified in the URL of such a request.

  If the requested video is received by the proxy server, function 16104 causes functions 16106 through 16110 to be performed. Function 16106 reduces the video to the subpixel resolution planned for the request from the client, optimizes the subpixel, function 16108 compresses the subpixel optimized video, and function 16110 Download it to the client that requested it.

  When the client receives the requested video from the proxy, function 16114 causes function 16115 to decompress the video and function 16116 displays the scaled and decompressed video on a subpixel addressable display.

  FIGS. 162-166 illustrate how the features of the present invention can be used to improve the appearance of digital ink. Digital ink is a black and white bitmap that is usually drawn on a screen in response to a user writing a character or drawing a graphic with a pointing device. Traditionally, digital ink bitmaps are typically represented at full pixel resolution where each pixel is displayed as black, white, or as a grayscale value on some devices.

  One feature of the present invention is the use of sub-pixel optimization to represent digital ink at higher resolution. When digital ink is represented in computer memory as points and lines, or curves between such points, the resulting mathematical description of the lines between such points is the total pixel resolution of the screen. In comparison, it can have a much higher resolution.

  FIG. 162 is a highly simplified pseudocode description for a program that can be used to optimize the transparency with which digital ink is visible.

  The digital ink code 16200 illustrated in FIG. 162 uses the pointing device in the digital ink drawing mode by recording the stroke of the pointing device as a set of points and a curve or straight line between such points. A function 16202 responsive to user input. Function 16206 draws ink on the screen using straight and curved sub-pixel optimization. This can be done using virtually any sub-pixel optimization scheme, but using a two-color sub-pixel optimization scheme, such as the two-color sub-pixel optimization scheme using the nonlinear color balance described above. It is desirable to execute.

  FIG. 163 illustrates several digital inks 16302 rendered on the screen of the handheld computer 16300. Since this illustration is printed with a printer that can only represent all pixel luminance values, the digital ink illustrated in FIG. 163 displays sub-pixel optimization as a grayscale anti-aliasing. Of course, when viewing an image on a subpixel addressable display, the image illustrated in FIG. 163 also appears more clearly.

  If the user of the digital ink program chooses to enlarge the representation of the portion of the digital ink, function 16208 causes function 16212 to use the two-color subpixel optimization with non-linear color balance and the user selected enlargement. Generate digital ink linear and curved sub-pixel optimized bitmaps by size. Next, function 16212 displays the magnified image on the user's screen.

  FIG. 164 illustrates an expanded representation 16302A for a portion of the digital ink 16302 illustrated in FIG. This is substantially compared to the representation generated by simply expanding the pixilation of the digital ink representation 16302 illustrated in FIG. 163, as illustrated by the bitmap 16302B illustrated in FIG. 165. A clearer representation of digital ink is provided.

  It should be noted that the bitmap representation illustrated in FIG. 165 is actually easier to see than some enlarged representations of digital ink. This is because the bitmap illustrated in FIG. 163 is printed with full pixel grayscale values using anti-aliasing not used in some digital ink representations.

  Function 16214 causes function 16216 to generate sub-pixel optimized bitmaps of ink lines and curves at a selected reduced size using two-color sub-pixel optimization with non-linear color balance, Function 16218 displays the reduced bitmap on a subpixel addressable display. The result of such processing is illustrated by the bitmap 16302C illustrated in FIG.

  These features of the present invention can be modified to accommodate digital ink recorded as all pixels on or off. By allowing the routine to evaluate the centerline of each stroke represented by such “on” pixels, and then generating subpixel optimal images of the digital ink centerline at various scales, as described above, You can do this. A more accurate but computationally expensive approach seeks an optimal fit between a continuous portion of such digital ink and a corresponding straight line and curve series, such as a Bezier curve.

  In other embodiments, performing subpixel optimization on a bitmap generated by digital ink drawing by simply performing subpixel optimized scaling on such a bitmap. I can do it.

  By replacing sub-pixel optimization with grayscale anti-aliasing, some embodiments of the present invention for digital ink can be used with non-sub-pixel optimized displays.

  FIG. 167 illustrates components that a server, client, proxy server, thin client, remote, desktop, or many of the other computers described above may have. Of course, not all of the components illustrated in FIG. 167 are present on all such computers, and most such computers have other components in addition to those illustrated in FIG.

  Most computers used with the various features of the present invention can execute programs 16702 to execute functions relating to those features of the present invention and to read and write data 16704 in accordance with the methods of such features. The figure is shown to clarify that it has some kind of processor 16716. The present invention relates not only to methods but also to such computer programs and data, and further to computer systems that are programmed and / or embedded in hardware to perform or use such methods. About.

  In most such computers, the program of the present invention is in machine readable form, such as RAM 16706, ROM 16707, or hard drive 16708, floppy drive 16709, CD-ROM drive 16711, and / or DVD drive 16713, for example. Stored in mass storage. Further, for example, the program of the present invention can be stored in a floppy disk 16710, a CD-ROM 16712, a DVD-ROM 16714, or any other virtual machine-readable storage medium. Further, the program and / or data of the present invention can be expressed as a propagation signal indicated by a numeral 16719 that can be received by a computer via a certain communication port such as a network interface 16720.

  FIG. 168 provides a 320 × 240 screen full pixel grayscale representation showing the small sub-pixel optimized font generated using the nonlinear color balance method described above with respect to FIGS. . This figure is the same as FIG. 56 except that the text portion is surrounded by a broken line portion 16800.

  FIG. 169 is an 8 × enlarged view of the portion of the bitmap illustrated in FIG. FIG. 169 illustrates that most of the vertical strokes in the font illustrated in FIG. 168 include a color balance distribution on the left side that blurs the clarity of such fonts. This same slight blurring occurs in the small fonts illustrated in FIGS. 1, 10, 11, 99, 101, 104, 110-114, 121-128, 130, and 144.

  One of the main advantages of the non-linear color balance method of generating sub-pixel optimized font bitmaps is that if such distribution is not necessary for color balance, the non-linear method tries to prevent the distribution of luminance values by blurring the character font shape Is the ability to reduce

  When observing an undesirable diffusion of color values to the left of the main stroke of the type of font illustrated in FIG. 169, the inventors of this feature of the present invention have determined whether such diffusion can be reduced. I tried. This inventor tried to figure out what the source of such diffusion was.

  Referring again to FIG. 170, the inventor found that the algorithm used to generate the non-linear color balanced bitmap produced two padding columns of subpixels 17000, the actual non-zero coverage value 17004. Is designed to automatically set to the left of the leftmost sub-pixel column 17002 in the rasterization of a character font shape that contains (ie, actually covered by a portion of the character font shape represented by the rasterization) I found what I was doing. This is done to give room for the two subpixel columns on the left side of the leftmost subpixel column containing such non-zero coverage values to allow the color balance color value to be diffused. This is desirable when such left-side dispersion is required by the nonlinear color balance algorithm described above that allows color balance distribution to the left two pixels of the partially covered sub-pixel.

  Unfortunately, padding a rasterized subpixel array using only two such subpixel columns 17000 will result in the leftmost subpixel column 17002 containing such coverage values being replaced by the pixel column containing two padding subpixel columns. There is a tendency to have an unfavorable influence on the rightmost subpixel column. In RGB display, this causes the leftmost sub-pixel column with the actual coverage value to correspond to the blue sub-pixel.

  This is undesirable because it causes the pixel in the leftmost pixel column in the font bitmap to have two leftmost subpixels that do not have an actual coverage value and a non-zero coverage value. This is because the non-zero coverage value is required to be distributed in order to have the rightmost sub-pixel and achieve color balance. This is the reason for the blurring of most of the major vertical strokes illustrated in FIG.

  The inventor has found that a character font shape hinted by a system capable of appropriately grading the boundary of the vertical stroke relative to the width of the sub-pixel column is, for example, the end 17100 shown in FIG. Those who hinted these character font shapes to start the leftmost edge of the leftmost vertical stroke at a very slight distance to the leftmost subpixel column containing the non-zero coverage value 17002 Said that it is often designed. This substantially reduces the amount of non-zero coverage values contained within the subpixel column 17002 that the non-linear color balance had to distribute, so that in the subpixel optimized representation of the character. This significantly reduces unwanted blurring.

  For example, the inventor requires any color balance diffusion in the second leftmost pixel column 17103 in the resulting bitmap when many of the best hinting combinations are used in such an algorithm. The first vertical stroke of a character, such as vertical stroke 17102 illustrated in FIG. 171, for example, has three consecutive subpixel columns on the right side so that it has one or more pixels that are totally covered. Using the total coverage at, we found that it has a leftmost edge that fits slightly into one subpixel column.

  In such an optimized hinting process, to cover three adjacent subpixel columns starting at a distance of 3, 6, or 9 subpixel columns from the rightmost end of the first vertical stroke, The next vertical stroke is placed. Thus, the next vertical stroke, such as vertical strokes 17104 and 17106 illustrated in FIG. 171, for example, has a plurality of pixels that are totally covered so that no color balance is required.

  Although the types of fonts illustrated in FIGS. 168 through 171 are easier to read than most sub-pixel optimized font bitmaps generated by prior art methods, as a result of these studies, the inventor has shown that FIG. As illustrated in FIGS. 172 to 174, a clearer method of generating subpixel optimized fonts has been discovered.

  Except for using a new and clearer method for generating and displaying font bitmaps, FIG. 172 is a sub-pixel optimized 320 × 240 for a web page of the type illustrated in FIG. Figure 2 illustrates an all pixel grayscale bitmap representing a pixel representation.

  FIG. 173 shows a four times enlarged view of the portion of FIG. 172 illustrated in the dashed box assigned the number 17200.

  FIG. 174 shows a further magnified view of the text portion shown in broken line portion 17300 in FIG. 173.

  As can be seen in FIGS. 172 through 174, the horizontal spread of color values resulting from many of the vertical strokes contained in the font bitmaps illustrated in these figures is relatively low. It should be noted that the regularity in FIGS. 173 and 174 is not due to any diffusion due to color balance, but because the text in these figures was selected from the portion of the web page of FIG. A light gray background. The font illustrated in these figures is substantially clearer than the font illustrated in FIGS. 168 and 169.

  The inventor made this improvement by aligning the leftmost edge of the leftmost vertical stroke of the character with the left edge of the pixel boundary. In many embodiments, this is done by inserting the three padding subpixel columns 17500 illustrated in FIG. 175 in front of the leftmost subpixel column having a non-zero coverage value. This automatically aligns the leftmost rasterization unit (ie, subpixel), wholly or partially covered by the character outline, to the leftmost end of the pixel column. If the character is hinted so that the leftmost outline edge is aligned with the leftmost edge of the rasterization unit, this will cause the leftmost outline edge to be the resulting font bitmap. Is automatically aligned to the leftmost edge of the pixel at. If the leftmost edge of the font outline is a vertical stroke, this makes it very easy to generate a font bitmap with a clear leftmost vertical stroke even after nonlinear color balance .

  FIG. 176 illustrates one of many possible hinting interfaces that can be used with the present invention. In this hinting interface, the dashed line 17602 is a line that can be moved by the user to interactively define the left-hand bearing of the desired character. Dashed line 17604 is a movable line defining the right bearing. The left bearing is the distance between the first reference point, sometimes referred to as the pen position, to the position where the character is drawn and the leftmost end of the bitmap of the drawn character. Line 17604 corresponds to the position relative to the bitmap where the pen position is normally placed at the drawing start point of the next successive character along the line of text. The right bearing is the distance between the line 17604 and the rightmost end of the drawn character bitmap. The advance width is defined as the distance between line 17604 and line 17602. This represents the normal full width between the pen position before drawing of the character bitmap and the pen position after drawing. Although not necessary in other embodiments, in some embodiments, the left bearing and advance width approximate multiple pixel widths. In some cases, the left bearing value and / or the right bearing value can be negative. For example, this often occurs in italic fonts where bitmaps associated with consecutive characters often overlap each other's advance width portions.

  Each of the small rectangular dots 17606 illustrated in FIG. 176 corresponds to the center of the rasterization unit corresponding to an individual subpixel in the subpixel optimized font bitmap. In this special hinting interface, rasterized units that are more than half covered by the outline of the character font shape are shown in black. However, more advanced interfaces can display such rasterized units with grayscale coverage values. The outline of the character font shape is shown in the figure, and each point defining a segment in the outline is assigned a number, whether it is a control point or a segment end point.

  FIGS. 177 through 181 are used to help explain some of the steps described in the highly simplified pseudo code contained in FIG.

  The pseudo code illustrated in FIG. 182 focuses on computational features associated with the improved method for generating a clearer non-linear color balance sub-pixel optimized bitmap described above with respect to FIGS. 172 through 175. 182 is a highly simplified pseudo code description for program 6000A that generally corresponds to the pseudo code illustrated in FIG.

  This pseudo code has a function 18202 that determines the tightest rectangular array of rasterized units that can arrange character font shapes, taking into account the array of character font shapes for those rasterized units defined by hinting. Is put in.

  The position of the font outline relative to the individual rasterization unit where it occurs is not changed by this function. Thus, if the leftmost point of such an outline occurs outside of the left end of the rasterization unit in which it is located, that rasterization unit will be leftmost in the tightest rectangular array produced by function 18202. The leftmost point of the outline occurs in the leftmost rasterization unit column of the rectangle, but does not occur at the leftmost end of the leftmost column.

  Figures 177 and 178 are used to help illustrate this function. FIG. 177 corresponds to the outline of the hinted character font shape. FIG. 178 illustrates a rasterization unit rectangle (each size corresponding to a subpixel) returned by function 18202 for the character outline illustrated in FIG. 177. This grid corresponds to the tightest and smallest rectangle to which the rasterization unit containing the character font shape fits.

  When the function 18202 is completed, functions 6002A to 6006 are executed. These correspond to steps 6002 to 6006 in FIG. They are used to determine the coverage value for each rasterization unit contained in the rectangle returned by function 18202. Each such coverage value represents the percentage of subpixels covered by the rasterized high resolution character font shape outline.

  FIG. 179 illustrates the calculated coverage values for each rasterization unit in the array illustrated in FIG. There, the coverage is expressed as a percentage of rasterized units colored black. In FIG. 179, if the corresponding portion of the unit covered by the character font shape outline occurs at the top of the rasterization unit, the resulting bar graph portion in each rasterization unit representing coverage is Installed at the top of the unit.

  In FIG. 180, the bar graphs for all of the individual rasterization units correspond to the corresponding subpixels in order to more closely correspond to the coverage value representations illustrated in FIGS. 46 to 52 and FIGS. 92 to 93 described above. Installed from the bottom of the unit.

  Once the character font shape has been rasterized, step 18204 maps the resulting array of subpixel coverage values to an array of subpixel addressable pixels. Step 18204 does this by aligning the first column of the rasterization unit in the tight rectangle described above with the leftmost subpixel of the pixel row. This places the leftmost column of the rasterization unit having a non-zero coverage value as the leftmost subpixel column for all pixels described above with respect to FIG. In the example illustrated in FIGS. 177-181, this causes the resulting subpixel array to appear as illustrated in the central five pixel columns labeled 18102 in FIG. 181.

  Step 18206 then applies the bitmap array generated for the current character to a pixel column 1804 composed of three subpixels. Place this padding column on the left side of the pixel with the leftmost subpixel column containing the actual non-zero coverage value. Thus, the sub-pixel array of this example appears as shown by the combination of pixel columns 18104 and 18102 in FIG.

  Next, step 18208 adds the right two or more subpixel columns to the bitmap array so that the sum of the subpixel columns in the bitmap is an even multiple of 3, ie, an even multiple of all the pixel columns. Hit it. Thus, the sub-pixel arrangement of this example appears as shown by the combination of pixel columns 18104, 18102, and 18106 in FIG.

  Step 18210 adjusts the left bearing value and / or the right bearing value to offset the addition of padding pixel columns. Thus, for example, a bitmap that would otherwise have a 1 pixel wide left bearing would be modified to have a zero left bearing to offset the addition of the left padding column. Similarly, a bitmap with an extra pixel column added to the right will reduce the right bearing by one pixel width.

  Next, function 18212 performs a non-linear color balance corresponding to the steps described by the loop 6008 illustrated in FIG. 60 above, in many embodiments.

  When this is done, in an embodiment using a packed color value representation of the type described in FIG. 96 above, step 18214 determines that the pixel color values that result after the color balance process are more limited to a color palette. To the corresponding value from.

  Note that the method of FIG. 182 provides room for any color balance that may be required without tending to cause the unwanted color diffusion described above with respect to FIGS. 168 and 169. It is that. The method of FIG. 182 does this by ensuring that there are at least two subpixels to the left and right of any subpixel that corresponds to the area covered by the rasterized font shape. .

  In another embodiment of this aspect of the invention, the font shape and the leftmost edge of the vertical stroke and the rightmost character are used to maximize the ability of non-linear color balance to reduce blurring. Other methods for aligning the edges to all pixel boundaries are used. In some such embodiments, whether padding pixel columns have been added to the left or right side of the font bitmap can be a function of whether color balance distribution was required at those locations.

  FIG. 183 describes a function for drawing a character string using the bitmap generated by the method described in FIG. This pseudo code is similar to the pseudo code described above with respect to FIG. 97, except that it focuses on the features of the present invention used in the bitmap generated by the method of FIG.

  When the drawing character string function 18300 shown in FIG. 183 is called, step 18302 sets the pen position to the start position specified by the drawing character string call indicating the place where the display of the character string is the head.

  Next, a loop 9714A similar to the loop 9714 described in FIG. 97 is performed for each character in the character string for display.

  In this loop, step 9716 accesses the font bitmap of the current character. Next, step 18304 sets the starting point of the character to the current pen position. Step 18306 then adjusts the current pen position with the left bearing. As already mentioned above, the left bearing has been changed from its normal state to account for the fact that the character bitmap has an extra 1 pixel column on its left side, so the left side The bearing is reduced by the width of one pixel column.

  Step 9718A is then performed for each pixel in the font bitmap. It has a sub-step 18308 that performs a test to see if the current pixel value is non-zero. If the current pixel value is non-zero, draw the pixel at a position defined on the screen as a function of the current pen position.

  If the value of the current pixel is zero, it represents a totally transparent pixel, which means that the background color that previously existed at the current pixel position is left unchanged. In this embodiment of the invention, the function described in FIG. 96 retains a value of zero to represent such a totally transparent pixel.

  Avoiding writing transparent pixels applies to all pixels of the bitmap in the embodiment described in FIG. Avoiding writing transparent pixels is especially important for pixels in the padding column located at the leftmost end of the character font bitmap by step 18206 described above with respect to FIG. This is because the pixels in such a padding column do not have color values diffused into the padding column as a result of nonlinear color balance when the vertical stroke boundary is aligned with the vertical pixel boundary. As a result, such pixels become transparent, and color values that may have been placed at the pixel location by the left-hand letter can remain unchanged, thereby increasing coverage or color. Pixel columns of adjacent characters having balance information can be placed close to each other.

  If it has a color value from “w” that is transparent and not shown, and that can be displayed through the left padding column associated with “e”, for example, at the position indicated by numeral 17302 in FIG. Can be confirmed. Also, the pixel column between “r” and “e” is at the position indicated by numeral 17402 in FIG. 174 with a color value from “r” that is not superimposed on the transparent padding pixel column of “e”. This can be confirmed.

  As will be appreciated by those skilled in the art, function 9718A provides a certain iteration that controls where the pixels are drawn in order to draw each of the bitmaps at the appropriate location for each row of the font bitmap. To be repeated.

  Of course, in other embodiments of the invention, rather than a function that simply shows through the non-transparent color values from one character when the corresponding pixel of the next character is transparent. Rather, it can provide a function that can combine the overlap with non-transparent pixel values from neighboring characters.

  It would be desirable for such processing to allow the combination of such transparency values based on subpixel-to-subpixel. Such processing can provide more accurate representations of closely spaced characters, although more computational processing is required.

  One way to achieve this result is as follows. That is, each of the three corresponding alpha component values associated with any overlapping pixels between characters is added, and any component value is clipped at the maximum possible value. It then draws each of the resulting pixels and uses the combined component alpha value to determine how much foreground and background colors should be drawn at that location.

  FIG. 184 illustrates a hinting interface similar to the interface described above with respect to FIG. 176, except that it has interface characteristics 18402 configured with lines or controls movable by the user. With this control, the user can selectively position the character font shape outline so as to align with the leftmost end of the pixel column associated with the leftmost padding pixel column.

  Such interface characteristics are desirable when hinting fonts that have a leftmost edge other than a single vertical stroke. For example, if you are processing a character font shape that has a leftmost major vertical stroke with a small serif type protruding less than the maximum pixel width and protruding at its left edge, the hinting performer will Rather than type, you may wish to have a primary leftmost edge that is aligned to the full pixel boundary. The interface characteristics illustrated in FIG. 184 make it easier for hinters to select such an arrangement.

  Whether to add only two sub-pixel padding columns, as described above with respect to FIG. 170 or 171, or FIGS. 175, 181, and 182, as described above with regard to FIG. As described above with respect to, the user can choose whether to add three or more such sub-pixel padding columns.

  The method for clarifying the nonlinear color balance subpixel optimized image just described is not only a small font of the type shown in FIGS. 172 to 174 but also a relatively large font shown in FIG. It can also be applied to larger fonts.

  Of course, sub-pixel optimization can typically represent a font bitmap with three different types of pixels: foreground pixels, background pixels, and intermediate color balanced pixels. The foreground pixel represents the part of the font image that is entirely covered by the represented font shape, and is drawn in the foreground color where the characters are represented. The background pixel expresses a portion of the font image that is not entirely covered by the font shape, and is drawn on the uppermost portion where the font is displayed with the background color. An intermediate pixel represents a pixel that is partially covered with a font shape and / or a pixel that receives a color balance distribution for neighboring pixels that are partially covered. Each color of the sub-pixel is determined separately by the color balance.

  When a prior art non-linear color balance of the type described above with reference to FIGS. 46, 47, 52, and 93 is applied to a font, the character shape will be corrected even if the character shape edges are perfectly aligned with the pixel boundaries. Beyond each edge, color balance is performed in the direction of subpixel color change. No matter how good the hinting is, this leads to spatial blurring of all character shapes.

  Hinting can be used to greatly reduce the spatial blur caused by color balance when nonlinear color balance of the type described above with respect to FIGS. 48, 49, 51 and 91 is applied to the font. . Color balance distribution is often not required across pixel boundaries in character-shaped portions whose edges are aligned with pixel boundaries. This is because only such non-linear color balance distributes the color imbalance that occurs within a given pixel. This places the foreground pixel at that location next to the background pixel along the direction of the subpixel color change, greatly improving the perceived clarity of the font shape. This is illustrated in FIGS. 173 and 174, in which a substantial portion of the vertical stroke is illustrated with 8 pixels / em so that the end of the vertical stroke is aligned with the pixel boundary. Hinting at. As a result, the foreground pixels are placed horizontally next to the background pixels along a substantial portion of the ends of many such vertical strokes. Even when using the less optimal hinting of the leftmost vertical stroke end illustrated in FIGS. 168 and 169, the total color balance ambiguity is the ambiguity resulting from the prior art linear color balance. Substantially smaller than the total.

  FIGS. 185 through 190 are highly simplified regarding user interface innovations that can be used to improve browsing of web pages, especially when browsing is performed on a relatively small or relatively low resolution screen. Pseudo-code description.

  FIG. 185 is a higher level description of the selected text reflow method described above with respect to FIGS. This method 18500 includes a function 18502 for accessing the content of the web page and a function 18504 for performing an initial layout of the content of the web page, and placing the text at different horizontal positions indicated for the text in the web page. set.

  The markup language used to describe web pages is a number of methods (only two examples) that indicate that different parts of the text are drawn at different horizontal positions or different horizontal ranges of the web page. For example, it has a table and a frame).

  When such a layout is performed, function 18506 displays the elements of the layout at a predetermined scale and relative position determined by the initial layout. After this display is performed, step 18508 allows the user to select a portion of the text at a predetermined horizontal position in the display of the initial layout. One way to enable this is described above with respect to FIG.

  If such a selection is performed, function 18510 causes functions 18512 and 18514 to be performed. Function 18512 performs a second layout of the text selected by the user. This second layout reflows the selected text beyond the new column line where the text has a different and larger than normal font size compared to the line width of the column. When this second layout is performed, function 18514 displays the layout of the new column at a scale that satisfies at least two-thirds of the screen on which the web page is displayed, or the screen window width.

  As described above with respect to FIGS. 135-137, the second layout in the text reflow method so selected allows the user to view selected portions of the layout of the web page in a generally readable font size. . This is a huge advantage for low resolution screens, both small screens and / or screens that are relatively remote from the viewer. The first layout in such a way allows the user to see how the web page is planned to look in a more normal display, and what part of the text they want to see is the larger font size. You can select more quickly whether or not to redisplay.

  FIG. 186 is a high-level pseudocode description for the general kind of zoom-to-fit method 18600 described above with respect to FIGS.

  The method includes a function 18602 for accessing the contents of the web page and a function 18604 for laying out the contents of the web page.

  When a display of such a layout is displayed on the screen, function 18608 allows the user to drag the pointing device beyond this display. During such dragging, if dragging continues beyond the boundary associated with the edge of the screen, function 18610 causes function 18612 to remove the portion of the layout that did not previously exist on the screen opposite the screen edge. Scroll on the screen. This is done at the start of the drag so that there is a portion of the layout that is too large to fit entirely on the screen at the current display scale, or that only a portion is on the screen. This is because the user can select the portion of the layout that has been positioned by dragging.

  If the user releases the drag, function 18614 causes functions 18616 and 18618 to be executed. Function 18618 defines a portion of the layout as being selected based on the position on the layout that matches the beginning and end of the drag. The part so selected corresponds to the part of the layout with the horizontal or vertical extent of the drag, or the part of the layout with the diagonals corresponding to the beginning and end of such drag. Is possible. Function 18618 then displays the selected portion of the layout at a scale that substantially fits the screen. Substantially conforming to the screen means that the selected width, height, or area is at least two-thirds of the corresponding dimension or screen dimension, or portion of the screen where the display is being performed. Means to occupy.

  FIG. 187 is a high-level pseudocode description for a drag scroll method 18700 that allows a user to easily navigate within the display of a web page layout.

  The method includes a function 18702 for accessing the content of the web page, a function 18704 for performing the layout of the content of the web page, and a function 18706 for displaying all or part of the layout at a predetermined magnification. Thereafter, function 18708 allows the user to drag the pointing device beyond the display of the layout. Function 18710 responds to any such drag across the boundary associated with the screen edge by scrolling on the screen across the screen edge a portion of the layout that did not previously exist on the screen. .

  This method can be used as part of the zoom selection function or independent of the zoom selection function. This method has the advantage that the user can scroll the display of the layout of the web page by simply dragging the pointing device across the border at or near the edge of the display screen. This method is much more natural and faster than scrolling using a scroll bar.

  FIG. 188 is a high level pseudo-code description for the click zoom method 18800 that allows the user to choose to zoom in on the desired portion of the layout of the web page. The method includes a function 18802 for accessing web page content, a function 18804 for performing web page content layout, and a function 18806 for displaying all or part of the web page layout at a first scale. Function 18808 allows the user to click on the pointing device at the selected position in the display of the layout displayed at the first scale, and function 18810 surrounds the position in the layout where the click was performed. Respond to such clicks by performing a zoomed-in view of the portion of the layout. In general, zoom-in display is centered on the position in the layout where the click was performed.

  FIG. 189 is a highly simplified pseudocode description for the zoom click method 18900 described above with respect to FIGS. 121-128.

  This method includes a function 18902 for accessing web page content, a function 18094 for performing layout of the content, and a front or part of the web page layout on a display screen having an associated pointing device. It includes a function 18906 that displays at one scale. In the particular embodiment relating to this method described in FIG. 189, the screen is a touch screen and it is intended that the pointing device can be a human finger.

  When the display of the layout is executed at the first scale, the function 18908 corresponds to the pressing of the touch screen display. If such a press occurs, functions 18910 through 18922 are executed by this function.

  Function 18910 replaces the display of the portion of the first scale web page with a zoomed-in display of the portion of the larger scale web page on the screen. This zoomed portion includes the selected position in the layout associated with the touch screen press. It is desirable that the selected layout position has substantially the same position on the screen as the position that was displayed in the display at the first scale at the time of selection. Substantially the same position means that the selected position should have a position on the screen immediately before and after zooming that appears to correspond to the same contact located on the screen. This is because the change in the screen position of the selected position is more than 20% of the width or height of the screen immediately after such zooming (and preferably significantly compared to that amount). It is desirable to mean no change (in a small amount).

  When the zoom-in display is displayed, function 18912 displays a cursor above the position where the screen is touched to indicate the selected position in the layout of the web page associated with the touch. Some touch screen devices do not require such a cursor, especially with devices designed for use with styluses that have relatively precise points, but this allows the user to determine the point at which the screen is touched. This is because it can be seen fairly accurately. However, in touch screens designed to be used with a finger as a pointing device, in many cases, the screen will allow the user to pinpoint the position in the display of the screen associated with such contact. It is desirable to set the cursor on the position where the is touching. This is particularly desirable when the method is used with a display that is relatively small compared to the size of a human finger, for example as illustrated in FIGS.

  While the contact continues, function 18914 corresponds to any movement of the contact by moving the cursor correspondingly in the zoom display. Also, while such contact continues, function 18916 is moved to the screen edge by scrolling across the screen edge at the zoomed scale, beyond the edge of the screen, the portion of the layout that was not previously displayed on the screen. Respond to any movement of contact across the associated boundary. This allows the user to scroll quickly and conveniently within the zoomed display of the web page in the zoom click mode.

  Function 18918 responds when the user releases contact at a predetermined position in the zoomed display of the web page. When contact is released, function 18920 operates as if a pointing device click occurred at a position in the web page corresponding to the release position. For example, if the contact is released at the layout position corresponding to the web link, the system responds by selecting the link, and if the contact is released at the position of the radio button, the system changes the state of the radio button. Respond by switching.

  When this is done, function 18922 converts the zoomed-in layout display on the screen to the layout display at the same magnification at which the web page was displayed before the pointing device press was detected by function 18908. Replace.

  As described above with respect to FIGS. 121-128, zoom clicks allow a user to quickly select a desired portion of a web page at a zoomed-in scale that makes it easier to read the content of the selected portion and to select it accurately with a pointing device. It provides a useful technique for enabling visual recognition and selection.

  FIG. 190 is a highly simplified pseudocode description of a method 19000 that allows a user to view a zoomed-out view of a web page using greeking to represent a text line. Gleeking is an unreadable graphic representation of the size with which a portion of text is laid out in a document.

  The method includes a function 19002 for accessing web page content, a function 19004 for performing web page content layout, a function 19006 for detecting a scale selected by a user to have a web page content display layout, and 19014 is included.

  If the user chooses to display the web page layout at a predetermined larger scale, function 19006 causes function 19008 to display the portion of the web page layout at its larger scale. This involves performing a function 19010 to represent the image of the layout in a bitmap image that has been scaled for display at a larger scale and a separate size having a size suitable for display at a larger scale. It includes executing a function 19012 that represents a layout of a character string of a web page with a bitmap composed of bitmaps.

  On the other hand, if the user selects such a display scale that is too small to read at least some of the web page text in a readable size at a predetermined smaller display scale, function 19014 causes function 19016 displays the portion of the web page layout at a smaller scale. This is done by executing function 19018 that represents a layout image using a bitmap image that has been reduced for display at a smaller scale, and for displaying at a smaller scale, Performing a function 19020 that represents at least some strings using a bitmap composed of a greeked textual representation of the location.

  In many cases, the bitmap used to represent a string in such greeking is a width and / or height that matches the size of the corresponding string in the web page layout at a small scale. It is only a straight line or a rectangle having a thickness.

  If the layout is displayed in a size that the text is too small to be readable, using a greeked representation of the text makes it easier and more comfortable to see such display, Such greeking requires less calculation processing to generate a corresponding character string image from a small font bitmap that cannot be read.

  One of the primary uses for the method illustrated in FIG. 190 is to allow the user to immediately see an overview of the layout of the web page, such as the web page described above with respect to FIGS. 136 and 137, etc. The ability to select different parts of such web pages instantly.

  Those skilled in the art of computer user interfaces can combine some of the methods described in FIGS. 185-190 in combination with each other and with other features of the invention described above as part of a single user interface mode. Although it can be used, it will be appreciated that other methods are generally used with different user interfaces or different user interface modes.

  FIGS. 191 through 229 illustrate features of the invention that are particularly useful in very small screen displays, such as cellular phones, but can be used in other types of computers. In this specification, these features of the present invention are collectively referred to as the novel features of the present invention and claimed in any of the related applications listed at the beginning of this application from which this application claims priority. I believe it is not.

  FIG. 217 is a highly simplified representation of the main loop 21700 executed on the proxy server in an embodiment relating to the novel features of the invention described with respect to FIGS. 217-225. This embodiment uses a client proxy system of the type described above with respect to FIGS. As illustrated by text 21702 in FIG. 217, the steps of this main loop are similar to the steps described above with respect to FIG. 115 except for the differences indicated by texts 21704-21706.

  As illustrated by text 21704, the particular embodiment illustrated in FIGS. 217-225 does not support the rescaling described in functions 11526-11532 of FIG. 115. Other embodiments of the novel features of the present invention can support such scaling in the overview window and / or the magnified view window.

  As indicated by the text 21706, in certain embodiments relating to the novel features of the present invention described in FIGS. 217-225, in addition to the fact that the proxy requires a desired virtual screen width, the layout width As a subset of the layout, it does not have a concept for virtual screens. The layout engine attempts to limit the virtual layout to this requested width unless the size of the layout element requires a larger virtual layout width. In the embodiment described in FIG. 115, the virtual screen needs to be moved first, but in this embodiment, the proxy processes the entire layout as a virtual screen and moves the virtual screen first. It can handle clicks that occur anywhere in the virtual layout without need.

  The embodiment illustrated in FIGS. 217 through 225 downloads a display list of the entire layout of a given web page to the thin client, as in the embodiment described with respect to FIG. This allows the client computer in the downloaded web page to display the client moved to a new portion of the web page without incurring the delay necessary to cause the proxy to download a continuous portion of the web page. The display can be immediately moved to an arbitrary position.

  In other embodiments of the invention, the entire web page may not be downloaded at once. For example, one representative example of these new features of the present invention downloaded a 400 × 473 pixel display list corresponding to the 800 × 945 portion of the proxy server virtual layout. In such embodiments, if the user navigates away from the previously downloaded portion of the web page, the client then notifies the downloaded proxy of the new portion of the web page layout.

  FIG. 218 illustrates a portion of the client computer browser program main loop 21800 associated with the novel features of the present invention. Although not shown in the figure, it should be appreciated that the client main loop includes steps similar to those described above with respect to FIG. In addition, it includes the functions indicated by the numbers 21802 to 21878.

  The function 21802 corresponds to receiving a display list regarding the requested web page from the proxy server by causing the functions 21804 to 21806 to be executed. Function 21804 sets the first selected portion of the display, i.e., the portion initially displayed in any screen window, to the upper left corner of the layout of the web page, as shown in the downloaded display list. . This is the portion of the web page layout that is displayed on any display screen before the user takes any action to select other portions of the web page for display purposes.

  In an alternative embodiment, the portion of the web page that is initially selected for display purposes can be set by other means, such as using a view setting of the type described above with respect to step 11502 of FIG.

  Function 21806 accommodates receipt of those elements by the client by converting successive elements of the downloaded display list into a display list data structure stored on the client computer. In the current embodiment, the downloaded display list is later set by the client browser in step 21806 to determine the corresponding data structure that will be set to the display list that represents the web page on the client. Actually communicated in the page description language being parsed.

  In some embodiments, bitmaps for the overview and enlarged view windows are dynamically generated from a display list corresponding to the downloaded web page layout. In another embodiment, the layout bitmap is created from the downloaded display list at the reduced resolution specified in the display list, and the enlarged display simply displays a portion of the bitmap at the maximum resolution. The overview displays a further reduced version of the bitmap.

  Function 21808 illustrates a branch function corresponding to receiving a predetermined type of input by directing the program flow to the function or functions intended under the received type of input of FIG. .

  The bifurcation function 21808 can generate the corresponding input by any suitable user interface hardware and / or software routine virtually. When using a cell phone of the type illustrated in FIG. 191, by using a menu 19104 of a type similar to the cell phone illustrated in FIGS. 191 and 192, and in the phone pad 19105 of such cell phone Such a selection can be performed by pressing a key having a number corresponding to the desired menu item. It is also possible to select one or more of such inputs by pressing one of buttons 19108 and 19112, each of buttons (19108) and 19112 being key definition tabs 19106 and 19110 at a predetermined time. Each with a current function defined. In other embodiments, including those running on a personal digital assistant (PDA), such as the embodiment illustrated in FIG. 229, any user interface suitable for entering user input can be used.

  As illustrated by functions 21810 and 21812, if the user generates an input indicating a desire to view a split view of the type illustrated in FIGS. 195-199, the program sets the view mode to split view mode. . This runs the client browser in split view mode by executing the function indicated by the numbers 21856 and 21858 illustrated in FIG. 218, corresponding to the function of FIG. 219.

  Functions 21814 and 21816 correspond to a user selection to view an overview only mode of the type illustrated in FIGS. 191 to 193 by setting the view mode to an overview only mode. Thus, the client browser functions in the overview only mode by executing the functions indicated by numerals 21860 and 21862 in FIG. 218, corresponding to the functions in FIG.

  As shown by functions 21818 and 21820, when the user selects the type of enlarged only view shown in FIG. 194, the view mode is set to the enlarged only mode. As a result, the enlarged only mode function indicated by numerals 21860 and 21862 corresponding to the function of FIG. 221 is executed.

  Functions 21822 and 21824 correspond to a user selection to view a magnifying glass view of the type illustrated in FIGS. 208-211 by setting the view mode to magnifying glass mode. Thereby, the magnifier mode function indicated by numerals 21864 and 21866 corresponding to the function of FIG. 224 is executed.

  If the user selects a reflow text only view of the type illustrated in FIGS. 205-207, functions 21826 and 21830 set the view mode to reflow text lonely view. Thereby, the functions indicated by numerals 21868 and 21870 corresponding to the functions of FIG. 222 are executed.

  Functions 21832 and 21834 correspond to a user selection to view a reflow text split view of the type illustrated in FIGS. 201-203 by setting the view mode to reflow text split view mode. As a result, the functions indicated by numerals 21872 and 21874 corresponding to the functions in FIGS. 223A and 223B are executed.

  If the user performs a reflow text size selection, functions 21836 and 21838 set the current reflow text size to the selected size. In many preferred embodiments, the reflow text size is selected from a series of sizes that can be labeled as, for example, “small”, “medium”, “large”, and “extra large”. Indicates. With each such size selection, web page text with a different size font associated with it usually expands in a way that is somewhat proportional to such different sizes, for example by making the heading text larger than the body text. / Reduced.

  If the user chooses to switch the column width limit on / off, functions 21840 and 21842 will switch to the on state if the column width limit is previously off, and if it is on beforehand. Switch to the off state. As described below with respect to FIGS. 225-228, the column width restriction limits the width of the text column in the layout of the web page to a width that fits within the enlarged view window.

  If the user selects cursor navigation, functions 21844 and 2184 set the navigation mode to cursor navigation. In many cell phone embodiments, this causes, for example, the switch 19114 illustrated in FIG. 191 to cause the movement of the navigation rocker switch to move the cursor 19116 illustrated in FIGS. Move directly.

  If the user selects view navigation, functions 21848 and 21850 set the navigation mode to view navigation. Thereby, the up / down / left / right movement of the navigation switch directly moves the enlarged view, the text reflow view, or another type of view in the corresponding direction.

  For example, if the user selects a link by pressing a select button when the cursor is positioned on a link displayed on the web page, the functions 21852 and 21854 may cause the proxy server to associate the web page associated with the link. Respond to the proxy server in the manner described above with respect to FIGS. 115 and 217, including the request for the web page from its source, perform the layout of the web page, and then scale down to correspond to the layout Create a display list and download it to the client.

  As described above, the functions corresponding to the numbers 21856 and 21858, the numbers 21860 and 21862, the numbers 21864 and 21866, the numbers 21868 and 21870, the numbers 2172 and 21874, and the numbers 21762 and 21878 are shown in FIGS. 223A and 223B and 224 are described in further detail.

  FIG. 219 describes a series of functions 21900 that are executed when the view mode is set to split view. Screen shots produced by these functions are illustrated in FIGS. 195-199.

  Function 21902 displays the elements created by display functions 21904 through 21910 simultaneously.

  As will be appreciated by those skilled in the art of screen user interfaces, in many embodiments, the display functions illustrated in the figure and in FIGS. 220-224 are typically event driven, Those display functions usually only make changes to the bitmap displayed on the screen in response to an event that indicates creating, changing, and moving the display of the associated display element. Means that.

  The function 21904 displays, for example, an overview window such as the overview window 19200A illustrated in FIG. 195 on the first horizontal portion of the screen.

  The embodiment for the split screen illustrated in FIGS. 195 through 199 is displayed at the 176 × 220 pixel resolution used in the smartphone specification supported by Microsoft Corporation. In this embodiment, the overview window has a total size of 176.times.132 pixels, including any scroll bars that may be located therein, on most desktop and laptop computers. On the other hand, a common aspect ratio of 4 to 3 is assigned to the size.

  In the embodiment described in detail, a portion of the layout corresponding to 800 pixels wide in the virtual layout of the proxy server and 400 pixels wide in the reduced layout displayed by the downloaded display list. The overview window 19200A shows a portion of the layout described in the downloaded display list, scaled down horizontally and vertically from the resolution listed in the display list. If the overview window includes a vertical scroll bar, the portion of the displayed layout is reduced to the width required for the scroll bar.

  In other embodiments, the user can be provided with the ability to have the entire web page scaled to fit the overview window, but the web page size is too large for the viewer. In many cases this is undesirable if such a scaled view becomes almost useless.

  Function 21906 displays the enlarged display window 19400A illustrated in FIG. 195 on the second horizontal portion of the screen. In the embodiment illustrated in FIGS. 195-199, the generally enlarged window has a maximum 176 pixel width and 88 pixel height of the screen on which it is displayed. This corresponds to a size of 352 × 176 pixels in the virtual layout of the proxy server formed on the proxy server.

  The enlarged view window displays some subparts of the layout shown in the overview window at the maximum resolution described in the downloaded display list. In the illustrated embodiment, the enlarged view window is “enlarged” relative to the overview window. In fact, it is scaled down by a factor of 2 in relation to the virtual layout formed on the proxy server and the size of many of its elements found on most desktop or laptop computers.

  The text in the enlarged view window is displayed with an anti-aliased font bitmap, which can be used to optimize or improve readability for the actual resolution displayed in the enlarged view window. It has the shape and pixel arrangement of the selected character outline. If the screen displaying the enlarged window has a subpixel string that runs in the vertical direction, the vertical direction is identical or similar to the font described above with respect to FIGS. 55 through 97 and FIGS. 168 through 184 Sub-pixel optimized fonts can be used. When sub-pixel stripping is performed in the horizontal direction, a font that is sub-pixel optimized in the horizontal direction can be used, such as a font that uses the horizontal conversion of nonlinear sub-pixel optimization described above.

  Function 21908 displays an enlarged view indicator 19504 displayed in the overview window of FIGS. 195-199. This indicates the position of the portion of the layout currently displayed in the enlarged view window relative to the web page layout displayed in the overview window.

  The function 21910 displays the cursor 19116 shown in FIGS. 195 to 199 in both the overview window and the enlarged view window. The cursor has the same position relative to the layout in both the overview and the magnified view.

  This cursor may have different shapes as a function of whether it is located in different embodiments, in different view windows, or over selectable links, text, text areas, or other types of web content. I can do it. In various embodiments with respect to FIGS. 191 to 211, the cursor is illustrated as a small cross 1919, a link selection symbol 19116A, or a text cursor 19116B.

  In low resolution displays, the cursor tends to be relatively large, which in some embodiments tends to further hinder the viewing of text and images compared to high resolution displays. The user is given an option in all or several viewing modes to switch the cursor display off.

  By executing functions 21914 through 21948 in the split view, function 21912 corresponds to receiving navigation input, such as pressing a rocker switch illustrated in FIG.

  If the current navigation mode is in view navigation mode, i.e., a mode in which the user can navigate by directly moving a portion of the layout displayed in the enlarged window, function 21914 causes functions 21916 through 21928 to be executed. Is done.

  Function 21916 causes functions 21918 and 21920 to correspondingly (a) a portion of the web page layout displayed in the enlarged view window in the direction indicated by the received navigation input, and (b ) Move both the enlarged view indicator associated with the layout in the overview window.

  The movement performed by functions 21916 to 21920 causes the enlarged view indicator to extend beyond a portion of the web page layout in the overview window and above the previously hidden portion of the web page layout. If a part is moved, function 21922 also causes functions 21924 to 21928 to be executed as part of the movement.

  Function 21924 scrolls the previously hidden portion of the layout in the overview window. Function 21926 moves the magnified view indicator over the previously hidden portion of the web page layout. The function 21928 scrolls the layout subpart displayed in the enlarged view window. In different embodiments, the instructions of such functions can be changed.

  FIGS. 196 through 199 illustrate view navigation performed by functions 21914 through 21928 in split screen mode.

  FIG. 196 illustrates a split view using an enlarged view indicator 19504 at the left edge of the web page layout. FIG. 197 illustrates the magnified view indicator and the split view after moving a portion of the layout in the magnified view to the right by the operation of functions 21916 and 21920.

  FIG. 198 shows the divided view after the user navigates the enlarged view downward from the position of the enlarged view in FIG. 197 to the lower end of a part of the layout displayed in the overview window. It is shown.

  FIG. 199 shows that the user has been enlarged below the portion of the overview displayed in the overview window in FIG. 198 through the operation of functions 21922 to 21928 as well as the operation of functions 21916 to 21920. The split view is displayed after navigating down the view.

  By the action of functions 21922 to 21928 and the action of a similar function included somewhere in the figure, the user can move the enlarged window beyond the boundary of a part of the layout shown in the overview window. You can navigate smoothly.

  In the embodiment illustrated in FIGS. 196-199, the cursor is automatically moved to hold in the enlarged view. In other embodiments, the position of the cursor can be independent of the position of the magnified view. Typically, when such other embodiments are in the cursor navigation mode, the magnified view moves to hold the cursor in that view, which allows the user to use a larger scale displayed in the magnified view. Thus, it is possible to visually recognize the position of the cursor.

  When the current navigation mode is in the cursor navigation mode, that is, the mode in which the user can directly move the cursor when the split screen is displayed, the functions 21932 to 21948 are executed by the function 21930.

  Function 21932 moves the cursor directly in the direction indicated by the input, relative to the portion of the web page layout displayed in both the enlarged view and the overview.

  This function moves the cursor over a part of the web page layout that was previously hidden in the magnified view window, beyond a subpart of the web page layout displayed in the magnified view window. If so, function 21934 causes functions 21936 through 21940 to be executed as part of this movement.

  Function 21936 moves the cursor over the previously hidden portion of the web page layout. Function 21938 scrolls the web page layout in relation to the enlarged view window, so that the portion that was hidden before the cursor was moved is scrolled in the enlarged view window. Then, the function 21940 moves the enlarged display indicator in the overview window.

  Functions 21934 through 21940 show that in cursor mode, the user can scroll the enlarged view window relative to the layout by pressing the cursor against the enlarged view boundary. . In an alternative embodiment related to the present invention, the user may be able to scroll the cursor outside the magnified view.

  The movement performed by function 21932 moves the portion of the magnified view indicator beyond the portion of the web page layout in the overview window over the previously hidden portion of the web page layout. If so, function 21944 performs functions 21944 through 21948 as part of this movement.

  Function 21944 scrolls the previously hidden portion of the layout in the overview window. Function 21946 moves the magnified view indicator over the previously hidden portion of the web page layout. The function 21948 then scrolls the sub-parts of the layout displayed in the enlarged view window.

  FIG. 220 illustrates a series of functions 22000 executed by the client browser when the user chooses to put the client browser into overview-only mode. FIGS. 191 to 193 are screenshots of 176 × 220 resolution generated in this mode.

  By function 22002, functions 22004 and 22006 cause the overview window 19200 and cursor 19116 to be displayed simultaneously on the display screen of the client browser.

  The overview window displayed by the function 22004 is displayed on almost the entire screen. The overview window displays a part of the layout described by the downloaded display list, and is reduced from the resolution described in the display list.

  In the illustrated embodiment, the overview window in the overview only mode reduces this layout by the same magnification as the overview window illustrated in the split mode described above with respect to FIG.

  In some embodiments, for example, to leave room for control and other purposes, such as the “Menu” tab and the “Quit” tab illustrated at the bottom of FIG. 193. The overview window covers almost all of the screen, not everything.

  When describing the size of an overview-only window, the term “substantially all” is used, but this may be used to mean that it covers at least 80% of the screen area. It means that it is not so extreme compared to the situation. In some embodiments, such “substantially all” coverage is associated with the portion of the screen occupied by the browser program.

  If the current navigation mode is the view navigation mode and the client computer receives navigation input from the user in the overview only mode, functions 22010 and 22012 are displayed in the overview window in the direction indicated by the input. Scroll through a portion of the displayed layout.

  On the other hand, when the client receives the navigation input and is in the cursor navigation state, the functions 22016 to 22022 are executed by the function 22014.

  Function 22016 moves the cursor directly in relation to the portion of the web page layout displayed in the overview window in the direction indicated by the input.

  If this move performed by function 22016 causes the cursor to move over a portion of the web page layout in the overview window over a previously hidden portion of the web page layout, function 22018 Functions 22020 and 22022 are also performed as part of this movement. Function 22020 scrolls the portion of the layout that was previously hidden to the overview window. Function 22022 then moves the cursor over the previously hidden portion of the web page layout. These functions allow the user to scroll the overview window.

  FIG. 221 illustrates a series of functions 22100 that are executed by the client browser program when the client browser is in the expanded-only view mode. A screen shot of 176 × 220 resolution for the display generated in this mode is shown in FIG.

  As shown in FIG. 194, a function 22102 causes a view window 19400 and a cursor 19116 in which the functions 22104 and 22106 are enlarged to be displayed simultaneously. In magnified only view, the magnified view window occupies almost all of the client screen (ie, at least 80% of the client screen).

  In the illustrated embodiment of the present invention, the enlarged view window (a) displays a portion of the web page layout at the maximum resolution listed in the downloaded display list, and (b) is enlarged. The same anti-aliasing or sub-pixel optimization process with character outlines formed and arranged using bitmap pixels to improve or optimize the readability of the resolution displayed in the view It is similar to the window displayed in the split view described above with respect to FIG. 219 in that the text is displayed using the done font bitmap.

  If navigation input is received in an enlarged only view, the navigation is performed in relation to the substantially maximum screen magnified view rather than the approximately maximum screen magnified view in FIG. , Functions 22110 to 22122 perform navigation in the same way as when each is performed in overview only view.

  FIG. 222 illustrates a series of functions 22200 being executed by the client computer when in the reflow text only view mode. Simulations of screenshots generated in this mode are shown in FIGS.

  If the text of the column where the cursor is currently located or nearest is not reflowed with the reflow text size, the functions 22201 and 22202 of FIG. 222 are reflowed with the currently selected reflow text size. Lay out the text of the column beyond the line of the text column.

  In the illustrated embodiment, this text reflow is performed in a manner similar to that described above with respect to FIGS. 130-134, except that the user does not need to select the width of the column to be reflowed. I can do it. Instead, the system selects the column where the cursor is currently positioned as the reflowed column, or is closest to the current cursor position if the cursor is not currently positioned in a column containing any text As a matter, the column containing the text is selected.

  In some embodiments, when a split screen is displayed, and when the user chooses to change to reflow text mode, if the system is in enlarged view navigation mode, it will automatically Selected text column is the column closest to the center of the enlarged view.

  In the embodiment described herein, text column reflow is performed on the client browser to allow for faster display of the reflowed text. The operations associated with this layout are usually not as complex as the operations associated with the layout of many web pages, so text column reflow can be easily performed. In other embodiments, text column reflow for reflowed text views can be performed on the proxy server.

  Perform text reflow using a font having a size determined by the currently selected reflow text size. Typically, the reflowed text size relative to the reflowed column width is larger than the text font selected relative to the column width in the web page layout. If the text column has different sized fonts, one or more such individual font sizes can be individually scaled as a function of the selected reflowed font size in a generally proportional manner.

  Function 22203 causes functions 22204 and 22206 to simultaneously display both the reflowed text window 20502 illustrated in FIGS. 205-207 and the cursor 19116A or 19116B. Function 22204 displays the reflowed text window on substantially all of the client computer's screen (ie, 80% or greater). This display shows the maximum width of the reflowed text column so that lines of text can be read without having to move the screen alternately to the right and left.

  Although not illustrated in the pseudo-screen shots of FIGS. 205-207, the anti-aliasing where the character outline shape and pixel placement is selected to improve or optimize readability for the resolution displayed in the window It is desirable to display the text in this window using a font bitmap. In some embodiments, this includes users of non-linear sub-pixel optimized fonts, such as, for example, the vertical sub-pixel optimized fonts described above with respect to FIGS. 55-97 and FIGS. 168-184.

  The operation of functions 22201 to 22206 is described in FIGS. FIG. 204 illustrates the position of the cursor 19116 in relation to the web page in the split view formed by the function of FIG. FIG. 205 illustrates a reflow text only view that occurs when the user selects that view mode when the cursor is positioned at the position illustrated in FIG. As can be seen from a comparison of FIGS. 204 and 205, the text column closest to the cursor 19116 is displayed in the resulting reflowed text window 20502 illustrated in FIG.

  Functions 22212 through 22232 are executed by function 22210 when navigation input is received in reflow text-only view mode when view navigation is selected.

  If the navigation input is intended for horizontal movement, function 22212 causes function 22214 to be executed. This function tests to see if there is a text column on the left or right side of the text column displayed prior to receiving this input, and if there is a text column, this function 22216 to 22220 are executed.

  Function 22216 moves the cursor to the closest position in the column. Function 22218 reflows the text present in that column into the reflowed text column in the same manner as described above with respect to function 22202. Function 22220 displays the text in this newly reflowed text column that is closest to the cursor in the reflowed text window.

  205 and 207 illustrate the operation of the functions 22216 to 22220. FIG. 207 illustrates the display of text in the column to the right of the column illustrated in FIG. 205 that occurs when the user sends a navigation command that moves the view to the right. In FIG. 205, the displayed text corresponds to the text column occupying most of the enlarged view in FIG. In FIG. 207, the displayed text corresponds to the text in the column starting from the right side of the enlarged view in FIG.

  In the reflow text only view, when a vertical movement is received by view navigation, functions 22224 to 22232 are executed by function 22222.

  Function 22224 scrolls a portion of the reflowed text column displayed in the reflowed text window in the vertical direction indicated by the input. Such scrolling is indicated by a change between FIG. 205 and FIG. 206, which displays a sub-portion of the text illustrated in FIG.

  If such scroll reaches the top or bottom of the reflowed column, function 22226 causes functions 22228 to 22232 to be executed. Function 22228 places the cursor at the bottom or top of the nearest text or text column above or below, respectively. Function 22230 reflows the text in the new column to fit within the reflowed text window. The function 22232 displays the cursor in the reflowed text window and the text in the vicinity thereof.

  If the system is in cursor navigation mode, functions 22236 through 22254 are executed by function 22234 upon receipt of navigation input.

  Function 22236 moves the cursor directly in the direction indicated by the navigation input in relation to the portion of text in the reflowed column displayed in the reflowed text window.

  If the cursor is moved with a horizontal left or right movement beyond the end of a given line, functions 22238 and 22240 may cause the cursor to end at the top of the upper line or at the beginning of the lower line, respectively. Move.

  If the movement of function 22236 or 22240 exceeds the top or bottom of the current reflowed column, function 22242 causes functions 22244 to 22248 to be executed. Function 22244 places the cursor at the bottom or top of the nearest text column above or below, respectively. Function 22246 reflows the text in the new column to fit within the reflowed text window. Function 22248 displays the cursor and nearby text in the reflowed text window.

  The movement of 22236 or 22240 causes one of the text columns that were previously hidden in the reflowed text window to cross some of the text columns displayed in the reflowed text window before the navigation input is generated. When moving the cursor over a section, function 22250 causes functions 22252 through 22254 to be executed as part of such movement.

  Function 22252 moves the cursor over the previously hidden portion of the reflowed text column, and function 22252 scrolls the reflowed text column relative to the reflowed text window, thereby The portion that was hidden before the cursor was moved is scrolled to the reflowed text window.

  223A and 223B illustrate a series of functions 22300 that are executed when the client is in a reflow text split view. 201 to 203 are pseudo screenshots regarding the view generated in this display mode.

  A diagram that may match the function assigned a similar number in FIG. 222 if the text of the column where the cursor is currently or closest is not reflowed with the current reflowed text size Functions 22201 and 22202 of 223A lay out the text of the column beyond the line of the newly reflowed text column at the currently selected reflow text size.

  Function 22302 in FIG. 223A causes functions 22304 through 22310 to simultaneously display the elements of the reflowed text split view.

  Function 22304 displays an overview window 19200A on the first horizontal portion of the display screen. The overview window displays a portion of the layout expressed by the downloaded display list and is reduced from the resolution expressed in the display list. In the illustrated embodiment, this overview window corresponds to the split view window described above with respect to FIGS. 195-199 and 219.

  Function 22306 displays the reflowed text window 20102 illustrated in FIGS. 201-203 on the second horizontal portion of the screen. This window displays the maximum width of the reflowed text column so that the text can be read without having to scroll left and right for each line.

  Text in reflowed text window 20102 with partial screen using the same anti-aliased font bitmap as described above with respect to text window 20502 reflowed with approximately maximum screen, illustrated in FIGS. It is desirable to display.

  Function 22300 displays the displayed text indicator 20104 illustrated in FIGS. 201-203 in an overview window showing a portion of the text displayed in the overview window currently displayed in the reflowed text window. .

  Function 22310 displays a cursor in both the overview window having the same position and the reflowed text window for the text in each such window.

  The remainder of the functions in FIGS. 223A and 223B illustrate navigation in the reflowed text split view. Except for the text underlined in FIGS. 223A and 223B corresponding to changes or additions, these functions correspond to the functions 22210-22254 illustrated in FIG. In FIGS. 223A and 223B, the function corresponding to the function illustrated in FIG. 222 has the same number followed by the letter “A”.

  The differences between the navigations of FIGS. 223A and 223B relative to the navigation illustrated in FIG. 222 are displayed in the reflowed text window as indicated by (1) functions 22312, 22314, 22316, 22324, 22326, and 22328. The displayed text indicator is moved to accommodate changes in the text being displayed, and (2) displayed in the reflowed window as shown by functions 22318 through 22324 and functions 2230 through 22326 Scrolling a portion of the web page layout displayed in the overview window when it is necessary to make a portion of the displayed text visible in the displayed text indicator on the overview window; ) Function 2223 As indicated by A, including movement of at cursor both reflow text window and the overview window.

  FIG. 224 illustrates a series of functions 22400 that work when the view mode is set to magnifying glass view. Such a series of functions includes functions 22402 to 22454. Screen shots of this mode are illustrated in FIGS.

  The function 22402 causes the functions 22404 to 22410 to simultaneously display the elements of the magnifier display.

  Except for the portion covered by the magnifier window, function 22404 displays the overview window 19200B on substantially all of the display screen. The overview window displays a part of the layout expressed by the downloaded display list, and is reduced from the resolution expressed in the display list. In the illustrated embodiment, the overview displays the web page layout at the same resolution as the overview window in split view, overview only mode, and reflow text split view.

  When the overview window is described as “covered” in the enlarged view window, a portion of the overview window appears to be covered by the enlarged view on the display from the user interface perspective. Means. The overview window appears to be covered by the magnified view when the magnified view is moved so that it is positioned between FIGS. 209 and 210 and between FIGS. 210 and 211. This is because the previously “covered” portion is displayed so that the enlarged view leaves the previously “covered” portion of the overview window. Of course, in most embodiments, pixels in the portion of the display screen that is covered by the magnified view at any given time will only display the bitmap of the magnified window.

  Function 22408 displays an enlarged view window 19400B beyond a portion of the layout illustrated in the overview window. The enlarged view displays an enlarged subpart of the covered subportions of the layout displayed in the overview window at the maximum resolution represented in the downloaded display list.

  Function 22408 displays one or more magnified subpart markers that indicate the position of the subpart of the layout displayed in the magnified view in relation to the layout displayed in the overview. In the illustrated embodiment, the enlarged subpart markers are labeled 20804 and 20806 in FIGS. 208-211 and are placed at the end of the enlarged view window. A marker 20804 indicates a vertical position with respect to a part of the overview window displayed in the enlarged view window, and a marker 20806 displays a horizontal position of the part of the layout of the overview window.

  Function 22410 displays user movable cursor 19116 in the enlarged view window. Since the portion of the overview window corresponding to the cursor position is covered with an enlarged view, the cursor is not displayed in the overview window in this mode.

  When the client computer receives navigation input in the magnifier mode, functions 22414 through 22426 are executed by function 2212.

  If the input is a vertical navigation input, function 22414 causes functions 22416 through 22420 to be performed. The function 22416 vertically moves the enlarged view window relative to the overview window in the vertical direction corresponding to the input, if there is room. Function 22418 moves the position of the enlarged subpart of the layout displayed in the enlarged view vertically. Function 22420 then moves the vertically magnified subpart markers vertically, so that their positions correspond to the vertical position of the magnified subpart in relation to the layout in the overview window after such movement. .

  Usually, the operation of functions 22416 to 22420 moves the magnified view so that its vertical center remains in relation to the overview window and covers the position of the magnified portion of the web page layout. Yes. When this condition occurs, the position of the vertically enlarged subpart marker is moved relative to the entire display screen, but remains the same relative to the enlarged window. In the illustrated embodiment, when the magnified view reaches the top or bottom of the space corresponding to the largest overview window, further upward navigation or downward navigation, respectively, is magnified view window. However, the sub-part marker expanded vertically is moved upward or downward.

  The difference between FIGS. 209 and 211 illustrates the vertical navigation of the type of magnified view performed by functions 22414 to 22420.

  When receiving horizontal navigation input, function 22422 performs functions 22424 and 22426.

  Function 22424 moves the position of the expanded subpart of the layout horizontally. Function 22426 then moves the vertically enlarged subportion marker horizontally to accommodate changes in the horizontal position of the enlarged subportion in relation to the layout in the overview window.

  The differences between FIGS. 208 and 209 illustrate horizontal navigation in the magnifier view mode. It should be noted here that the enlarged view itself does not move in the horizontal direction, but only the layout portion displayed in the enlarged view and the horizontal subportion marker 20806 move. This is because in the illustrated embodiment, the enlarged view enlarges the maximum width of the screen to allow the display of longer lines of text, thereby eliminating the possibility of moving horizontally. Because.

  In another embodiment, the overview window is below the magnified view window to hold a portion of the layout displayed in the magnified view located below the center of the magnified view. It is possible to scroll horizontally, so that it behaves more like a real magnifying glass.

  In other embodiments, and in particular in embodiments having a wider or higher resolution screen, the width of the magnifying glass is actually narrower than the width of the screen, so that in the illustrated embodiment In this case, the magnifying glass can be moved horizontally in the same manner as the magnifying glass can move vertically.

  When the client browser displaying the magnifier display view receives navigation input in the cursor navigation mode, functions 22430 through 22438 are executed by function 22428.

  Function 22430 moves the cursor directly in the direction indicated by the input in relation to the portion of the web page layout displayed in the enlarged view. The function 22432 moves the function 22430 to move a part of the web page layout that was previously hidden in the enlarged view window beyond the sub-part of the web page layout displayed in the enlarged view window. Above, test to see if you move the cursor. When the cursor moves as described above, function 22432 causes functions 22434 through 22438 to be executed as part of that movement.

  Function 22434 moves the cursor over a previously hidden portion of the web page layout. Since function 22434 scrolls the web page layout in relation to the enlarged view window, the portion that was hidden before the cursor was moved is scrolled to the enlarged display window. Then, if such an indicator needs to correspond to the position associated with the overview corresponding to the portion of the web page displayed in the magnified view, function 22438 sets the magnified view indicator to this. Scroll.

  The web page layout was previously moved beyond the portion of the layout corresponding to the overall overview window, including the portion covered by the magnified view, by movement performed in either view navigation or cursor navigation mode. If the subportion of the enlarged subpart of the layout is moved over the portion that was not displayed in the function, functions 22442 to 22446 are executed by function 22440 as part of such movement.

  Function 22442 scrolls the previously hidden portion of the layout to the area corresponding to the overall overview window. In contrast, function 22444 scrolls the subparts of the layout displayed in the enlarged view window. Then, if necessary, the function 22448 sets the enlarged subportion marker to maintain a position-related correspondence in the overview window corresponding to the portion of the web page displayed in the enlarged view. Move.

  FIGS. 212 through 216 and FIGS. 225 through 228 relate to one feature of the present invention that limits the width of each column in a multi-column web page layout.

  This feature of the present invention can be used in conjunction with many features of the present invention described above, and in particular the magnified only view and split screen view described above with respect to FIGS. 219 and 221. Scroll the individual text columns to the web page so that you can read the entire text in each of the multiple lines of those columns without having to scroll to the right to read each line and then scroll to the left. / Has the advantage of being narrow enough to fit the range of the enlarged view.

  As is known to those skilled in web page programs, a web page has a column specification, including definition of what exists in the column, column width, and horizontal column replacement. . The predetermined column width and horizontal permutation is unlimited pixels depending on the content size of the predetermined column or the right or left column width of the predetermined column, for example as a percentage of the available display window size. It is defined in several ways, such as when a specification is specified. Such specifications may be explicit or implicit.

  FIG. 226 presents a specification example of three columns defined as an HTML table.

  In the figure, the line labeled 22602 is the first line of the table definition. Line 22604 begins the definition of a table row. This special table has only one row formed by three columns.

  Line 22606 defines a specification in which the width of the first column is 200 pixels. A line 22608 indicates the text that is the content of the first column.

  Line 22610 defines a central column and does not give a clearly specified width for that column, but the column width is automatically defined in the way the table is defined. This increases the column width to have all available widths in the display window not occupied by the other two columns. Line 22612 contains the contents of the middle column.

  Line 22614, like the first column, specifies the specification such that the third column has a width of 200 pixels. Finally, line 22616 specifies the content specification for the third column.

  FIG. 227 illustrates another method of defining a set of three columns similar to the columns defined in FIG. 226 using a reference to the external cascading style sheet (CSS) illustrated in FIG. Yes.

  FIG. 212 illustrates a part of the content of the web page displayed in FIGS. 213 to 216. FIG. 212 displays content arranged in a designated multi-column using one or more different methods that can define web page columns, such as the method illustrated in FIG. 226 or 227, for example. ing. In the figure, the vertical and horizontal boundaries of the column are indicated by gray lines.

  FIG. 213 is a simulation of the overview-only view 19200 of the type illustrated in FIG. 193 for the layout of the web page content illustrated in FIG. 212 when the column width is not limited.

  FIG. 214 is a simulation of the split view of the layout shown in FIG. The figure can be adapted to an enlarged view window 19400A of the kind already illustrated in FIGS. 195 to 199 because the width of the text line in the main text column of the web page is too wide. It is not shown. This means that reading each maximum line of text in the main column requires left and right scrolling, which tends to make such text reading slow and cumbersome.

  FIG. 215 is a simulation of an overview-only view 19200 of the same web page laid out using the column width restriction process described below with respect to FIG.

  FIG. 216 is a simulation regarding a divided view having the same layout as the layout in FIG. 215. FIG. 216 illustrates that the entire width of the column, which was the widest text column on the web page, is once placed in the enlarged view window 19400A of the split view prior to width limitation by the width restriction process of FIG. , Which makes it easier to read such text.

  In many preferred embodiments relating to this aspect of the invention, the text of such width limited columns has been selected to improve or optimize readability for the pixel resolution at which such text is displayed. , Displayed with anti-aliased font bitmaps having the shape and pixel arrangement of their character outlines. Such a font can be a sub-pixel optimized font of the type described above with respect to FIGS. 55-97 and FIGS. 168-184.

  In FIGS. 214 and 216, the enlarged view window 19400A is only an approximation of what such windows actually have, because this means that the fonts in those windows are rendered in the figure. This is because the fonts are not of the type discussed in the previous paragraph, having their character outline shape and pixel arrangement selected for the chosen resolution. This is the text in the enlarged view window of FIGS. 214 and 216, shown in FIGS. 194, 195 to 199, and FIGS. 208 to 211, respectively, and selected for the rendered pixel resolution. That is why the shape and pixel arrangement of the character outline is more difficult to read than the enlarged view windows 19400, 19400A, and 19400B text that the font bitmap has.

  FIG. 225 illustrates a series of functions 22500 that can be used to set the elements of a multi-column web page into a layout in which the width of each column is limited. Methods for laying out web page content are well known in the art. The function illustrated in FIG. 225 focuses on operations associated with the width limiting feature of the present invention.

  In the client proxy browser scheme described in FIGS. 217 through 224, the multi-column layout of FIG. 225 is performed at the proxy server as part of the layout process described by function 11507 of FIG. In other embodiments of the present invention, such a layout can be performed locally on the end user's computer displaying the requested web page without the need for any intermediary proxy computer. . As small computers acquire more memory capacity and computing power, it becomes more attractive to perform web page layout locally on the client.

  The function 22502 of FIG. 225 executes a loop for each column specified in the web page. This loop includes functions 22504 to 22524.

  Function 22504 lays out the column at a defined horizontal position as a function of two factors. Of these two factors, the first factor is the designated horizontal permutation of the column, as illustrated by line 22506. If a column width limit is selected, the second factor is the designated column of the column laid out to the left of the current column caused by such column width limits, as illustrated on lines 22508 and 22510. Any decrease in width.

  As used in this specification and the appended claims, a specified horizontal permutation of a column is its horizontal in terms of, for example, a fixed pixel position and percentage with respect to the width of the display screen. It includes definitions as to which columns are to the left and right of the column, as well as any unambiguous definitions for substitution. Thus, even if the column horizontal permutation is changed by reducing the column width on its left side, it still has the same column on its left side, so its position is still a function of that horizontal permutation.

  If the column width limit is selected, function 22512 causes functions 22514 and 22516 to reduce any image that is in the current column of loop 22502 to a size that fits within the maximum desired column width.

  In the illustrated embodiment, the desired maximum column width is the width that fits within the enlarged view 19400A illustrated in FIG. 216 when displayed. In other embodiments, the desired maximum column width is the width used in such embodiments and fits within a given display window, which may be a split screen or the entire screen window. .

  Function 22518 lays out the position of any text and / or image in the current column with a width determined as a function of the two factors indicated by lines 22520 through 22524. Of these factors, the first factor is the specified width of the column, as shown in text 22520. If a column width limit is selected, the second of these factors is necessary to ensure that the column width does not exceed the maximum desired column width, as indicated by the numbers 22522 and 22524. An arbitrary reduction of the specified width.

  The web page content illustrated in FIG. 212 includes a main text column 21204 at the top with two narrow text columns 21206 and an image column 21208 associated with each of those narrow text columns. . Although not shown in the pseudo code in FIG. 225, the column widths are not shown as shown in FIG. 215, even if their width is narrower than the maximum desired column width without such width restrictions. The restriction reduces the width of those two narrow text columns. This is done to reduce the width of the main column 21204 of the web page. However, in order to maintain readability and the overall appearance of the web page, the column width limit process is used to reduce the width of the comprehensive column that the text column or image is part of. It is also preferable to have a lower limit for the width for reducing the image.

  Because of this lower limit, these two small text columns and the main column 21204 containing the images associated with them are made wider than the maximum desired column width.

  Therefore, the main body of the text in the main column is narrower than the column width of the main column 21204, which is indicated by the double-headed arrow in FIG. 21504 is laid out.

  In other embodiments of the invention, no effort is expended to reduce or limit the width of image or non-text elements in a column, only the width of text in the column is maximized. Limited to the desired column width. This achieves a worthwhile result of shortening the text line to a size that fits within the desired view window, so that most of the many columns are of a width that is filled with text. Tend to have only parts, creating a blank similar to the blank 21506 illustrated in FIG.

  The pseudo code in FIGS. 217 through 225 is designed to display a web page on a low resolution display so that, even in a so-called “enlarged view window”, it reduces the web page content. To do. In this embodiment, the web page is laid out by the proxy server at a virtual resolution having a layout factor of 1 (ie, its elements are not reduced in the virtual layout), after which the virtual layout content is In the display list that is downloaded and displayed, it is reduced by a display magnification of 2.

  In other low-resolution embodiments, such reduction can be performed by performing the original layout at a layout magnification that reduces the content of the web page, after which the reduced layout is originally laid out. A display magnification of 1 is used so that it is displayed in scale.

  In such low resolution display embodiments, depending on the layout scale and / or display scale, the display of the layout of the web page is independent of the column width limit, most of the elements of the web page including images and text characters, or Display everything with reduced pixel resolution. In such an embodiment, the column width limit is so wide that even the reduction used to compress all of the other features related to the display of the web page can be adapted to the desired window. Narrow column width.

  In some embodiments relating to the features of the present invention discussed with respect to FIGS. 191 to 228, the layout and scale can both be unity, and when combined, the web page content is totally reduced. Not, possibly even having values that are even magnified.

  In other embodiments, the column width limiting process can be performed in other ways. For example, in some embodiments, comparing the width of a given column with the maximum desired width without width restrictions and, if necessary, the column width may be excessive. In order to prevent this, the column width limiting process can be achieved by narrowing the width of a given column. Perhaps it will reduce the attractiveness of many web pages, but in some embodiments, the column width limiting process is all about having the same width, regardless of the specified width of the column. Narrowing the column width can be included. Therefore, if there is no width limitation, it is desirable to display columns that are narrower than the maximum desired width with such narrower widths.

  In such embodiments, the exception to the rule that the column width is not reduced is when the column width is otherwise narrower than the maximum desired column width in the situation as illustrated with respect to FIG. is there. That is, if a large number of generated columns are arranged horizontally in rows within the entire column, the maximum desired column width will be exceeded if there is an exception that it itself contains text directly and there is no column width restriction An exception of having a width may occur. In such a case, as shown in FIG. 215, the columns in the horizontal row are substantially smaller than the maximum desired column width so that the overall column width can be reduced. The width may be reduced to a short length.

  FIG. 229 is presented to illustrate that the features of the present invention described above with respect to FIGS. 191 to 228 can also be applied to other types of computing devices in addition to mobile phones. FIG. 229 illustrates a split screen view simulation on a personal digital assistant (PDA). In this example, the combination of the specified layout of the illustrated web page and the larger pixel width of the PDA allows the widest text column to be displayed without requiring a width limit, so the column Although no width restriction is required, a split view is displayed and column width restriction is implemented. The split view includes an overview window 19200C, an enlarged view window 19400C, an enlarged view indicator 19504C in the overview window, and a cursor 19116 displayed in both the overview and the enlarged view.

  Of course, in other embodiments, some of the features of the invention illustrated in FIGS. 191 through 228 may include other types of computer devices or desktop computers, laptop computers, wrist-mounted computers. It can operate in other types of displays, including head mounted displays and displays in windows formed on subportions of larger display screens.

  Also, in other embodiments, other resolutions can be used for the display screen and can be used for various window sizes.

  It should be understood that the foregoing description and drawings are used for illustration and illustration only, and that the invention is not to be construed unless the interpretation of the appended claims is so limited. It is not limited. Those skilled in the art having the disclosure of information can make modifications and variations therein without departing from the scope of the invention.

  As broadly claimed, many features relating to the invention of this application are not limited to use with any one type of operating system, computer hardware, or computer network; The embodiments can use different software and hardware systems.

  Further, it should be understood that the functions recited in the following claims can be performed by a number of different programs and data structures using substantially different configurations or procedures. This is because a program is an extremely flexible technique that can be expressed in a virtually unlimited way once any complex predetermined idea is understood by those skilled in the art. Thus, the claims are not meant to be limited to the exact steps and / or order of steps described in the drawings. This is especially true in order to more effectively communicate what needs to be known by those skilled in the art to implement the present invention without burdening the user by unnecessarily providing details. This is because the pseudo code described in the above text is highly simplified. For such simplification, the pseudo code structure described above is often quite different from the actual code structure used by programmers skilled in the art in practicing the present invention. Furthermore, many of the programmed operations shown to be performed with software in the specification can be performed with hardware in other embodiments.

  In many embodiments relating to the features of the invention discussed above, various features of the invention that may occur separately in other embodiments relating to those features of the invention may occur simultaneously. It is shown.

  Most of the various drawings relating to subpixel optimization and non-linear color balance described in various parts of this specification relate to RGB subpixel addressable displays with vertical subpixel striping. Of course, many features of the present invention related to non-linear color balance and sub-pixel optimization, such as BGR or other types of sub-pixel addressability, as well as sub-pixel displays with horizontal sub-pixel striping. Can be used with subpixel displays.

  In the non-linear color balance method shown above, only the luminance portion of the sub-pixel distributed by the color balance is a portion higher than the minimum sub-pixel luminance value in the pixel. However, in other embodiments, other parts of the sub-pixel brightness that cause color imbalance within the pixel, such as, for example, the average sub-pixel brightness of the pixel or a part that differs from the maximum sub-pixel brightness are distributed. I can do it. In such embodiments, in fact, subpixel luminances below the average or maximum value result in negative luminance values that can be distributed with a weighted decrease in subpixel luminance in the vicinity of such subpixels.

  All the non-linear color balance methods illustrated above only distribute the portion of the subpixel luminance that causes color imbalance within the subpixel corresponding to the pixel. This is done because the array of three consecutive RGB or BGR subpixels commonly found within all pixels is perceptually well color balanced. If all such sub-pixels are of equal brightness, the surface is more colored on the surface compared to a separate set of the same three-color sub-pixels displayed at the same intensity in the order that the green is not the central color. Tend to seem to be balanced. This is one of the reasons why the edges of fonts that appear outside of all pixel boundaries appear to be color unbalanced.

  However, other non-linear color balance embodiments need not be limited to only distributing sub-pixel brightness that causes color imbalance within all individual pixels. Other non-linear color balance embodiments determine the percentage of sub-pixel color imbalance in regions other than all pixels and distribute sub-pixel luminance values based on imbalance in whole or in part to such regions. I can do it. For example, for each of the multiple unbalance patterns that commonly occur, conduct research to discover which distribution of unbalance coverage values produced the smallest spatial spread while maintaining color balance perception Such perceptually selected distributions can be used to distribute color imbalances that occur in spatial regions other than the full pixel region.

  One aspect of the invention relates to the generation and use of sub-pixel optimized images that calculate luminance values for individual pixels by line coverage techniques. Of course, other features of the invention claimed below are limited to such methods of determining sub-pixel brightness without specific details regarding such line coverage functions or area coverage functions. Rather, for example, other well known for determining coverage values using source images composed of color bitmaps, grayscale bitmaps, fonts, and other shapes, including but not limited to region sampling techniques Can be used.

  In the above discussion, the source image window used to assign luminance or coverage values in the subpixel optimized bitmap is rectangular and has a size that matches all pixels in the multicolor subpixel optimized image, And a size that matches the sub-pixel in the two-color sub-pixel optimized image. In other embodiments, windows of different shapes and sizes can be used. For example, in a multicolor subpixel optimized image, the source image window may have a size that is somewhat smaller than the size that matches all output image pixels. In some embodiments, an irregular weighting function can be used to convert a coverage value or luminance value in the source image window to a coverage value or luminance value in the output image. For example, in a multicolor subpixel optimized image, it is preferable to weight more for the luminance in the portion of the source image window whose size and position matches the subpixel that is determining the luminance. In fact, the line coverage arrangement discussed with respect to FIGS. 17-19 provides such central weighting, but only through the portion of the source image window that matches the position of the subpixel for which the line coverage value is being determined. This is because the vertical line extends.

  While some features of the present invention are clearly related to the use of subpixel optimization, many other features are not dependent on subpixel optimization. In some such features of the invention, an anti-aliasing form that does not involve sub-pixel optimization can be used. A form of anti-aliasing that does not include sub-pixel optimization can make the image appear to have a higher resolution than can be provided without using such anti-aliasing. This is especially true for font images. For example, proper subpixel optimization makes small fonts easier to read, but the font has the right shape and is properly hinted, with or without subpixel optimization When anti-aliasing is used, a font as small as 7 pixels / em can be read relatively easily.

  In this specification and the appended claims, by referring to a “screen”, particularly a screen on which a reduced image, text, or web page layout is displayed, in general, for example, a screen. It can include either the entire screen or a portion of the screen, such as the graphic window above. For example, the referenced reduced screen image may be displayed in a fairly large on-screen window, or there may be space in certain graphical user interface (GUI) elements, such as the toolbar illustrated in FIG. You may display on a part of small screen left after providing. Also, of course, beyond all or a significant portion of the large screen, so that the content can be viewed at a higher spatial resolution, eg, enabled by a large screen, using non-subpixel optimization techniques. Thus, certain sub-pixel optimization features of the present invention can be used to display images and / or text.

In particular, used in accordance with several features of the present invention to enhance web browsing and / or other types of computer generated content display on systems with relatively low resolution screens. FIG. 1 illustrates a network computing environment in which the present invention can be implemented having a portable browser, a proxy server, a web server, and a font server. 1 illustrates an alternative network computing environment in which the present invention can be implemented having a browser and a web server. FIG. FIG. 4 illustrates a second alternative network computing environment in which the present invention can be implemented having a browser and a web server. FIG. 6 illustrates a third alternative network computing environment in which the present invention can be implemented having a browser and a web server. 1 is a diagram illustrating a computer system in which the present invention can be implemented having standard web content to be displayed and browser functionality including processing for scaling the content and / or sub-pixel optimization. FIG. The figure which shows the alternative computer system which can perform this invention which has the proxy processing for the content to be displayed, the expansion / contraction of the content, and / or subpixel optimization, and a browser function. FIG. 5 shows a second alternative computer system in which the present invention with previously scaled and / or sub-pixel optimized content can be implemented. The figure which shows the conventional vertical stripe type | mold RGB liquid crystal display device. FIG. 2 illustrates some features of the present invention involved in performing subpixel optimization for both images and text, referenced with respect to each of steps 108 and 112 of FIG. FIG. 6 illustrates the level of readability provided by the current embodiment of the present invention when displaying standard web content on a 320 × 240 color display. FIG. 6 shows pixel and sub-pixel grid mapping performed over a portion of a higher resolution source bitmap image 102. FIG. 13 is an enlarged view of a part of the mapping grid of FIG. 12. FIG. 6 shows the positioning of a window on a source image used to calculate the luminance of a red (R) subpixel of a low resolution display device. FIG. 4 shows the positioning of such a window used to calculate the brightness of the green (G) subpixel of a low resolution display device. FIG. 6 shows the positioning of such a window used to calculate the luminance of the blue (B) subpixel of a low resolution display device. In a higher resolution source image covered by one or more different color pixels, a red sub-pixel on a display device having a relatively low resolution is estimated by estimating a portion of the associated window of red pixels. The figure explaining the scan line used with the scan line coverage method for calculating the sub pixel brightness | luminance regarding a pixel. FIG. 18 illustrates a similar scan line used to calculate the luminance of the green (G) subpixel of the low resolution display device illustrated in FIG. 17. FIG. 19 shows a similar scan line used to calculate the luminance of the blue (B) subpixel of the low resolution display device shown in FIGS. 17 and 18. The figure which repeats FIG. 17 shown on the same sheet | seat as FIG. 21 and 22 for a simple comparison. FIG. 21 illustrates portions of the horizontal scan line illustrated in FIG. 20 covered with different source image pixels within a red pixel source image window. FIG. 21 illustrates portions of the vertical scan line illustrated in FIG. 20 covered with different source image pixels in a red pixel source image window. FIG. 26 is the same as FIG. 18 and is shown on the same sheet as FIGS. 24 and 25 for simple comparison. FIG. 24 illustrates portions of the horizontal scan line illustrated in FIG. 23 covered with different source image pixels in a green pixel source image window. FIG. 24 illustrates portions of the vertical scan line illustrated in FIG. 23 covered with different source image pixels within a green pixel source image window. FIG. 28 is the same as FIG. 19 and is shown on the same sheet as FIGS. 27 and 28 for simple comparison. FIG. 27 illustrates portions of the horizontal scan line illustrated in FIG. 26 covered with different source image pixels within a blue pixel source image window. FIG. 27 illustrates portions of the vertical scan line illustrated in FIG. 26 covered with different source image pixels within a blue pixel source image window. For example, a highly simplified pseudocode description for a subpixel optimization method that calculates subpixel luminance values based on line coverage values such as the line coverage values described with respect to FIGS. In an alternative embodiment for a “line coverage” method for calculating pixel color in a sub-pixel optimized and scaled image, two horizontal scan lines and two vertical scan lines are used. The figure which shows how it can be used. How to use two diagonal scanlines in an alternative embodiment for a "line coverage" method for calculating pixel color in sub-pixel optimized and scaled images The figure which shows whether it can do. In an alternative embodiment for a “line coverage” method for calculating pixel color in a sub-pixel optimized image, two diagonal scan lines, one horizontal scan line, and one The figure which shows how the combination of the vertical scan line of can be used. The figure which shows the line coverage with respect to two horizontal coverage lines in a half horizontal and vertical scaling. FIG. 34 is a diagram illustrating line coverage for two vertical coverage lines in the half horizontal and vertical scaling illustrated in FIG. 33. The figure which shows the line coverage with respect to two horizontal coverage lines in about 2/5 horizontal and vertical scaling. FIG. 36 is a diagram illustrating line coverage for two vertical coverage lines in about two-fifth horizontal and vertical scaling illustrated in FIG. The figure which shows the line coverage with respect to two horizontal coverage lines in about 2/3 horizontal and vertical scaling. FIG. 38 is a diagram illustrating line coverage for two vertical coverage lines in about two-thirds horizontal and vertical scaling illustrated in FIG. FIG. 6 shows a source image pixel window used in an “area coverage” method for calculating color values of a sub-pixel optimized and scaled image. According to one such “area coverage” method, in the source image window that is used to calculate the luminance value of the sub-pixel, it uses different hatching to indicate areas of different source image pixels. FIG. 40 is similar to FIG. 39 except for FIG. FIG. 41 illustrates a highly simplified pseudocode description for a subpixel optimization method that calculates subpixel luminance values based on, for example, the area coverage values discussed with respect to FIGS. 39 and 40. Figure 8 illustrates a source image window and associated scan lines that can be used in generating a scaled two-color subpixel optimized image for a bitmap image to associate a luminance value with a red subpixel. Figure. FIG. 6 illustrates a source image window and associated scan lines that can be used in generating a scaled two-color sub-pixel optimized image for a bitmap image to associate a luminance value with a green sub-pixel. Figure. Figure 8 illustrates a source image window and associated scan lines that can be used in generating a scaled two-color subpixel optimized image for a bitmap image to associate a luminance value with a blue subpixel. Figure. FIG. 6 illustrates the brightness of a series of source image grayscale pixels associated with a portion of an RGB subpixel display grid. FIG. 46 illustrates how the luminance associated with the individual subpixels illustrated in FIG. 45 is distributed with a conventional linear filtering method. FIG. 47 is a diagram illustrating subpixel luminance values resulting from the conventional linear filtering method illustrated in FIG. 46 applied to multiple subpixels in a pixel row. FIG. 6 shows the distribution of minimum subpixel luminance values under non-linear filtering. FIG. 49 is a diagram illustrating distribution of excess luminance values under the nonlinear filtering method illustrated in FIG. 48. FIG. 3 compares the original source pixel brightness with the results of linear and nonlinear filtering methods. FIG. 3 compares the original source pixel brightness with the results of linear and nonlinear filtering methods. FIG. 3 compares the original source pixel brightness with the results of linear and nonlinear filtering methods. A highly simplified pseudo-code description of a software method for creating a sub-pixel optimized representation of a two-color bitmap. The flowchart regarding the process in which a user can trade off color and positional resolution dynamically. FIG. 4 shows a mapping between a character font shape defined by an outline font description and a pixel array with subpixels used to represent the shape on a display that is addressable with subpixels. 4 is a screen shot of a 320 × 240 screen of a web page created in one embodiment of the present invention. FIG. 57 is a double enlarged view of the screen shot shown in FIG. 56. FIG. 4 shows how a computer can access a font bitmap or a font outline from a font server. FIG. 4 is a diagram showing how a computer can access a font bitmap or a font outline stored in the computer. A highly simplified pseudo-code representation for an algorithm for calculating a sub-pixel optimized bitmap of a character font shape using a non-linear color balance of the type described in FIGS. For example, the source image window in the character font shape image shown in FIG. 55, for example, used to determine the coverage value for each of the three sub-pixels of the individual pixels for the pixel array shown in FIG. The figure which shows size. For example, the source image window in the character font shape image shown in FIG. 55, for example, used to determine the coverage value for each of the three sub-pixels of the individual pixels for the pixel array shown in FIG. The figure which shows size. For example, the source image window in the character font shape image shown in FIG. 55, for example, used to determine the coverage value for each of the three sub-pixels of the individual pixels for the pixel array shown in FIG. The figure which shows size. FIG. 3 shows some prior art techniques used to calculate coverage values for non-square rasterized units (usually all pixels in the prior art). FIG. 3 shows some prior art techniques used to calculate coverage values for non-square rasterized units (usually all pixels in the prior art). FIG. 3 shows some prior art techniques used to calculate coverage values for non-square rasterized units (usually all pixels in the prior art). FIG. 3 shows some prior art techniques used to calculate coverage values for non-square rasterized units (usually all pixels in the prior art). An efficient computer-based method for calculating the coverage value of a rasterization unit using the weighted line coverage value used in some embodiments of the present invention for calculating the coverage value for a subpixel. FIG. An efficient computer-based method for calculating the coverage value of a rasterization unit using the weighted line coverage value used in some embodiments of the present invention for calculating the coverage value for a subpixel. FIG. An efficient computer-based method for calculating the coverage value of a rasterization unit using the weighted line coverage value used in some embodiments of the present invention for calculating the coverage value for a subpixel. FIG. An efficient computer-based method for calculating the coverage value of a rasterization unit using the weighted line coverage value used in some embodiments of the present invention for calculating the coverage value for a subpixel. FIG. An efficient computer-based method for calculating the coverage value of a rasterization unit using the weighted line coverage value used in some embodiments of the present invention for calculating the coverage value for a subpixel. FIG. An efficient computer-based method for calculating the coverage value of a rasterization unit using the weighted line coverage value used in some embodiments of the present invention for calculating the coverage value for a subpixel. FIG. An efficient computer-based method for calculating the coverage value of a rasterization unit using the weighted line coverage value used in some embodiments of the present invention for calculating the coverage value for a subpixel. FIG. An efficient computer-based method for calculating the coverage value of a rasterization unit using the weighted line coverage value used in some embodiments of the present invention for calculating the coverage value for a subpixel. FIG. An efficient computer-based method for calculating the coverage value of a rasterization unit using the weighted line coverage value used in some embodiments of the present invention for calculating the coverage value for a subpixel. FIG. An efficient computer-based method for calculating the coverage value of a rasterization unit using the weighted line coverage value used in some embodiments of the present invention for calculating the coverage value for a subpixel. FIG. An efficient computer-based method for calculating the coverage value of a rasterization unit using the weighted line coverage value used in some embodiments of the present invention for calculating the coverage value for a subpixel. FIG. An efficient computer-based method for calculating the coverage value of a rasterization unit using the weighted line coverage value used in some embodiments of the present invention for calculating the coverage value for a subpixel. FIG. An efficient computer-based method for calculating the coverage value of a rasterization unit using the weighted line coverage value used in some embodiments of the present invention for calculating the coverage value for a subpixel. FIG. An efficient computer-based method for calculating the coverage value of a rasterization unit using the weighted line coverage value used in some embodiments of the present invention for calculating the coverage value for a subpixel. FIG. An efficient computer-based method for calculating the coverage value of a rasterization unit using the weighted line coverage value used in some embodiments of the present invention for calculating the coverage value for a subpixel. FIG. An efficient computer-based method for calculating the coverage value of a rasterization unit using the weighted line coverage value used in some embodiments of the present invention for calculating the coverage value for a subpixel. FIG. An efficient computer-based method for calculating the coverage value of a rasterization unit using the weighted line coverage value used in some embodiments of the present invention for calculating the coverage value for a subpixel. FIG. An efficient computer-based method for calculating the coverage value of a rasterization unit using the weighted line coverage value used in some embodiments of the present invention for calculating the coverage value for a subpixel. FIG. An efficient computer-based method for calculating the coverage value of a rasterization unit using the weighted line coverage value used in some embodiments of the present invention for calculating the coverage value for a subpixel. FIG. An efficient computer-based method for calculating the coverage value of a rasterization unit using the weighted line coverage value used in some embodiments of the present invention for calculating the coverage value for a subpixel. FIG. FIG. 88 illustrates some of the other arrangements for coverage lines that can be used with the general type of weighted line coverage algorithm described above with respect to FIGS. FIG. 88 illustrates some of the other arrangements for coverage lines that can be used with the general type of weighted line coverage algorithm described above with respect to FIGS. FIG. 88 illustrates some of the other arrangements for coverage lines that can be used with the general type of weighted line coverage algorithm described above with respect to FIGS. The figure which shows the pixel arrangement | positioning to the one part image of a virtual font outline, and the mapping of each sub pixel. FIG. 92 is a diagram illustrating corresponding coverage values calculated for the sub-pixels in the array illustrated in FIG. 91. Corresponding to FIG. 46, as shown in FIG. 46, the prior art linear color balance method calculates all of the individual subpixel coverage values for a series of adjacent subpixels within a given pixel row. The figure which shows how to distribute. FIG. 61 illustrates a color balance filter that can be used using the non-linear color balance method described with respect to FIG. FIG. 61 illustrates a color balance filter that can be used using the non-linear color balance method described with respect to FIG. For example, an algorithm that can be used to represent the alpha value of all pixels calculated for a font bitmap in the more limited color space of the alpha value of all pixels by the method illustrated in FIG. Highly simplified pseudo code description. A highly simplified pseudo-code description for an algorithm for displaying text strings in a subpixel addressable display using a font bitmap created by a combination of the methods illustrated in FIGS. 60 and 96 . 98 and 100, each of which is a screen shot of a 640 × 480 layout of a different web page, and FIGS. 99 and 99 showing how the present invention can display each of these two web pages on a 320 × 240 screen. FIG. 101 is a diagram showing how the present invention can properly display a web page on a 320 × 240 screen using FIG. 101. 98 and 100, each of which is a screen shot of a 640 × 480 layout of a different web page, and FIGS. 99 and 99 showing how the present invention can display each of these two web pages on a 320 × 240 screen. FIG. 101 is a diagram showing how the present invention can properly display a web page on a 320 × 240 screen using FIG. 101. 98 and 100, each of which is a screen shot of a 640 × 480 layout of a different web page, and FIGS. 99 and 99 showing how the present invention can display each of these two web pages on a 320 × 240 screen. FIG. 101 is a diagram showing how the present invention can properly display a web page on a 320 × 240 screen using FIG. 101. 98 and 100, each of which is a screen shot of a 640 × 480 layout of a different web page, and FIGS. 99 and 99 showing how the present invention can display each of these two web pages on a 320 × 240 screen. FIG. 101 is a diagram showing how the present invention can properly display a web page on a 320 × 240 screen using FIG. 101. Schematic block for several data structures and programs used by proxy servers and thin client computers to reduce the thin client computer user's access to web content on a subpixel optimized screen Figure. Part of the HTML code of the web page described in FIG. 98 and FIG. FIG. 99 is a diagram showing a layout of a web page created by the proxy server and a part of the layout that fits in the virtual screen of the proxy server corresponding to the part of the web page shown in FIG. 99 in this example. A highly simplified pseudo-code description for the program on the proxy server illustrated in FIG. A highly simplified pseudo-code description for the program on the proxy server illustrated in FIG. A highly simplified pseudo-code description for a proxy server program to capture, reduce, sub-pixel optimize, and download a representation to a portion of a web page. A highly simplified pseudo-code description for a proxy server program to capture, reduce, sub-pixel optimize, and download a representation to a portion of a web page. A highly simplified pseudo-code description for a proxy server program to capture, reduce, sub-pixel optimize, and download a representation to a portion of a web page. 106. A highly simplified pseudo code description for a program for actually downloading a representation of a web page captured by the program of FIGS. 106A-106C. A highly simplified representation of the data downloaded to the thin client computer by the program described in FIG. FIG. 102 is a highly simplified representation of pseudo code for the program in the thin client illustrated in FIG. FIG. 102 is a highly simplified representation of pseudo code for the program in the thin client illustrated in FIG. FIG. 102 is a highly simplified representation of pseudo code for the program in the thin client illustrated in FIG. Describes how a pop-up keyboard is displayed that allows the user to enter text in the text entry field when the user clicks on the text entry field on the web page display in the thin client illustrated in FIG. To do. Describes how a pop-up keyboard is displayed that allows the user to enter text in the text entry field when the user clicks on the text entry field on the web page display in the thin client illustrated in FIG. To do. Describes how a pop-up keyboard is displayed that allows the user to enter text in the text entry field when the user clicks on the text entry field on the web page display in the thin client illustrated in FIG. To do. The figure explaining how a user can use the same pop-up keyboard in order to input URL which a user can access to a desired web page. Except for describing an embodiment of the present invention having a toolbar at the top of a thin client computer screen that includes a graphical user interface (GUI) button and a text input field for a URL, it is substantially the same as FIG. A certain figure. In each of the proxy server and thin client computer used in an alternative embodiment of the present invention that preserves the layout of the entire web page so that the client computer can scroll and zoom more quickly in relation to the web page. A highly simplified pseudo-code representation of programs In each of the proxy server and thin client computer used in an alternative embodiment of the present invention that preserves the layout of the entire web page so that the client computer can scroll and zoom more quickly in relation to the web page. A highly simplified pseudo-code representation of programs 119 is a schematic diagram used to help explain the operation of the pseudo code described in FIGS. 115 and 116; FIG. FIG. 4 illustrates how the present invention allows a user to quickly select a portion of a web page or screen and then zoom to the selected portion. FIG. 4 illustrates how the present invention allows a user to quickly select a portion of a web page or screen and then zoom to the selected portion. FIG. 4 illustrates how the present invention allows a user to quickly select a portion of a web page or screen and then zoom to the selected portion. The figure explaining one characteristic regarding the zoom click of this invention which a user can visually recognize a part of a web page or a screen more correctly, and can select. The figure explaining one characteristic regarding the zoom click of this invention which a user can visually recognize a part of a web page or a screen more correctly, and can select. The figure explaining one characteristic regarding the zoom click of this invention which a user can visually recognize a part of a web page or a screen more correctly, and can select. The figure explaining one characteristic regarding the zoom click of this invention which a user can visually recognize a part of a web page or a screen more correctly, and can select. The figure explaining one characteristic regarding the zoom click of this invention which a user can visually recognize a part of a web page or a screen more correctly, and can select. The figure explaining one characteristic regarding the zoom click of this invention which a user can visually recognize a part of a web page or a screen more correctly, and can select. The figure explaining one characteristic regarding the zoom click of this invention which a user can visually recognize a part of a web page or a screen more correctly, and can select. The figure explaining one characteristic regarding the zoom click of this invention which a user can visually recognize a part of a web page or a screen more correctly, and can select. A highly simplified pseudo-code description for a program that allows the user to select text that has been reflowed over a given screen at a large magnification. 129 is a diagram used to describe how the text reflow function illustrated in FIG. 129 is performed; FIG. 129 is a diagram used to describe how the text reflow function illustrated in FIG. 129 is performed; FIG. 129 is a diagram used to describe how the text reflow function illustrated in FIG. 129 is performed; FIG. 129 is a diagram used to describe how the text reflow function illustrated in FIG. 129 is performed; FIG. 129 is a diagram used to describe how the text reflow function illustrated in FIG. 129 is performed; FIG. 129 is a diagram used to describe how the text reflow function illustrated in FIG. 129 is performed; FIG. 129 is a diagram used to describe how the text reflow function illustrated in FIG. 129 is performed; FIG. 129 is a diagram used to describe how the text reflow function illustrated in FIG. 129 is performed; FIG. FIG. 5 is used to illustrate how multiple client computers can be programmed to access a common font server and / or a common proxy server. A highly simplified pseudo code description of a program that can be used on one or more font servers. On a remote computer in a reduced, sub-pixel optimized manner by an interrupt request to the remote computer's operating system executed by an application (which can include, but is not limited to, one or more web browsers) FIG. 6 illustrates how certain features of the present invention can be used to make a client computer visible to a screen rendered by such an application running on the computer. An application running on a given computer (which can include, but is not limited to, one or more web browsers) is sub-pixel optimized on a given computer screen to produce a reduced view How can a subpixel-optimized, reduced view have a screen output generated by such an application program due to an interrupt request to the computer's operating system executed by such application, even if not programmed for The figure explaining whether it can do. Certain features relating to the present invention enable portable small screen thin client computers to access web content and / or screen output of various application programs via local and / or Internet wireless communications. The figure explaining how it can be used. In some embodiments relating to the present invention, by rotating a computing device having an operating system programmed to run in the vertical direction, how the sub-pixel optimized output is displayed in the horizontal direction The figure used for explaining. In some embodiments relating to the present invention, by rotating a computing device having an operating system programmed to run in the vertical direction, how the sub-pixel optimized output is displayed in the horizontal direction The figure used for explaining. A highly simplified pseudo-code description for a program used to draw a simple shape with subpixel optimization resolution. A highly simplified pseudo-code description of how a web applet can be used to draw sub-pixel optimized elements on a computer screen. A highly simplified block diagram illustrating how a rollover image can be sub-pixel optimized. A highly simplified block diagram illustrating how a GIFF animation can be sub-pixel optimized. A highly simplified pseudocode description of how 3D animation can be sub-pixel optimized. A highly simplified pseudo-code description of how a client / server game system can be used to provide a sub-pixel optimized game image on a client computer. A highly simplified pseudo-code description of how a client / server game system can be used to provide sub-pixel optimized game images on a client computer. A highly simplified pseudo-code description of how a subpixel optimized display can be constructed with an image having a corresponding transparency map. A highly simplified pseudo-code description of how video using interpolation between keyframes can be sub-pixel optimized. A highly simplified pseudo-code description of how a sub-pixel can be optimized for an image that includes drawing screen changes to parts where the representation is less than a full frame. A highly simplified pseudo-code description of different ways to display moving images relative to a display window. A highly simplified pseudo-code description of different ways to display moving images relative to a display window. A highly simplified pseudo-code representation of how sub-pixel optimization can be applied to video represented by various compression techniques. A highly simplified pseudo-code representation of how sub-pixel optimization can be applied to video represented by various compression techniques. A highly simplified pseudo-code representation of how sub-pixel optimization can be applied to video represented by various compression techniques. A highly simplified pseudo-code notation for a program that allows a server computer to download sub-pixel optimized and reduced video to a client computer. Highly simplified for programs that allow client computers and proxy computers to access reduced, sub-pixel optimized video from other servers via proxy computers. Pseudo code notation. Highly simplified pseudo-code notation for programs that allow you to see electronic ink more clearly. 162 is a diagram used to help explain the advantages of the program described with respect to FIG. 162. FIG. 162 is a diagram used to help explain the advantages of the program described with respect to FIG. 162. FIG. 162 is a diagram used to help explain the advantages of the program described with respect to FIG. 162. FIG. 162 is a diagram used to help explain the advantages of the program described with respect to FIG. 162. FIG. The invention relates not only to methods but also to methods stored in machine-readable form or embodied in a propagated signal, and to carry out such methods and / or such programs and / or FIG. 6 illustrates things related to a programmed computer system for using data and / or a computer system embedded in hardware. FIG. 4 is used to illustrate a further improvement to the present invention to improve the clarity of a color balanced, sub-pixel optimized font bitmap created by the present invention. FIG. 4 is used to illustrate a further improvement to the present invention to improve the clarity of a color balanced, sub-pixel optimized font bitmap created by the present invention. FIG. 4 is used to illustrate a further improvement to the present invention to improve the clarity of a color balanced, sub-pixel optimized font bitmap created by the present invention. FIG. 4 is used to illustrate a further improvement to the present invention to improve the clarity of a color balanced, sub-pixel optimized font bitmap created by the present invention. FIG. 4 is used to illustrate a further improvement to the present invention to improve the clarity of a color balanced, sub-pixel optimized font bitmap created by the present invention. FIG. 5 is used to illustrate a further improvement to the present invention to improve the clarity of a color balanced, sub-pixel optimized font bitmap created by the present invention. FIG. 4 is used to illustrate a further improvement to the present invention to improve the clarity of a color balanced, sub-pixel optimized font bitmap created by the present invention. FIG. 5 is used to illustrate a further improvement to the present invention to improve the clarity of a color balanced, sub-pixel optimized font bitmap created by the present invention. FIG. 4 is used to illustrate a further improvement to the present invention to improve the clarity of a color balanced, sub-pixel optimized font bitmap created by the present invention. FIG. 4 is used to illustrate a further improvement to the present invention to improve the clarity of a color balanced, sub-pixel optimized font bitmap created by the present invention. FIG. 4 is used to illustrate a further improvement to the present invention to improve the clarity of a color balanced, sub-pixel optimized font bitmap created by the present invention. FIG. 5 is used to illustrate a further improvement to the present invention to improve the clarity of a color balanced, sub-pixel optimized font bitmap created by the present invention. FIG. 5 is used to illustrate a further improvement to the present invention to improve the clarity of a color balanced, sub-pixel optimized font bitmap created by the present invention. FIG. 5 is used to illustrate a further improvement to the present invention to improve the clarity of a color balanced, sub-pixel optimized font bitmap created by the present invention. FIG. 5 is used to illustrate a further improvement to the present invention to improve the clarity of a color balanced, sub-pixel optimized font bitmap created by the present invention. FIG. 5 is used to illustrate a further improvement to the present invention to improve the clarity of a color balanced, sub-pixel optimized font bitmap created by the present invention. FIG. 5 is used to illustrate a further improvement to the present invention to improve the clarity of a color balanced, sub-pixel optimized font bitmap created by the present invention. A higher level description of the selected text reflow method described with respect to FIGS. A high-level pseudo-code description for the general kind of zoom-to-fit method described with respect to FIGS. A high-level pseudo-code description of the drag scroll method that allows the user to easily navigate within the display of the web page layout. A high-level pseudo-code description for the click-zoom method that allows the user to quickly select zoom-in on the desired portion of the web page layout display. A highly simplified pseudocode description for the zoom click method described with respect to FIGS. 121-128. A highly simplified pseudo-code description of how a user can view a zoomed-out display of a web page that uses greeking. FIG. 219 is a top view of a mobile phone compatible with the Microsoft smartphone specification that can be used to generate the screenshots illustrated in FIGS. 192-216 on a 220 × 176 pixel screen. A reduced overview of the web page is displayed, a different menu display can be selected by the user, and a menu that can perform other functions common to web browsing is displayed in FIG. A screen shot illustrating an image of the screen of the illustrated mobile phone. 195 is a screenshot illustrating an overview of a web page that is identical to the web page overview in FIG. 192 without a menu being displayed. The overview illustrated in FIG. 193, which actually displays many web page elements in an enlarged view, approximately half the size normally found on a desktop or laptop computer. Screenshot illustrating a magnified view of a portion of. The top of the screen illustrates the overview window having the view scale illustrated in FIG. 193, and the bottom of the screen illustrates the enlarged view window having the scale illustrated in FIG. 194. Screenshot showing the split view shown. 195 is a screenshot illustrating the same screenshot as that of FIG. 195 for simple comparison with FIGS. 197 to 199 existing on the same sheet. 195 is a screenshot illustrating a view in an enlarged view that occurs when the user navigates the display in the window to the right from the position illustrated in FIG. 196 is a screenshot showing the view in the enlarged view window that occurs when the user navigates the display in the window downward from the position in FIG. Illustrates the view in the enlarged view window that occurs when the user navigates the view in that window further down from the position in FIG. 198 to scroll the display in the overview window. Screenshot to do. The same screenshot as that of FIG. 198 provided for simple comparison with FIGS. 201 to 203 residing on the same sheet. Although the top window is the overview window seen in FIG. 200, the bottom window is the width of the screen so that a selected portion of the text shown in the overview window can be read more easily. A simulated screenshot illustrating a split view, a reflowed text window that is reflowing with a larger font size in relation to. 201 is a screenshot illustrating a split view of the type illustrated in FIG. 201 after the user scrolls down in the reflowed text column. FIG. 201 is a screen shot illustrating the same split view as the split view of FIG. 201 after the user has scrolled to the right, displaying text in the next column on the right side in a reflowed text window. A screen shot identical to the screen shot of FIG. 200, provided for simple comparison with FIGS. 205-207 residing on the same sheet. FIG. 205 is a simulated screenshot illustrating a nearly full screen reflowed text view of the text in the column closest to the cursor position in FIG. 204. A simulated screenshot illustrating such a reflowed text view after the user scrolls down in the text column illustrated in FIG. FIG. 5 is a simulated screenshot illustrating such a reflowed text view after the user has scrolled to the right, displaying text on the right side from the next column of the web page. 209 is a screenshot illustrating a magnifying glass view of the same web page as illustrated in FIGS. 191 to 207, with the magnified view window displayed beyond a portion of the overview window. A screen shot illustrating the magnifier view after the user has scrolled the portion of the layout displayed in the magnified view to the right in relation to the view in FIG. 209 is a screenshot illustrating a magnifying glass view after the user scrolls down a portion of the layout displayed in the enlarged view in relation to the view in FIG. FIG. 23 is a screenshot illustrating the magnifier view after the user has scrolled further down the portion of the layout displayed in the magnified view in relation to the view in FIG. FIG. 3 is a representation of a portion of the content of a web page and illustrates the side, top, and / or bottom of many of its columns with thin gray lines. FIG. 23 is a simulated screenshot illustrating how the web page content illustrated in FIG. 212 may be laid out if column width restrictions are not used. FIGS. 195 to 199 are simulated screen shots illustrating the general types of split screen views illustrated in FIGS. 195 to 199, and the layout of FIG. The figure which illustrates that it cannot fit in an enlarged display window at once because it is too wide. A simulated screenshot illustrating how the web page content illustrated in FIG. 212 can be laid out using column width restrictions. FIG. 214 is a screen shot illustrating the same type of split screen view as shown in FIG. 214, and when using column width restrictions, the line width of the main part of the text on the web page is expanded into a view window enlarged at once Figure illustrating that it is narrow enough to fit. A highly simplified pseudocode display of programmed functions that can be used to create the screenshots illustrated in FIGS. 191 through 211 and FIGS. 213 through 216. A highly simplified pseudocode display of programmed functions that can be used to create the screenshots illustrated in FIGS. 191 through 211 and FIGS. 213 through 216. A highly simplified pseudocode display of programmed functions that can be used to create the screenshots illustrated in FIGS. 191 through 211 and FIGS. 213 through 216. A highly simplified pseudocode display of programmed functions that can be used to create the screenshots illustrated in FIGS. 191 through 211 and FIGS. 213 through 216. A highly simplified pseudocode display of programmed functions that can be used to create the screenshots illustrated in FIGS. 191 through 211 and FIGS. 213 through 216. A highly simplified pseudocode display of programmed functions that can be used to create the screenshots illustrated in FIGS. 191 through 211 and FIGS. 213 through 216. A highly simplified pseudocode display of programmed functions that can be used to create the screenshots illustrated in FIGS. 191 through 211 and FIGS. 213 through 216. A highly simplified pseudocode display for programmed functions that can be used to create the screenshots illustrated in FIGS. 191 through 211 and FIGS. 213 through 216. A highly simplified pseudocode display of programmed functions that can be used to create the screenshots illustrated in FIGS. 191 through 211 and FIGS. 213 through 216. A highly simplified pseudocode display of programmed functions that can be used to create the screenshots illustrated in FIGS. 191 through 211 and FIGS. 213 through 216. FIG. 6 illustrates two of the many ways in which web page content can provide content, width, and specifications for horizontal replacement for a series of columns arranged in a horizontal direction. FIG. 3 illustrates two of the many ways in which web page content can provide content, width, and specifications for horizontal replacement for a series of columns arranged in a horizontal direction. FIG. 3 illustrates two of the many ways in which web page content can provide content, width, and specifications for horizontal replacement for a series of columns arranged in a horizontal direction. The figure which illustrates that the characteristic regarding this invention can be used on computers other than a mobile telephone, such as a personal digital assistant (PDA) illustrated in the figure.

Claims (70)

Accepting digital content indicating the web page to be displayed;
Laying out the web page;
Displaying the layout portion of the web page in the overview window at a first magnification, so that multiple elements of the layout appear larger in the enlarged view window than in the overview window; Displaying the subpart of the layout portion displayed in the overview window in the enlarged view window using the display in the enlarged view window in the second magnification state; and Simultaneously displaying in the overview window an enlarged view indicator indicating the subpart of the layout portion displayed in the overview window currently displayed in the overview window. Web page split And forming the-menu,
Moving the subpart of the layout displayed in the enlarged view window, and moving the enlarged view indicator relative to the layout displayed in the overview window The subpart of the layout displayed in the enlarged view window in response to a navigation input from a user is transferred to the subpart of the layout indicated by the enlarged view indicator after such movement. Continuously responding,
The overview window and the enlarged view window are simultaneously displayed on different portions of a single rectangular display screen having a predetermined width and height;
A method of displaying media on a computing device, wherein each of the overview window and the enlarged view window is approximately the same width as the screen but each occupies a different portion of the screen height.   The media display method according to claim 1, wherein the height of the enlarged view window occupies less than half of the combined height of the enlarged view window and the overview window.   The method of claim 1, wherein the screen has a vertical direction relative to a direction in which the web page layout is displayed in the enlarged view window and the overview window.   Each of the overview window and the enlarged view window is longer in the horizontal direction than the vertical direction, so that the screen on which both the overview window and the enlarged view window are displayed is in the vertical direction. 4. The method of displaying media in a computing device according to claim 3, wherein each of the overview window and the enlarged view window has a horizontal direction in spite of the fact that the display window has a horizontal direction.   The media display method according to claim 1, wherein the divided view is displayed on a display screen of a personal digital assistant (PDA).   The method of displaying media on a computing device according to claim 1, wherein the divided view is displayed on a display screen of a mobile phone. Accepting digital content indicating the web page to be displayed;
Laying out web pages,
Displaying the layout portion of the web page in the overview window at a first magnification and making the elements of the layout appear larger in the enlarged view window than in the overview window. Using the display in the enlarged view window in a double magnification state to display sub-parts of the layout portion displayed in the overview window in the enlarged view window; and The enlarged view indicator showing the subpart of the layout part displayed in the overview window currently displayed in the window is displayed in the overview window, thereby simultaneously providing a split view. First of the web page By forming the-menu, and correspond to the first view selection input from the user,
Moving the subparts of the layout displayed in the enlarged view window when the split view is displayed, as opposed to the layout displayed in the overview window; Responsive to navigation input from the user by moving the magnified view indicator so that the subparts of the layout displayed in the magnified view window are moved by the magnified view indicator after such movement Continuously responding to the indicated subparts of the layout;
Responding to a second view selection input from a user by forming a second view that does not have a combination of the split views with respect to the overview window and the magnified view window. Media display method. The layout portion of the web page is displayed at a second magnification that allows multiple elements of the layout to appear smaller than in the enlarged window;
The overall display size is approximately equal to the combined size of the overview window and the enlarged view window in the split view,
The method of displaying media on a computer device according to claim 7, wherein the second view is an overview only view. The layout portion of the web page is displayed at a scale factor that makes multiple elements of the layout appear larger than in the overview window;
The overall display size is approximately equal to the combined size of the overview window and the enlarged view window in the split view;
The media display method in a computer device according to claim 7, wherein the second view is an enlarged only view. The digital content has text displayed as part of the web page;
The correspondence to the second view selection input is that the layout width of the selected text in the layout used in the split view is related to the line width of the new column beyond the line of the new text column. Reflowing the selected portion of the text with one or more font sizes larger than the font size of the selected text associated with the second of the selected portion of the text of the web page Have a layout run and
The method of displaying media on a computing device according to claim 7, wherein the second view comprises a reflowed text window in which the maximum width of the new text column is displayed on the screen.   The media of claim 10, wherein the reflowed text window of the second view is approximately the same size as the combined size of the enlarged view window and the overview window for the first view. Display method. The second view displays the reflowed text window and makes the elements of the layout appear smaller than in the enlarged view, and text is displayed in the reflowed text window. And a second overview window that simultaneously displays a second overview window that displays a layout portion of the web page at a magnification that makes it look like a small font,
The method of displaying media on a computing device according to claim 10, wherein the second overview window includes a display of the portion of text currently displayed in the reflowed text window.   The method of claim 7, wherein the first view and the second view are displayed on a display screen of a personal digital assistant (PDA).   The method of claim 7, wherein the first view and the second view are displayed on a display screen of a mobile phone. Receiving digital content displaying the displayed web page, including text displayed in one or more columns as part of the web page;
Performing a first layout of the web page;
In relation to the line width of the new column, with one or more font sizes larger than the font size of the selected text in relation to the selected text width in the first layout; Performing a second layout of the selected portion of the text of the web page comprising reflowing the selected portion of the text beyond the line of a new text column;
Displaying a portion of the web page layout at a first magnification in an overview window; and a maximum width of the new text column to display a portion of the text displayed in the reflowed text window. Simultaneously displaying the reflowed and selected text in a reflowed text window displayed on the screen at a second magnification that displays in a larger font than in the window. A method for displaying media on a computing device, comprising: creating a split view of the web page by performing. Displaying a displayed text indicator in the overview window indicating the subpart of the text displayed in the overview window currently displayed in the reflowed text window;
Moving the portion of the text displayed in the reflowed text window and moving the displayed text indicator relative to the layout displayed in the overview window. In response to a navigation input from a user, whereby the subpart of the text displayed in the reflowed text window is the subpart of the text indicated by the displayed text indicator after such movement. The media display method according to claim 15, further comprising continuously responding to. Receiving digital content displaying a displayed web page;
Laying out the web page;
Displaying a portion of the web page layout in the overview window at a first magnification, and displaying elements of the layout larger in the enlarged view window than in the overview window. Using the display in the enlarged view window in a double magnification state to display the partial subparts of the layout displayed in the overview window in the enlarged view window; and Simultaneously displaying, in the overview window, an enlarged view indicator showing the subpart of the part of the layout shown in the overview window currently displayed in the overview window To split the web page. And generating,
Moving the subpart of the layout displayed in the enlarged view window and correspondingly moving the enlarged view indicator in relation to the layout displayed in the overview window In response to navigation input from a user, whereby the subpart of the layout displayed in the enlarged view window is indicated by the enlarged view indicator after such movement. Continuously responding to the subpart of
Scrolling the web page layout in relation to the overview window, whereby the previously hidden portion is scrolled to the overview window, and the portion of the enlarged view indicator Over the previously hidden portion of the web page layout, and correspondingly scrolling the subpart of the layout displayed in the enlarged view window, thereby The navigation input is generated by the subpart displayed in the enlarged view window corresponding to the subpart of the web page layout indicated by the enlarged view indicator after such movement. Before the above -Beyond the part of the web page layout displayed in the window, above the previously hidden part of the web page layout, the correspondence to navigation input is part of the enlarged view indicator. A method for displaying media on a computing device, including a response to navigation input to move the object.   The media display method in a computer device according to claim 17, wherein the divided view is displayed on the display screen of a personal digital assistant (PDA).   The method according to claim 17, wherein the divided view is displayed on the display screen of a mobile phone. Receiving digital content displaying a displayed web page;
Laying out the web page;
Displaying a portion of the web page laid out at the first magnification in the overview window, and making the layout elements appear larger in the enlarged view window than in the overview window. Displaying a partial subpart of the layout displayed in the overview window in the enlarged view window in a state of the second magnification to be displayed in the enlarged view window; Displaying an enlarged view indicator in the overview window indicating a subpart of a portion of the layout displayed in the overview window displayed in the enlarged view window;
Moving the subpart of the layout displayed in the enlarged view window, and moving the enlarged view indicator relative to the layout displayed in the overview window In response to navigation input from a user, whereby the subpart of the layout displayed in the enlarged view window is displayed in the layout of the layout indicated by the enlarged view indicator after such movement. Continuously responding to the subparts, and generating a split view of the web page by simultaneously performing,
Corresponding to the navigation input corresponds to a first form of navigation input by directly moving the enlarged view indicator and a portion of the web page layout displayed in the enlarged view window. And corresponding to a second form of navigation input by moving a cursor directly in response to the portion of the web page layout displayed in the enlarged view. Media display method. The correspondence to the second form of the navigation input is as follows:
Being previously hidden in the enlarged view window before the predetermined second form of the navigation input is generated beyond a subpart of the web page layout displayed in the enlarged view window. Moving the cursor over a portion of the web page layout;
Scrolls the web page layout in relation to the enlarged view window so that the previously hidden portion to which the cursor is moved is scrolled and displayed in the enlarged view window And
Correspondingly, the magnified view indicator is scrolled in the overview window so that the sub-part of the web page layout indicated by the magnified view indicator is in the magnified view window. 21. A media display on a computing device according to claim 20, comprising responding to a predetermined second format for navigation input by continuously responding to the displayed sub-part of the web page layout. Method.   21. A media display in a computing device according to claim 20, wherein the cursor is displayed at the same location relative to the web page layout in both the enlarged view window and the overview window. Method. The split view is displayed on a screen that is part of a browser computer capable of viewing digital content;
The browser computer includes a browser program that responds to user input requesting a predetermined web page by requesting the web page from a proxy server over a computer network;
The proxy server communicates the user request over a computer network to one or more servers that store or dynamically generate the requested web page;
The one or more servers send the requested web page to the proxy server;
The proxy server receives the downloaded web page as the reception of digital content;
The proxy server performs the layout of the web page;
The proxy server downloads the web page layout to the browser computer;
Thereafter, the viewing program running on the browser computer creates the display of the downloaded web page layout in the overview window and the enlarged view window, thereby dividing the segmentation. The method of displaying media on a computing device according to claim 20, wherein the view is created.   24. A media display method in a computer device according to claim 23, wherein the browser computer is a personal digital assistant (PDA).   24. A media display method in a computer device according to claim 23, wherein the browser computer is a mobile phone.   21. The method of displaying media on a computing device according to claim 20, wherein the first view and the second view are displayed on the display screen of a personal digital assistant (PDA).   The method of claim 20, wherein the first view and the second view are displayed on the display screen of a mobile phone. Receiving digital content displaying a displayed web page;
Laying out the web page;
Displaying a portion of the web page laid out at a first magnification in an overview window; and displaying elements of the layout larger in the enlarged view window than in the overview window. Using the display in the enlarged view window in a double magnification state, displaying the partial subpart of the layout displayed in the overview window in the enlarged view window; Simultaneously displaying an enlarged view indicator in the overview window indicating the subpart of the portion of the layout displayed in the overview window currently displayed in the enlarged view window. By doing the web page And generating the split view,
Correspondingly, a navigation input from a user is accommodated by moving a cursor in relation to the portion of the web page layout displayed in the enlarged view window,
The correspondence to a navigation input is hidden in the enlarged view window beyond the subpart of the web page displayed in the enlarged view window before the predetermined navigation input is generated. The cursor is moved over a part of the web page layout that is present, and the previously hidden part to which the cursor is moved is scrolled and displayed in the enlarged view window. Scrolling the web page layout relative to the magnified view window, while scrolling the magnified view indicator in the overview window, thereby The web page label indicated by the view indicator That out of the subpart corresponding continues to the subpart of the web page layout displayed on the enlarged view window includes a corresponding to a predetermined navigation input by, in the media display method on a computer device.   The cursor is displayed in the same position in relation to the web page layout in both the enlarged view window and the overview window so that the cursor is moved in response to the navigation input. 30. A method for displaying media in a computing device according to claim 28. The split view is displayed on a screen that is part of a browser computer capable of viewing digital content;
The browser computer includes a browser program that responds to a user input requesting a predetermined web page by requesting the web page from a proxy server over a computer network;
The proxy server communicates the user request over a computer network to one or more servers that store or dynamically generate the requested web page;
The one or more servers send the requested web page to the proxy server;
The proxy server receives the downloaded web page as the reception of digital content,
The proxy server executes the layout of the web page;
The proxy server downloads the web page layout to the browser computer;
Thereafter, the browser program running on the browser computer generates the split view by creating the display of the downloaded web page layout in the overview window and the enlarged view window. The method for displaying media in a computer device according to claim 28.   31. A media display method in a computer device according to claim 30, wherein the browser computer is a personal digital assistant (PDA).   31. A media display method in a computer device according to claim 30, wherein the browser computer is a mobile phone.   29. The media display method in a computer device according to claim 28, wherein the first view and the second view are displayed on the display screen of a personal digital assistant (PDA).   30. The method of displaying media in a computer device according to claim 28, wherein the first view and the first view display are displayed on the display screen of a mobile phone. Receiving digital content displaying a displayed web page;
Laying out the web page;
A second magnification state in which a portion of the web page layout is displayed in an overview window at a first magnification, and the elements of the layout are shown larger in the enlarged view window than in the overview window. Displaying the partial subpart of the layout displayed in the overview window in the enlarged view window using the display in the enlarged view window being Simultaneously displaying, in the overview window, an enlarged view indicator indicating the subpart of the part of the layout displayed in the overview window displayed in the enlarged view window. , Split view of the web page And generating,
In response, navigation input from the user is accommodated by moving a cursor in relation to the enlarged view window and the portion of the web page layout displayed in the overview window. A medium in a computing device comprising: the cursor pointing to the same part of the layout in both windows so that the cursor is moved in response to the navigation input Display method. Receiving digital content displaying the displayed web page, including text displayed as part of the web page;
Laying out the web page;
Displaying a portion of the web page layout in the overview window at a first magnification, and a second magnification that makes the elements of the layout appear larger in the enlarged view window than in the overview window. In the state, using the display in the enlarged view window, displaying the partial subpart of the layout displayed in the overview window in the enlarged view window; Simultaneously displaying in the overview window an enlarged view indicator indicating the subpart of the part of the layout displayed in the overview window displayed in the overview window. Create a split view of a web page And that,
Moving the subpart of the layout displayed in the enlarged view window and correspondingly moving the enlarged view indicator in relation to the layout displayed in the overview window In response to a navigation input from a user, whereby the subpart of the layout displayed in the enlarged view window of the layout indicated by the enlarged view indicator after such movement. Continuing to correspond to the subpart,
Given the font resolution of the font bitmap in the enlarged view window, with the character shape and pixel placement being selected to increase the percentage of placement of the end of the character shape, A media display method in a computer device, wherein each of a plurality of text characters in the web page is displayed in the enlarged view window using pixel boundaries of the font bitmap. The web page layout defines a layout of the font bitmap used in the enlarged view window;
The display of the web page layout displayed in the overview window uses the pixel boundaries of the low resolution font bitmap display in which the low resolution display is displayed in the overview window, and the arrangement of the ends of the character shape 37. The computer of claim 36, comprising a reduced low pixel resolution display for the font bitmap included in the layout that does not have the character shape and pixel arrangement selected to increase the percentage of Media display method on the device.   The font bitmap used to display characters in the enlarged view window is anti-aliased for the resolution at which the font bitmap is displayed in the enlarged view window. 38. A method of displaying media on a computing device according to claim 36, wherein the media display method is an anti-aliased font bitmap.   39. The computing device of claim 38, wherein the text displayed in the enlarged view window is composed of Latin alphabetic characters and is displayed in a font that is displayed with an advance width of most 4 lowercase letters or less. Media display method. The overview window and the enlarged view window are displayed on a screen having pixels configured with a predetermined arrangement of sub-pixels of different colors, individually addressable;
The anti-aliased font bitmap used to display characters in the enlarged view window has a coverage value that represents the percentage of a given subpixel covered by the character shape indicated by the font bitmap; Color unbalance resulting from the difference between the coverage value for a given subpixel and the coverage value for a set of one or more neighboring subpixels of different colors, at least for some subpixels of the font bitmap A color balance distribution relating to a ratio of the coverage value of the predetermined sub-pixel to the coverage value of the neighboring sub-pixel, including at least the sub-pixels of different colors, How many bitmaps The sub-pixels of different colors as a function of one or more of the color balance distributions from coverage values from one or more neighboring sub-pixels to coverage values of a given sub-pixel. A sub-pixel optimization bitmap that assigns a luminance value to each predetermined sub-pixel of a screen pixel having said predetermined arrangement of pixels;
40. The computing device of claim 38, wherein the color balance distribution is substantially limited to a distribution of a portion of the subpixel coverage value that causes color imbalance within all pixels that are part. Media display method. The digital content has both an image and a text string,
In order to assign horizontal and vertical virtual positions in the layout to each of the images and each position of a text string displayed on a predetermined line, the layout of the web page includes the images and Laying out the image and text at a virtual pixel resolution using a virtual layout pixel size for the text;
The second magnification is the second magnification relative to the portion of the web page layout displayed in the enlarged view window in relation to a corresponding position in the layout performed at the virtual pixel resolution. Have a non-single value to display with pixel resolution scaled down by magnification,
The image and text in the enlarged view window are displayed at pixel coordinates corresponding to the position of the image and text in a virtual web page layout as reduced by the second magnification,
The image and text are rendered in the enlarged view window at a reduced pixel size corresponding to the pixel size used for the image and text in a virtual layout as reduced by the second magnification. ,
The image of the text string in the enlarged view window has a pixel size reduced by the second magnification in relation to the size assigned to the text string in the virtual layout. Consisting of a series of font bitmaps,
The shape and pixel placement of the character indicated by the font bitmap used to display one of the characters in the enlarged view window is such that the font bitmap is in the enlarged view window. 37. A method for displaying media in a computing device according to claim 36, wherein at a resolution to be displayed, the pixel boundaries of the bitmap are used to increase the percentage of placement of the end of the character shape. The split view is displayed on a screen that is part of a browser computer capable of viewing digital content;
The browser computer has a browser program corresponding to a user input requesting a web page by requesting a predetermined web page from a proxy server on a computer network;
The proxy server communicates the user request over a computer network to one or more servers that store or dynamically generate the requested web page;
The one or more servers send the requested web page to the proxy server;
The proxy server receives the downloaded web page as the reception of digital content,
The proxy server performs the layout of the web page;
The proxy server downloads the web page layout to the browser;
37. The computer of claim 36, wherein the browser program running on the browser computer then performs the display of the web page layout in the overview window and the enlarged view window. Media display method on the device.   43. A media display method in a computer device according to claim 42, wherein the browser computer is a mobile phone.   43. A media display method in a computer device according to claim 42, wherein the browser computer is a handheld computer or a wearable computer.   38. A media display method in a computer device according to claim 36, wherein the divided view is displayed on the display screen of a personal digital assistant (PDA).   40. The method of claim 36, wherein the split view is displayed on the display screen of a mobile phone. Receiving digital content displaying a displayed web page;
Laying out the web page;
Displaying the portion of the web page layout in the overview window at a first magnification, and the enlarged view window at a second magnification that makes the layout elements appear larger than in the overview window. Displaying an enlarged view window on a subportion of the layout displayed in the overview window, displaying an enlarged subpart of a covered subportion of the layout displayed in the overview window; and Generating a view of the web page by doing simultaneously,
Moving the magnified view window relative to the overview window and correspondingly moving the magnified subpart of the layout displayed in the magnified view window, thereby The portion of the web page layout displayed in the enlarged view window continues to correspond to a subpart of the covered subportion of the layout displayed in the overview window after such movement. And a method for displaying a media in a computer device, which corresponds to navigation input from a user.   In association with the layout displayed in the overview window, display one or more enlarged subpart markers that indicate the position of the subpart of the layout displayed on the enlarged view window. The method for displaying media in a computer device according to claim 47, further comprising:   One or more of the enlarged subpart markers extend along one or more of the enlarged view window's edge width and / or height subparts and are displayed in the overview 49. Media in a computing device according to claim 48, each showing a horizontal and / or vertical extent and position of the subparts of the layout displayed in the enlarged view window in relation to the arranged layout Display method. The enlarged view window has substantially the same width as the overview window;
The correspondence to the navigation input by the user is displayed in the enlarged view window relative to the overview window and in relation to the layout, the vertical movement of the enlarged view window. Moving the position of the enlarged subpart of the layout vertically and the change in the vertical position of the enlarged subpart of the layout in relation to the layout displayed in the overview window. Moving one or more of the enlarged subpart markers vertically to accommodate a vertical navigation input, and in relation to the layout, to the enlarged view window. The enlarged subpart of the displayed layout One or more to move the position horizontally and to accommodate the change in the horizontal position of the enlarged subpart of the layout in relation to the layout displayed in the overview window. Moving the enlarged sub-part marker horizontally with respect to the overview window and responding to horizontal navigation input without substantially moving the enlarged view window; 49. A method for displaying media in a computing device according to claim 48, comprising: The digital content includes text;
The layout lays out the text in the web page layout;
At the resolution displayed in the enlarged view window, using the font bitmap's pixel boundaries, the character shape and pixel placement are selected to increase the proportion of edge placement of the character shape. 48. A method for displaying media in a computing device according to claim 47, wherein each of the plurality of characters of the text is displayed in the enlarged view window by a font bitmap.   Use the same font bitmap as the font bitmap displayed in the enlarged view window, but reduce the character shape and pixel array to a lower pixel resolution than the selected pixel resolution, 52. A method of displaying media on a computing device according to claim 51, wherein the overview window displays a reduced representation of the same web page layout displayed in the enlarged view window.   The font bitmap used to display characters in the enlarged view window is an anti-alias that is anti-aliased for the resolution at which the font bitmap is displayed in the enlarged view window. 52. A method of displaying media in a computer device according to claim 51, wherein the media display method is a normalized font bitmap.   54. The text displayed in the enlarged view window is displayed in a font composed of letters of the Latin alphabet, with the majority of lowercase letters displayed at an advance width of 4 pixels or less. A method for displaying media on a computer device. The screen on which the overview window and the enlarged view window are displayed has pixels configured in a predetermined arrangement with respect to sub-pixels of different colors that are individually addressable;
The anti-aliased font bitmap used to display characters in the magnified view has a coverage value that indicates the percentage of a given sub-pixel covered by the character shape indicated by the font bitmap; At least some of the sub-pixels of the font bitmap, including sub-pixels of different colors, coverage values for the predetermined sub-pixels and coverage for a predetermined set of one or more neighboring sub-pixels of different colors A color balance distribution for a ratio of coverage values of a given sub-pixel from the coverage value to a coverage value of a neighboring sub-pixel, created to avoid color imbalance resulting from a difference from the value, and at least the font bits Several sub-maps In the case of a ksel, sub-pixels of different colors as a function of the coverage value from one or more neighboring sub-pixels and one or more of the color balance distributions to the coverage value of the predetermined sub-pixel. A sub-pixel optimization bitmap that assigns a luminance value to each predetermined sub-pixel of a screen pixel having the predetermined arrangement of:
54. The computing device of claim 53, wherein the color balance distribution is substantially limited to a distribution of a portion of the subpixel coverage value that causes color imbalance within all pixels that are part. Media display method. The digital content includes one or more images and one or more text strings,
In order to assign horizontal and vertical virtual positions in the layout to each of the images and to respective portions of text strings displayed in a predetermined line, the layout of the web page is the image And laying out the image and text at a virtual pixel resolution using a layout pixel size for the text,
A portion of the web page layout displayed in the enlarged view window at a pixel resolution reduced by the second magnification is associated with a corresponding portion in the layout performed at the virtual pixel resolution. Displayed,
The image and text in the enlarged view window are displayed at pixel coordinates corresponding to the position of the image and text in a virtual web page layout, as reduced by the second magnification,
With the reduced pixel size corresponding to the pixel size used for the image and text in a virtual layout as reduced by the second magnification, the image and text are the enlarged view window. Drawn on
The image of the text string in the enlarged view window is reduced by the second scale factor in relation to the size assigned to the text in the text string in the virtual layout. The shape and pixel arrangement of the character shape having a size and represented by each such font bitmap to increase the proportion of the arrangement of the end of the character shape using the pixel boundaries of the bitmap 48. A method for displaying media on a computing device according to claim 47, comprising a series of selected font bitmaps. The overview window and the enlarged view window are displayed on a screen that is part of a browser computer capable of viewing digital content;
The browser computer includes a browser program that responds to a user input requesting a predetermined web page by requesting the predetermined web page from a proxy server over a computer network;
The proxy server communicates the user request over a computer network to one or more servers that store or dynamically generate the requested web page;
The one or more servers send the requested web page to the proxy server;
The proxy server receives the downloaded web page as the reception of digital content,
The proxy server performs the layout of the web page;
The proxy server downloads the website layout to the browser;
48. The computing device of claim 47, wherein the browser program running on the browser computer then performs the display of the web page layout in the overview window and the enlarged view window. Media display method.   58. A media display method in a computer device according to claim 57, wherein the browser computer is a mobile phone.   58. The method of displaying media on a computing device according to claim 57, wherein the browser computer is a handheld computer or a wearable computer. Multi-column column web pages, including multiple column specifications, each of which includes multiple displayable text strings, column character strings, column layout width, and column horizontal layout replacement specifications, respectively. Accessing the content of
If the horizontal layout position of each column is determined as a function of the specifications for the horizontal replacement of the column, the web page is laid out in a multi-layer column format that uses layout magnification, such as determining the layout position for each of the columns And
Allowing a user to select a predetermined subpart of a multi-layer column layout from among a plurality of different horizontally replaced subparts of the layout;
Displaying the layout portion selected by the user on a predetermined screen window having a predetermined horizontal resolution at a predetermined display magnification,
The multilayer column includes a set of columns arranged in a lateral direction having an overall multilayer layout width from the leftmost end of the leftmost column to the rightmost end of the rightmost column;
If the width corresponding to each of the specified layout widths of the one or more columns is longer than the maximum desired column width as scaled up / down at the layout magnification, the layout is Or limiting the width in which text in a plurality of the laterally arranged columns is laid out to a width shorter than or equal to the maximum desired column width,
The maximum desired column width is shorter than the total multi-layer column layout width, and when displayed in the predetermined window at the predetermined magnification, the web page display is shorter than or equal to the width of the predetermined window Method. The accessed web page content includes one or more images;
The specifications for individual columns include specifications where text and / or one or more images are present in the columns;
The layout of the web page includes laying out the text and / or the positions of a plurality of images existing in the column in each column,
If the width corresponding to the image is longer than the maximum desired column width as scaled by the layout magnification, text in one or more of the horizontally arranged columns is laid out The limitation of the width to be limited is to limit the width of any image laid out in such a column to a width that is shorter than or equal to the maximum desired column width by reducing the size of such an image. The web page display method according to claim 60, further comprising: One of the laterally arranged columns has a layout width corresponding to a specified width of the column shorter than the maximum desired column width;
61. The method of claim 60, wherein the text in the column is laid out with the layout width corresponding to the specified width of the column. The width in which the text of the first column of the laterally arranged columns is arranged is reduced by the width limitation;
The second column of the horizontally arranged columns laid out on the right side of the first column is at the layout width of the text of the first column so that it is enlarged / reduced by the layout magnification. 61. The web page display method according to claim 60, wherein the web page display method is laid out with a horizontal displacement in the right direction that is reduced relative to the horizontal replacement specification of the second column as a function of the reduction.   The display of the layout portion selected by the user on the predetermined screen window at the predetermined display magnification according to the layout magnification and / or the display magnification is the implementation of the restriction on the layout width of the text column. 61. The web page display method according to claim 60, wherein elements of the web page such as characters of the text are displayed independently at a reduced text resolution.   The individual characters of the text in the display may increase the percentage of the character array edge array having a pixel boundary for the resolution at which a font bitmap is displayed in the display. The web page display method according to claim 64, wherein each of the shapes and the pixel arrangement is displayed by a selected font bitmap.   66. The web page display method of claim 65, wherein the font bitmap displayed in the display is an anti-aliased font bitmap in which anti-aliasing is performed on the resolution at which the font bitmap is displayed on the display. .   68. The web page display method according to claim 66, wherein the text displayed in the display is composed of Latin alphabet characters, and the lowercase letters are displayed in a font displayed with an advance width of 4 pixels or less. A predetermined screen window is individually addressable and has pixels arranged in a predetermined arrangement with respect to sub-pixels of different colors;
The anti-aliased font bitmap used to display characters in the display includes a coverage value indicating a percentage of a predetermined sub-pixel covered by the character shape displayed in the font bitmap, and at least the In the case of several subpixels of a font bitmap, this is done to prevent color imbalance resulting from the difference between the coverage value of the given subpixel and a given set of neighboring pixels of one or more different colors. A color balance distribution for the ratio of the coverage value of a given subpixel from the coverage value to the coverage value of a neighboring subpixel, including subpixels of different colors, and at least for some subpixels of the font bitmap , From one or more neighboring subpixels Each predetermined sub-pixel for a screen pixel having the predetermined arrangement for sub-pixels of different colors as a function of a leverage value and one or more of the color balance distributions for the coverage value of the predetermined sub-pixel. Is a sub-pixel optimization bitmap that assigns brightness values to
68. The method of claim 66, wherein the color balance distribution is substantially limited to placement of a portion of the subpixel coverage values that cause color imbalance within all pixels that are part. Executing the display of the portion selected by the user of the layout in the predetermined window which is an enlarged view window at the predetermined display magnification;
The web larger than the user selected portion of the layout displayed in the enlarged view window at a different display magnification that makes the elements of the layout appear smaller in the enlarged view window Displaying part of the page layout in the overview window;
Displaying an enlarged view indicator in the overview window indicating the portion of the layout displayed in the overview window currently displayed in the enlarged view window;
Moving the user-selected portion of the web page layout displayed in the enlarged view window, and moving the enlarged view indicator in the overview window correspondingly. Corresponding to navigation input from a user, whereby the portion of the layout displayed in the enlarged view window is indicated by the enlarged view indicator after such movement. 61. The web page display method according to claim 60, wherein a split view of the web page is created by simultaneously responding to the portion of the web page. The screen on which the display is made is a part of a browser computer capable of browsing digital content,
The browser computer includes a browser program that responds to a user input requesting a predetermined web page by requesting the web page from a proxy server over a computer network;
The proxy server communicates the user request over a computer network to one or more servers that store or dynamically form the requested web page;
The one or more servers send the requested web page to the proxy server;
The proxy server receives the downloaded web page as the reception of digital content,
The proxy server performs the layout of web pages;
The proxy server downloads the web page layout to the browser;
61. The web page display method according to claim 60, wherein the browsing program executed on the browser computer then executes the display relating to the portion selected by the user of the layout.

Images