WO2000058821A9 - System, computer program product, computing device and associated methods for form identification and information manipulation - Google Patents
System, computer program product, computing device and associated methods for form identification and information manipulationInfo
- Publication number
- WO2000058821A9 WO2000058821A9 PCT/US2000/008769 US0008769W WO0058821A9 WO 2000058821 A9 WO2000058821 A9 WO 2000058821A9 US 0008769 W US0008769 W US 0008769W WO 0058821 A9 WO0058821 A9 WO 0058821A9
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- data
- computing device
- page
- enabled computing
- nature
- Prior art date
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
- G06F3/0487—Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser
- G06F3/0488—Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser using a touch-screen or digitiser, e.g. input of commands through traced gestures
Definitions
- the present invention relates generally to data processing with pen enabled computing devices and, more particularly, to a system, computer program product, computing device, and associated methods for electronically identifying preprinted forms used in conjunction with pen enabled computing devices and accessing and manipulating information associated therewith.
- Pen-enabled computing is a relatively recent development wherein a user interfaces with a computing system by way of a writing stylus instead of a mouse or a keyboard.
- a pen-enabled computing system provides an electronic writing tablet having an overlaid writing surface.
- Such a writing surface may comprise, for example, a blank sheet of paper or a preprinted form.
- the writing stylus may comprise a device capable of inputting data (a "handwriting input' " ) into the pen-enabled computing system while providing both a visible, or
- Such a writing stylus may comprise, for example, a conventional pen, a conventional pencil, a radio transmitter, a magnetic or electric field device, an ultrasound transceiver, or combinations thereof.
- the data may be stored as an "electronic ink” copy, wherein the handwriting input is captured as written, typically as a sequence of (x, y) coordinate pairs, a bit map or the like, or as a "text" copy, wherein the handwriting input is captured, recognized, and translated into the corresponding text, typically by automatic online handwriting recognition or the like.
- the pen-enabled computing system may be capable of producing both an electronic ink and a text copy of the handwriting input. Since the writing stylus is generally capable of providing a written ink copy of the handwriting input on the writing surface, the user is automatically provided with a hard copy, or visual feed back, of the entered data.
- a written ink copy of the handwriting input may not be provided in some instances, wherein the handwriting input is then just captured by the electronic writing tablet and stored in the pen-enabled computing system as an electronic copy.
- pen enabled computing devices include the CrossPadTM portable digital notepad by the A. T. Cross Company and the Palm Connected OrganizerTM personal data assistant (PDA) by Palm, Inc.
- PDA Palm Connected OrganizerTM personal data assistant
- One example of the capabilities of pen enabled computing devices is provided by U.S. Patent No. 5,629,499 to Flickinger et al., the contents of which are incorporated by reference herein.
- Flickinger describes an electronic board used to capture and store electronic ink data. According to Flickinger, a page having a writing surface is secured to the electronic board with clip.
- Pen input data is then entered by writing on the writing surface with a pen, wherein the writing is electronically captured by the board to create an electronic ink copy thereof.
- the electronic board is described by Flickinger as only being capable of recording pen input data for subsequent transfer of the pen input data to a processing device.
- the Flickinger electronic board is also described as being usable with a blank piece of paper or a preprinted form, wherein a user-initiated operation indicates to the board that a page has been positioned thereon. A subsequent operation is then required to identify the page to the board.
- the user must activate a switch outside the writing area to identify the page.
- having a separate switch for indicating each type of form supported by the device limits the capabilities and form type capacity thereof while disadvantageously adding components to the device.
- a toggle switch requiring multiple actuations in order to scroll through the list of form types supported by the device may be inconvenient and may also limit the capacity of the device.
- a pen enabled computing device having the capability to detect the presence of a page engaged with the device and to determine the type of form with a minimum of separate identification devices and required user-initiated operations.
- Flickinger describes a bar code imprinted on the back of the page, opposite to the writing surface, wherein the bar code indicates the form identity.
- a complementary bar code reader is then provided on the electronic board to read the bar code and identify the form.
- the process of adding a bar code to each page used with the device undesirably adds printing expense and complexity since the proper bar code must first be determined to identify the type of form and then a double-sided print or copy must be made such that the bar code is imprinted on the back of the page.
- the bar code reader disadvantageously adds components to the device.
- a pen enabled computing device having the capability of determining the type of form engaged therewith without requiring additional components and special modification of the form to indicate the form type to the device.
- another embodiment of the Flickinger device discloses a single small printed box provided on the form that must be checked by the user with the pen to indicate the form type. Accordingly, each form has a separate box disposed in a different position about the writing surface. Once a box is checked by the user to identify the particular form being used, the identification data corresponding to the marked box must be transmitted to another computing device to identify the corresponding form type.
- having a dedicated box for each form type, with each box being disposed in a dedicated position on the writing surface further limits the capacity of the device at the expense of the available area of the writing surface. This limitation is magnified if the form comprises multiple pages, wherein a separate box must be provided to identify each page of the form.
- this form type indicating system does not provide for the identification of multiple pages of a single form type.
- a method of identifying a form to a pen enabled computing device that is capable of indicating form type while also uniquely identifying forms having multiple pages as well as multiple pages of a single form type and while occupying minimal space on the writing surface of the page.
- Flickinger further discloses a device capable of receiving and storing pen input data which is then subsequently processed by a separate computing device.
- This device limitation may undesirably affect portability and the size of the device.
- the combination of a remote processing device and a simple form identification process does not provide the capability for identifying a specific page on which future revisions or additions to the entered data can be expediently and efficiently performed by way of the corresponding form on the electronic board.
- the Flickinger form identification process has a further limitation in that accessing of previously stored data is limited to a general form type and does not allow the user to access a more focused data set such as, for example, data input into a specific field on a specific page of a certain type of form.
- Flickinger' s inability to specifically identify the page being used with the electronic board may preclude a wide range of data manipulation processes that depend on the specificity and focus of a data set that can be sorted by a specific page identifier.
- a pen enabled computing device capable of specifically identifying the page being used therewith in order to facilitate future data modification, access, and manipulation in an efficient and expedient manner to provide "seamless" operation with minimal inconvenience and enhanced flexibility for the user.
- a pen enabled computing device having the innate capability to process or otherwise manipulate recorded pen input data.
- the pen enabled computing device may also be desirable for the pen enabled computing device to have the capability of detecting the presence of a page engaged with the device and to determine the type of form with a minimum of separate identification devices and required user-initiated operations. Still further, the pen enabled computing device should have the capability of determining the type of form engaged therewith without requiring additional components and special modification of the form to indicate the form type to the device. In addition, it would be desirable to have an associated method of identifying a form to a pen enabled computing device that is capable of indicating form type as well as uniquely identifying forms having multiple pages and multiple pages of a single form type, while occupying minimal space, if any, on the writing surface of the page.
- the pen enabled computing device may also be desirable for the pen enabled computing device to be capable of identifying the form without requiring dedicated identification boxes on the writing surface which must be specifically checked by the user.
- a pen enabled computing device capable of specifically identifying the page being used therewith in order to facilitate future data modification, access, and manipulation in an efficient and expedient manner to provide "seamless" and automatic operation with minimal inconvenience and enhanced flexibility for the user.
- the present invention provides a system for identifying a preprinted form and interacting therewith.
- the system comprises a pen enabled computing device having a writing stylus, a preprinted form, and a computer program product.
- the preprinted form is cooperable with the pen enabled computing device and comprises a page having a writing surface, a visual form identifier disposed on the writing surface and adapted to identify the nature of the page to the user, and a plurality of fields defined by the writing surface.
- the computer program product executes within the pen enabled computing device and cooperates therewith to determine the disposition of the writing stylus adjacent to the writing surface in order to actuate the pen enabled computing device.
- the pen enabled computing device is capable of detecting, capturing, and storing data input into the fields according to the nature of the page, wherein the nature of the page comprises a function and a specific identity.
- the nature of the page is also identifiable by the pen enabled computing device based on the data input into a plurality of the fields on the writing surface.
- the system further comprises an electronic handwriting tablet capable of interacting with a pen enabled computing device and the preprinted form so as to sense the position and movement of the writing stylus with respect to a page engaged with the handwriting tablet.
- the preprinted form may comprise a plurality of pages having a predetermined order such that a function and a specific identity may be determined for each page according to the order.
- the fields may have predetermined spatial arrangement on the writing surface according to the nature of the page such that the pen enabled computing device detects and captures data input in a plurality of the fields to determine the nature of the page.
- data may be input into each of a plurality of the fields, wherein the pen enabled computing device and the computer program product cooperate to process the data-containing fields to define a corresponding base n number that uniquely identifies the nature of the page to the pen enabled computing device.
- the pen enabled computing device may be configured to cooperate with the computer program product to define a base n number corresponding to a transposable matrix having n rows and m+1 columns, thereby forming an (n)*(m+l) matrix elements with each element corresponding to a field on the writing surface.
- Each matrix element thereby has a value co ⁇ esponding to the j*n', wherein i may vary from 0 to m and j varies from 0 to n-1.
- the base n number may be, for example, the summation of the values of all of the elements within the matrix co ⁇ esponding to field on the writing surface in which the user entered data.
- the pen enabled computing device may be further configured to enable access to other data stored therein that is associated with the specific identity of the page. Once the nature of the page has been identified by the pen enabled computing device, data stored therein that is associated with and/or independent of the specific identity of the page may be accessed. In some instances, the data may be provided to the user via a user interface. Still further, after the nature of the page has been determined, the pen enabled computing device cooperates with the computer program product to determine whether data input into the fields continues to co ⁇ espond to the determined nature of the page.
- the pen enabled computing device accordingly indicates an error if the input data fails to co ⁇ espond. If an e ⁇ or is indicated, the e ⁇ or is indicated to the user via the user interface. As such, the user may thereafter be prompted to co ⁇ ect the e ⁇ or through the pen enabled computing device.
- the user interface is further capable of indicating to the user the fields containing data, the data contained within the data-containing fields, and/or the number of fields containing data.
- the computer program product generally cooperates with the pen enabled computing device to detect, capture, and store handwritten data input into a field with the writing stylus. The handwritten data are then spatially segmented such that the spatial segments co ⁇ elate with the nature of the page.
- the computer program product is further capable of assigning a stamp to data input into a field, wherein the stamp co ⁇ esponds to the nature of the page and may comprise a time, a date, and/or a unique identifier.
- the nature of a page may comprise a function, a specific identity, and a stamp.
- the pen enabled computing device may further comprise an editing actuator, wherein the computer program product is capable of cooperating with the pen enabled computing device to edit the data upon actuation of the editing actuator.
- the computer program product may cooperate with the pen enabled computing device to allow the addition of data to data contained within a field, to allow insertion of data between data elements contained within a field, to allow deletion of data contained within field, and/or to allow replacement of data contained within field with alternate data.
- the computer program product may also be capable of determining whether data is being input into a data-containing field having a preassigned stamp. If this is the case, the user is prompted, through the user interface, to verify that the user intends to edit the data within the data-containing field.
- the computer program product may be capable of cooperating with the pen enabled computing device to allow the exportation of data or importation of data in relation to an external computing device or within either of the pen enabled computing device and the external computing device.
- the computer program product may be capable of reconciling the transmission of data corresponding to the nature of a page between the pen enabled computing device and the external computing device, within the pen enabled computing device, and/or within the external computing device.
- the computer program product is capable of allowing the pen enabled computing device and/or the external computing device to access data co ⁇ esponding to the nature of the page and to examine and otherwise manipulate data contained within the fields thereof.
- the pen enabled computing device may cooperate with the computer program product to selectively direct data contained within a field to be converted into text by the pen enabled computing device and/or the external computing device upon actuation of a conversion actuator.
- the system may be capable of selectively converting data contained within a field into text.
- Another aspect of the present invention comprises a computer program product adapted to execute within a pen enabled computing device having a writing stylus and to identify the nature of a preprinted form to the pen enabled computing device, wherein the form comprises a page having a writing surface defining a plurality of fields and the nature of the page comprises a function and a specific identity.
- the computer program product generally comprises a number of executable portions executing within the pen enabled computing device.
- the computer program product may comprise a first executable portion for determining the disposition of the writing stylus adjacent to the writing surface such that the writing stylus is capable of interacting with the fields in order to actuate the pen enabled computing device.
- a second executable portion directs the pen enabled computing device to detect, capture, and store data input into the fields with the writing stylus, wherein the data is input according to the nature of the page.
- a third executable portion identifies the nature of the page to the pen enabled computing device based upon data input into a plurality of the fields.
- the computer program product is adapted to facilitate interaction between the pen enabled computing device and an electronic handwriting tablet capable of sensing position and movement of the writing stylus with respect to a page engaged therewith.
- the computer program product may further comprise a fifth executable portion for determining a function and a specific identity for each page according to the order.
- the fields may have a predetermined spatial a ⁇ angement on the writing surface according to the nature of the page such that the second executable portion of the computer program product detects and captures data input in a plurality of the fields to enable the pen enabled computing device to determine the nature of the page.
- data may be input into each of a plurality of the fields, wherein the computer program product then processes the data- containing fields to define a co ⁇ esponding base n number that uniquely identifies the nature of the page to the pen enabled computing device.
- the pen enabled computing device may be configured to cooperate with the computer program product to define a base n number co ⁇ esponding to a transposable matrix having n rows and m+1 columns, thereby forming an (n)*(m+l) matrix elements with each element co ⁇ esponding to a field on the writing surface.
- Each matrix element thereby has a value co ⁇ esponding to the j*n', wherein i may vary from 0 to m and j varies from 0 to n-1. Therefore, the base n number may be, for example, the summation of the values of all of the elements within the matrix co ⁇ esponding to field on the writing surface in which the user entered data.
- the fourth executable portion may also access data stored within the pen enabled computing device, wherein the data may be with and/or be independent of the specific identity of the page.
- the computer program product further comprises a sixth executable portion adapted to cooperate with the user interface to provide data to the user.
- the computer program product may also comprise a seventh executable portion for determining whether data input into the fields continues to co ⁇ espond to the determined nature of the page and indicating an e ⁇ or if the input data fails to co ⁇ espond.
- the seventh executable portion may also prompt the user to co ⁇ ect the e ⁇ or through the pen enabled computing device.
- the seventh executable portion may be capable of indicating to the user through the user interface the fields containing data, the data contained within the data-containing fields, and/or the number of fields containing data.
- the second executable portion of the computer program product may detect, capture, and store handwritten data input into a field with the writing stylus by spatially segmenting the handwritten data and co ⁇ elating the spatial segments with the nature of the page.
- the computer program product may further comprise an eighth executable portion for assigning a stamp to data input into a field, wherein the stamp co ⁇ esponds to the nature of the page and may comprise a time, a date, and/or a unique identifier.
- the pen enabled computing device may comprise an editing actuator
- the computer program product further comprises a ninth executable portion for enabling editing of data upon actuation of the editing actuator.
- the ninth executable portion may also be capable of adding data to data contained within a field, inserting data between previous data elements contained within a field, deleting data contained within a field, and/or replacing data contained within a field with alternate data.
- the computer program product may further comprise a tenth executable portion capable of determining whether data is being input into a data- containing field having a preassigned stamp and, if so, prompting the user, through the user interface, to verify that the user intends to edit the data within the field.
- the computer program product may further comprise an eleventh executable portion capable of exporting data to and/or importing data in relation to an external computing device or within either the pen enabled computing device or the external computing device.
- the eleventh executable portion may be capable of reconciling transmission of data co ⁇ esponding to the nature of the page between the pen enabled computing device and the external computing device, within the pen enabled computing device, and/or within the external computing device.
- the computer program product may further comprise a twelfth executable portion capable of allowing the pen enabled computing device and/or the external computing device to access data co ⁇ esponding to the nature of a page and to examine and otherwise manipulate data contained within the fields thereof.
- the pen enabled computing device may further comprise a conversion actuator
- the computer program product also further comprises a thirteenth executable portion that is capable of selectively directing data contained within a field to be converted into text by the pen enabled computing device and/or the external computing device upon actuation of the conversion actuator.
- the computer program product may further comprise a fourteenth executable portion capable of selectively converting data contained with a field into text.
- Yet another aspect of the present invention comprises a pen enabled computing device adapted for use with a preprinted form, the preprinted form comprising a page having a writing surface, a visual form identifier disposed on the writing surface and adapted to identify the nature of the page to a user, and a plurality of fields defined by the writing surface and a ⁇ anged to co ⁇ espond to the nature of the page, further wherein the nature of the page generally comprises a function and a specific identity.
- Such a pen enabled computing device comprises a writing stylus and a sensing device, wherein the sensing device is adapted to engage the preprinted form and cooperates with the writing stylus such that the pen enabled computing device is actuated when the writing stylus is disposed adjacent to the writing surface in interactable relation with the fields, thereby enabling the pen enabled computing device to determine the nature of the page as data is input into the fields with the writing stylus according to the user-identified nature of the page.
- the sensing device may comprise an electronic handwriting tablet adapted to engage the preprinted form and capable of sensing position and movement of the writing stylus with respect to the page.
- the pen enabled computing device may be configured to be capable of determining the nature of a form comprising a plurality of pages having a predetermined order where each page has a function and specific identity co ⁇ esponding to the order.
- the pen enabled computing device may be configured to require data to be input into a plurality of the fields in order to determine the nature of the page therefrom, wherein the fields have a predetermined spatial a ⁇ angement on the writing surface according to the nature of the page.
- the pen enabled computing device may be configured to define a base n number that uniquely identifies the nature of the page from the data input into each of a plurality of the fields.
- the pen enabled computing device may be configured to cooperate with the computer program product to define a base n number co ⁇ esponding to a transposable matrix having n rows and m+1 columns, thereby forming an (n)*(m+l) matrix elements with each element co ⁇ esponding to a field on the writing surface.
- Each matrix element thereby has a value co ⁇ esponding to the j*n', wherein i may vary from 0 to m and j varies from 0 to n-1.
- the base n number may be, for example, the summation of the values of all of the elements within the matrix co ⁇ esponding to field on the writing surface in which the user entered data.
- the pen enabled computing device may be further configured to enable access to other data stored therein that is associated with the specific identity of the page.
- a still further aspect of the present invention comprises a method for identifying the nature of a page of a preprinted form with a pen enabled computing device having a writing stylus and interacting therewith.
- the nature of the page comprises a function and a specific identity
- the page further has a writing surface, a visual form identifier disposed on the writing surface and adapted to identify the nature of the page to a user, and a plurality of fields defined by the writing surface.
- Such a method comprises determining the disposition of the writing stylus adjacent to the writing surface such that the writing stylus is capable of interacting with the fields and actuating the pen enabled computing device, receiving data into the fields via the writing stylus and according to the nature of the page, determining the nature of the page from a plurality of the fields in which data is received, and identifying the nature of the page to the pen enabled computing device.
- the method may further comprise allowing access to other data stored within the pen enabled computing system, wherein the data may be associated with and/or independent of the specific identity of the page.
- the method may also further comprise determining a function and specific identity for each page co ⁇ esponding to the order.
- the fields may be defined by the writing surface according to a predetermined spatial a ⁇ angement, wherein the method then further comprises determining the nature of the page from the spatial arrangement of the plurality of the fields in which data is received.
- the method may comprise determining the nature of the page from a plurality of the fields in which data is received, wherein the data input into each of the plurality of the fields is processed to define a base n number uniquely co ⁇ esponding to the nature of the page. Accordingly, the input data may be determined by sensing the position and movement of a writing stylus with respect to a page with an electronic handwriting tablet engaged with the page.
- the method may also comprise providing data to the user via a user interface, wherein the user interface is cooperable with the pen enabled computing device.
- the method may also comprise determining whether data input into the fields continues to co ⁇ espond to the determined nature of the page and indicating an e ⁇ or if the input data fails to co ⁇ espond.
- the method may further comprise indicating an e ⁇ or to the user through the user interface and prompting the user to co ⁇ ect the e ⁇ or using the pen enabled computing device.
- the user interface may also be configured such that the method may further comprise indicating to the user the fields containing data, the data contained within the data-containing fields, and/or the number of fields containing data.
- the method may comprise detecting, capturing, and storing handwritten data input into a field with the writing stylus by spatially segmenting the handwritten data and co ⁇ elating the spatial segments with the nature of the page.
- the method may also comprise assigning a stamp to data input into a field, wherein the stamp co ⁇ esponds to the nature of the page and comprises a time, a date, and/or a unique identifier.
- the method may further comprise actuating an editing actuator in order to enable the pen enabled computing device to edit the data.
- the pen enabled computing device may also be enabled to add data to data contained within a field, insert data between data elements contained within a field, delete data contained within a field, and replace data contained within a field with alternate data.
- the method may further comprise determining whether data is being input into a data-containing field having a preassigned stamp and, if so, prompting the user through the user interface to verify that the user intends to edit the data within the field.
- the method may comprise enabling the pen enabled computing device to export data and/or import data in relation to an external computing device or within either the pen enabled computing device or the external computing device.
- the transmission of data may also comprise reconciling the transmission of data co ⁇ esponding to the nature of a page between the pen enabled computing device and the external computing device, within the pen enabled computing device, and/or within the external computing device. Accordingly, the pen enabled computing device and/or the external computing device may have access to data co ⁇ esponding to the nature of a page with the capability to examine and otherwise manipulate data contained within the fields thereof.
- the method may also comprise selectively directing data contained within a field to be converted into text, wherein the method may generally comprise selectively converting data within a field into text.
- Yet another advantageous aspect of the present invention comprises a method for converting handwriting into a machine-usable format using a pen enabled computing device having a writing surface and a writing stylus selectively communicable with the writing surface.
- a pen enabled computing device having a writing surface and a writing stylus selectively communicable with the writing surface.
- the position and movement of the writing stylus with respect to the writing surface is detected by the pen enabled computing device to form a handwritten input.
- Bounding boxes are then defined around each stroke of the handwritten input.
- the handwritten input and the bounding boxes are then stored in a machine-usable format, thereby allowing the handwritten input to be edited by adding textual data to the handwritten input, inserting textual data between adjacent bounding boxes, deleting at least one bounding box, and replacing at least one bounding box with alternate data.
- detecting the position and movement of the writing stylus may further comprise sampling the motion of the writing stylus with respect to the writing surface so as form a coordinate representation of each stroke of the handwritten input.
- editing the handwritten input may comprise receiving an indication from a user that the handwritten input is to be edited and receiving textual input from a keyboard to form the textual data with which the handwritten input is edited. Accordingly, editing the handwritten input may further comprise altering the coordinate representation of at least some strokes of the handwritten input in order to alter spacing of the at least one bounding box so as to insert the textual input between adjacent bounding boxes and/or replace a bounding box containing handwritten input with the textual input.
- Yet still another advantageous aspect of the present invention comprises a system for converting handwriting into a machine-usable format, wherein the system comprises a pen enabled computing device having a memory element, a writing surface, and a writing stylus selectively communicable with the writing surface.
- the system may also comprise a computer program product executing within the pen enabled computing device and cooperating therewith for detecting the position and movement of the writing stylus with respect to the writing surface to form a handwritten input, defining a bounding box about each stroke of the handwritten input, storing the handwritten input and the bounding box su ⁇ ounding each stroke in the memory element in a machine-usable format, and editing the at least one handwritten element to add textual data to the handwritten input, insert textual data between adjacent bounding boxes, delete at least one bounding box, and/or replace at least one bounding box with alternate data.
- a computer program product executing within the pen enabled computing device and cooperating therewith for detecting the position and movement of the writing stylus with respect to the writing surface to form a handwritten input, defining a bounding box about each stroke of the handwritten input, storing the handwritten input and the bounding box su ⁇ ounding each stroke in the memory element in a machine-usable format, and editing the at least one handwritten element to add textual
- the computer program product may sample the motion of the writing stylus with respect to the writing surface so as to form a coordinate representation of each stroke of the handwritten input.
- the pen enabled computing device may cooperate with the computer program product to receive an indication from a user of the handwritten input to be edited and also receive textual input from a keyboard to form the data with which the handwritten input is edited.
- the computer program product edits the handwritten input by altering the coordinate representation of at least some strokes of the handwritten input in order to alter spacing of the at least one bounding box so as to insert the textual input between adjacent bounding boxes and/or replace a bounding box containing a handwritten element with the textual input.
- embodiments of the present invention provide a pen enabled computing device having the innate capability of processing or otherwise manipulating recorded pen input data.
- Embodiments of the present invention also provide a pen enabled computing device having the capability of detecting the presence of a page engaged with the device and determining the type of form with a minimum of separate identification devices and required user-initiated operations.
- the user uses the form according to its identified function and does not otherwise have to independently indicate the nature of the form to the pen enabled computing device.
- embodiments of the present invention provide a pen enabled computing device having the capability of determining the type of form engaged therewith without requiring additional components or special modification of the form to indicate the type of form to the device.
- Embodiments of the present invention also provide an associated method of identifying a form to a pen enabled computing device, while indicating the type of form as well as uniquely identifying forms having multiple pages and multiple pages of a single type of form.
- a method accomplishes the identification of the page to the pen enabled computing system while occupying minimal, if any, space on the writing surface of the page.
- embodiments of the present invention provide a pen enabled computing device capable of specifically identifying the page being used therewith in order to facilitate future data modification, access, and manipulation in an efficient and expedient manner to provide enhanced seamless and automatic operation with minimal inconvenience and enhanced flexibility for the user.
- FIGURE 1 is a schematic of a pen enabled computing system according to one embodiment of the present invention.
- FIGURE 2 is a schematic of a pen enabled computing device according to one embodiment of the present invention.
- FIGURE 3 is an example of a contacts form in accordance with one embodiment of the present invention.
- FIGURE 4 is an example of a daily schedule form in accordance with one embodiment of the present invention.
- FIGURE 5 is an example of a to-do list form in accordance with one embodiment of the present invention.
- FIGURE 6 is an example of a notes form in accordance with one embodiment of the present invention.
- FIGURES 7(a) and 7(b) is a flowchart of a method of identifying a form and manipulating data associated therewith with a pen enabled computing device according to one embodiment of the present invention.
- FIG. 1 is a schematic of a system for identifying a form and manipulating associated information with a pen-enabled computing device according to one embodiment of the present invention, the device being indicated generally by the numeral 100.
- the device 100 generally comprises a handwriting capture interface 200, a central processing subsystem 300, a user interface 400, a computer program product 500, and an external device interface 600.
- the handwriting capture interface 200 generally comprises an apparatus for sensing the movements of a writing stylus.
- the handwriting capture interface typically provides a digitized output that defines a number of (x,y) coordinate pairs, each of which defines a mark made by the writing stylus upon a writing surface.
- the writing stylus is typically a writing instrument which may be capable of creating visible marks upon a writing surface such as, for example, a blank of piece of paper 210 or a paper-based preprinted form 220.
- the handwriting capture interface 200 comprises a writing stylus and an electronic handwriting tablet area. The stylus and the handwriting tablet are interfaced such that the position or motion of the writing stylus, co ⁇ esponding to writing strokes, with respect to the handwriting tablet can be detected and captured by the device 100.
- the stylus may interact with the handwriting tablet such that the position or motion of the stylus is detected when proximate to the tablet or, alternatively, in contact with the handwriting tablet through a writing surface, wherein the writing surface may comprise, for example, the paper 210 or form 220, a multiplicity thereof, or a surface integral with the handwriting tablet.
- a handwriting tablet may not be necessary if the position and motion of the writing stylus may be directly detected by the handwriting capture interface 200 or other components of the device 100.
- the central processing subsystem 300 may comprise, for example, a central processor unit 320, non- volatile memory storage 340, and RAM storage 360.
- the central processing unit (CPU) 320 is generally capable of executing all computer processing activities required to support the device 100.
- Non-volatile memory resources 340 such as, for example, an EEPROM or a disk, are typically used for storing and/or executing a computer program product 500 as well as storing useful data such as, for example, electronic ink.
- the RAM storage 360 is used for loading and executing the computer program product 500.
- the user interface 400 includes provisions for facilitating interaction between the device 100 and a user.
- the user interface 400 may comprise, for example, one or more buttons for initiating functions within the device 100 where, in some instances, the buttons may comprise a part of a keyboard or a keypad.
- the user interface 400 may also comprise an apparatus for allowing the user to view displayed data or status information such as, for example, a viewing screen or monitor.
- the user interface 400 may also include an audio mechanism such as, for example, a buzzer, bell, or other indicator or other devices capable of supporting voice interaction between the device 100 and the user.
- a vibration mechanism may also be used as an alert or acknowledgement indicator. Accordingly, the user interface 400 may be implemented in many different manners, combinations thereof, and/or in conjunction with different components of the device 100.
- the computer program product 500 generally cooperates with the central processing subsystem 300 to control the device 100.
- the computer program product 500 further drives the handwriting capture interface 200 to interface and cooperate with a writing surface.
- the computer program product 500 enables the device 100 to, for example, sense a plurality of fields associated with the paper 210 or the preprinted form 220, wherein the function and specific identity thereof may then be electronically identified to the device 100.
- the function and specific identity of the form may also be refe ⁇ ed to herein as comprising the nature of the form.
- handwriting capture interface 200 may be implemented with, for example, a piece of paper 210, a form 220, or a surface integral with the interface 200
- reference to the handwriting capture interface 200 will be made herein to an embodiment of the invention having a form 220 interacting therewith. It will be understood, however, that this reference is for illustration only and such reference may include embodiments, otherwise mentioned herein as examples, as well as other configurations of a handwriting capture device consistent with the spirit and scope of the present invention.
- the system is then able to act upon data entered in the fields associated with the form, the data comprising, for example, a handwriting input.
- the handwriting input is captured and stored by the computer program product 500 as a coordinate representation of the interaction of the writing stylus with the handwriting tablet.
- the handwriting capture interface provides a plurality of (x,y) coordinate pairs representative to respective marks made by the writing stylus upon the handwriting tablet.
- the central processing subsystem defines a bounding box around the (x,y) coordinate pairs that cumulatively represent a particular stroke.
- a stroke is generally defined as a continuous marking by the writing stylus beginning with the commencement of contact or other interaction between the writing stylus and the handwriting tablet and terminating with the removal of the writing stylus from the handwriting tablet.
- a "c” is generally formed of one stroke
- a "t” is generally formed of two strokes.
- the computer program product 500 may further have the ability to stamp any data entries as they are entered into the system 100 with, for example, a time, a date, and/or another unique identifier. The device 100 is then able to process these data fields with respect to the stamp. As these functions are executed, the computer program product 500 also controls the user interface 400 such that information relevant to ongoing stylus or device 100 activities are displayed for the user. In one advantageous embodiment, the computer program product 500 may be capable of formatting, sending, receiving, reconciling, and interpreting messages flowing through the external device interface 600. The computer program product 500 is described in more detail herein.
- the external device interface 600 is in communication with the central processing subsystem 300 and enables the device 100 to communicate with external computing devices in a unidirectional or bidirectional manner. Generally, the communication is accomplished via a communications link 700 to an external computing device 800. Accordingly, the external device interface 600 may be generic in terms of protocol and communication medium as is known to one skilled in the art.
- the communications link between the device 100 and the external computing device 800 may be either direct or indirect connections via wireline or wireless devices.
- the communications link 700 may comprise a variety of external communications links as well as a combination of these links such as, for example, wired serial interfaces, wired parallel interfaces, wireline modems, wireless modems, LAN connections, and WAN connections.
- a handheld computer component 110 such as, for example, a modified personal digital assistant (PDA) or a smart-phone provides most of the functionality of the device 100. Accordingly, the handheld computer component 110 comprises the central processing subsystem 300, the user interface 400, the computer software program product 500, and the external device interface 600 as shown in FIG. 1.
- the user interface 400 may include an interactive display 120, one or more control buttons 130, and a speaker 140.
- the external communications interface 600 may include a wireless transceiver (not shown) and an antenna 150.
- the handwriting capture interface 200 comprises an electronic handwriting tablet 230 that may be co-operable with, for example, a blank piece of paper 210 or a paper-based preprinted form 220.
- the electronic handwriting tablet 280 may be serially interfaced with the handheld computer component 110.
- a dual-mode writing stylus 290 is employed by the user to form visible marks on the paper-based preprinted form 220, wherein the motion and position of the writing stylus 290 are also detectable by the electronic handwriting tablet 280.
- a pen-enabled computing device 100 may comprise many different combinations of components than the embodiment previously described.
- an electronic handwriting tablet may be combined with a desktop personal computer, wherein the electronic handwriting tablet and the personal computer are physically connected via a wired interface and the personal computer is connected to a local area network (LAN).
- LAN local area network
- an electronic handwriting tablet may be combined with a laptop personal computer and a central server, wherein the electronic handwriting tablet and the personal computer are connected via a wireless interface and the personal computer has a network connection to the central server via a LAN.
- the central server is further capable of communicating with various application servers.
- a pen- enabled computing device may be accomplished in many different manners consistent with the spirit and scope of the present invention.
- one purpose of the pen enabled computing device is to identify the nature of a form 220 engaged therewith.
- forms 220 may include a contact list form as shown in FIG. 3, a calendar form as shown in FIG. 4, a to-do list form as shown in FIG. 5, and a general notes form as shown in FIG. 6.
- Other examples of forms include, for instance, a trip planner form, a memorandum form, an educational form, a training form, an insurance form, a healthcare form, an expense accounting form, and a wide variety of other forms consistent with the spirit and scope of the present invention.
- FIG. 3-6 wherein such forms 220 may include a contact list form as shown in FIG. 3, a calendar form as shown in FIG. 4, a to-do list form as shown in FIG. 5, and a general notes form as shown in FIG. 6.
- Other examples of forms include, for instance, a trip planner form, a memorandum form, an educational form, a training form, an insurance form, a healthcare form, an expense accounting form, and
- a particular contacts form 220 comprises a writing surface 222, at least one visual form identifier 224, and a plurality of fields 226 for receiving data via the writing stylus.
- the fields 226 are generally defined by a series of spatial coordinates so as to generally define a box configured to receive a data input therein co ⁇ esponding to a particular function.
- a field labeled "Address” is capable of identifying itself to a user and is expected to receive an input therein with the writing stylus co ⁇ esponding to the address of a particular contact.
- fields labeled, for example, "G" and "7" could indicate that the user wishes to store an entered record on the seventh page of all contacts having data within the "Name" field having the letter "G".
- the fields 226 defined by the writing surface 222, along with the visual form identifiers 224, may combine to indicate a function of the form as well as a specific identity should the form comprise multiple pages or should there be multiple pages of one type of form.
- the user interacts with the form 220 according to the user-identified function imparted by the visual form identifiers 224.
- the user then enters data within the separate fields according to the functions thereof.
- data entered into a field it is spatially referenced with respect to the box defining the field.
- data entered into a "Name” field may be referenced to the leftmost vertical boundary or possibly a combination of the leftmost vertical boundary and the lowermost horizontal boundary.
- data entered into an "Address” field may be referenced to the leftmost vertical boundary of the address box, which also co ⁇ esponds to the rightmost vertical boundary of the name field.
- data entered into defined fields on a form according to the functions thereof may be spatially referenced so as to be indicative of the nature of the form 220.
- FIG. 4 shows an example of a daily scheduler form 228 which also includes a writing surface 230, at least one visual form identifier 232, and a plurality of fields 234 defined by the writing surface 230.
- the fields 234 of the daily scheduler form 228 are spatially defined in a different manner than the previous contacts form 220 as shown in FIG. 3. Thus, the fields for indicating the month and day in the daily scheduler form 228 may fall outside any of the fields defined by the contacts form 220. Conversely, certain fields in the contacts form 220, such as the "Address" or "E- mail" fields may be spatially disposed away from the spatial reference points for the fields of the daily scheduler form 228.
- spatial representation of fields disposed on a form can also serve to differentiate between different types of forms.
- various fields may also be spatially disposed so as to define a unique code over other forms. For example, checking the appropriate fields on the daily scheduler form 228 for a month and day differentiates the particular form from daily scheduler forms for other days throughout the year.
- the pen enabled computing device 100 is particularly adapted to interact with a form 220, wherein entering data in fields on the form 220 with a writing stylus 240 provides both a written ink copy of the particular form in addition to an electronic ink copy detected, captured, and stored by the device 100.
- the device 100 is configured to be capable of identifying the nature of the form 220, wherein the nature of the form comprises a function and a specific identity.
- the nature of the form 220 may be defined by a code co ⁇ esponding to data entered into particular fields on the form 220.
- the particular fields may be disposed and configured so as to form a unique code depending on the particular fields in which data is entered.
- the pen enabled computing device 100 may be able to define a base n number that indicates the nature of a particular form.
- the pen enabled computing device may be configured to cooperate with the computer program product to define a base n number co ⁇ esponding to a transposable matrix having n rows and m+1 columns, thereby forming an (n)*(m+l) matrix elements with each element co ⁇ esponding to a field on the writing surface.
- Each matrix element thereby has a value corresponding to the j*n', wherein i may vary from 0 to m and j varies from 0 to n-1. Therefore, the base n number may be, for example, the summation of the values of all of the elements within the matrix co ⁇ esponding to field on the writing surface in which the user entered data.
- selecting a particular combination of matrix elements within the matrix provides a unique identifier which may be used to indicate the nature of the form and such defined coding may be appropriately scaled or otherwise altered to match the form type support capacity of the device 100. In this manner, the area occupied on the writing surface by the particular fields may be reduced or minimized since the number of fields required to produce a unique form identification code requires less area on the writing surface compared to specifically dedicated identification fields.
- the pen enabled computing device 100 may be configured to selectively apply a stamp to data entered within a particular field, wherein the stamp may comprise a time, a date, or some other type of unique identifier that may be further referenced by the pen enabled computing device 100.
- the pen enabled computing device 100 may be configured so as to be capable of identifying the nature of the form based upon the spatial disposition of the fields defined by the writing surface.
- the pen enabled computing device 10 is capable of determining the spatial disposition of data entered on the writing surface in comparison to the spatial references for particular fields for the various forms supported by the pen enabled computing device 100.
- fields on different forms can be staggered or otherwise uniquely positioned with respect to other forms supported by the system so that each form has a unique combination of fields that can be automatically identified by the pen enabled computing system as the user enters data in one or more of the fields.
- the identification of a unique combination of fields maybe sufficient to indicate the nature of the form to the device 100.
- the enhanced determination of the nature of the form provides further utility in that future editing of the input data is facilitated as well as future editing of the particular paper-based form.
- error detection and co ⁇ ection are also facilitated.
- the device 100 is capable of, for example, detecting whether the user has entered data outside of an expected defined field and may apply a spatial criteria to determine whether the data has been insignificantly entered outside the boundary of the field by the user.
- the device 100 may alert the user with an alarm or some other user interface device and present the user with the opportunity to resolve the abe ⁇ ant data. For example, the user could be asked to identify to the device 100 the user-identified nature of the page. Alternatively, the user could be presented with the opportunity to move the abe ⁇ ant data to an expected field or given the opportunity to save the abe ⁇ ant data as an incidental note not associated with a particular expected field. Accordingly, it is understood that such applied criteria may facilitate a variety of possibilities for resolving the misentered data as well as other data processing operations.
- detailed identification of the nature of the page facilitates the indication of data to the user as the user is interacting with the device 100.
- the device could display to the user any previous data entered into the co ⁇ esponding fields, the nature of that data, or possibly the number of fields in which the data has been entered.
- the device 100 may also be able to determine whether the user is entering data in a field in which data had been previously entered. In detecting the situation, the device 100 may further be capable of alerting the user as to the presence of the previous data as well as given the user the opportunity to add to the data in the field, replace the data in the field, or abort the input of data into that particular field.
- FIG. 7 A and 7B provide a flowchart of a method of identifying a form and manipulating information using a pen enabled computing device according to one embodiment of the present invention.
- the device 100 determines the position and senses the movement of the writing stylus with respect to the writing surface (block 1000) and a text, receives, and stores the data entered into the fields of the page (block 1010). If the user is entering data into a field defined by a previous stamp, the stamp comprising a time, date, or other unique identifier (block 1020), the device 100 prompts the user to indicate whether the user wishes to proceed with editing the existing field (block 1030).
- the device 100 indicates an e ⁇ or message to the user (block 1040) and allows the user to select an alternate field on the writing surface or, for example, resolve a conflict in form types.
- the device 100 allows the user to edit the data and then saves the edited data and assigns a new stamp to the field (block 1060).
- the field had contained no previous stamp, the user would be permitted to input new data into the field which would be assigned a stamp following the completion of data entry by the user (block 1070).
- the data is stored by the device 100 (block 1080).
- the device 100 may then proceed to determine the spatial a ⁇ angement of the fields in which data was entered by the user (block 1090) and/or proceed to define a base n number based on the fields having a data input (block 1100).
- the nature of the particular page can then be determined (block 1110) and the device 100 is then able to verify that the data entered into the selected fields co ⁇ esponds to the identified nature of the page (block 1120).
- the device indicates an e ⁇ or message to the user (block 1130) and further prompts the user to select a different field or resolve the conflict between form types.
- the device 100 allows access to other data based on the nature of the page (block 1140) or according to a more focused search criteria specified by the user.
- data including the data input by the user, the data contained within fields within the form, and other data associated with the nature of the particular page may also be provided to the user through the user interface (block 1150) such that the user is able to monitor the data status and to edit or otherwise manipulate the data associated with the particular page.
- the device 100 that a variety of data manipulation functions are possible using the device 100 and/or an external computing device 800 according to the spirit and the scope of the present invention.
- the present invention also provides a technique for seamlessly editing handwritten and/or textual data by means of a pen-enabled computing device 100 and a computer program product 500 executing thereon.
- the handwritten input is initially captured by the handwriting capture interface 200 which detects the position and movement of the writing stylus 290 with respect to the writing surface 280 of the pen-enabled computing device 100.
- the handwriting capture interface 500 generally provides a sequence of (x,y) coordinate pairs that define respective marks by the writing stylus 290 upon the writing surface 280.
- the central processing unit 320 of the pen-enabled computing device 100 in conjunction with the computer program product 500 then define bounding boxes around each stroke.
- each bounding box is a rectangular box that includes at least a portion of the (x,y) coordinate pairs of the respective stroke with one x coordinate of the rectangular box defined by the smallest x of any of the coordinate pairs and the x coordinate of the rectangular box at a point defined by the largest x, with the y coordinates extending between successive baselines.
- the bounding boxes may also have included in the definitions thereof a time variable which may affect the delegation of a stroke to a particular bounding box.
- a bounding box is described herein in spatial terms as at least partially encompassing a single stroke, the boundary box can be defined differently in other applications, typically based on spatial or temporal characteristics of one or more strokes in order to bundle one or more strokes together as a unit for editing purposes.
- the central processing unit 320 in cooperation with the computer program product 500 operating thereon also determines the baseline for each of the lines so as to provide a point of reference for the characters in the respective line.
- the baseline typically co ⁇ esponds to the lowermost portion of most alphanumeric characters, although several letters, such as p, g and j have lower portions that fall below the baseline.
- the baseline can be determined in various manners. For example, the user of the pen-enabled computing device 100 can place the baseline by dragging a baseline into the proper position, by positioning a baseline with repeated actuation of up and down a ⁇ ow keys or any other like means.
- baselines can also be determined based upon changes in a one-dimensional histogram in which the histogram axis co ⁇ esponds to the y or vertical direction of the page that the user is writing upon.
- the histogram includes counts for a number of y values that define the number of strokes that pass through the respective y value. Since the y values that have large counts generally co ⁇ espond to lines of handwriting, a baseline is typically defined immediately below y values having large counts.
- the central processing unit 320 can define the baselines in a spatial a ⁇ angement co ⁇ esponding to the spatial arrangement of lines on the page 220.
- the handwritten input can be edited by initially displaying the handwritten input to the user upon a display 400 associated with the pen-enabled computing device 100.
- the user selects an actuation position relative to the handwritten input, such as by positioning the cursor through the use of a mouse, stylus, a track ball or the like, relative to the display of the handwritten input.
- the pen-enabled computing device 100 may shift the cursor position somewhat.
- the central processing unit 320 in cooperation with the computer program product 500, determines if the cursor position is located within a bounding box.
- the pen-enabled computing device 100 determines the x coordinate of the center of the largest bounding box. If the cursor position is at an x position greater than the x position of the center of the largest bounding box, the central processing unit 320 shifts the cursor position to co ⁇ espond to or be slightly larger than the largest x value associated with the bounding box, i.e., shifts the cursor to the right side of the largest bounding box.
- the central processing unit 320 shifts the cursor position to an x position that is equal to the smallest x value associated with the bounding box, i.e., shifts the cursor position to the left of the smallest bounding box. If, however, the cursor position is not within a bounding box, the cursor position need not be shifted whatsoever.
- the pen-enabled computing device 100 determines the editing function based upon the user input. If text is being inserted, for example, the text may be input by means of keystrokes on the keypad or by a handwritten insert accomplished by further movement of the writing stylus 290 over the writing tablet 280 as described above. If text is being inserted, such as by means of the keypad, the strokes that define the handwritten input to the right of the insertion are moved further to the right by an amount equal to the size, in the widthwise direction, of the inserted text.
- the central processing unit 320 determines the number of characters being inserted and the font of the characters and moves the bounding boxes containing the handwritten input to the right of the insert further to the right by an amount equal to the multiple of the number of characters being inserted and the font size of the inserted characters (hereinafter referenced as the width of the insert).
- the x coordinate value of each coordinate point within a bounding box on the same line as and to the right of the insert is increased by an amount equal to the width of the insert.
- a handwritten insertion is inserted by initially forming bounding boxes about the strokes that form the handwritten input. Thereafter, the bounding boxes around existing handwritten input on the same line as and to the right of the insert are moved further to the right by an amount equal to the width of the insert, which is defined in this instance as the difference in the smallest x value associated with the leftmost bounding box of the handwritten insertion and the largest x value associated with the rightmost bounding box of the handwritten insertion.
- the bounding boxes of the existing handwritten input that are being moved are altered by increasing the x coordinate of each coordinate pair falling within the bounding boxes by an amount equal to the width of the insert.
- the central processing unit 320 In addition to inserting text and/or a handwritten insertion, at least a portion of the existing handwritten input can be deleted and the remaining handwritten input moved so as to close up any gaps in the resulting handwritten input.
- the central processing unit 320 effectively moves the bounding boxes surrounding the remaining handwritten input to the left by an amount equal to the width of the deleted bounding boxes, typically by reducing the x values of the coordinate pairs that fall within each of the bounding boxes of the remaining handwriting input.
- the central processing unit 320 can also wrap the resulting handwritten input as appropriate depending upon the predefined margins and the like.
- While the above- described editing capabilities of the present invention relate to the editing of a handwritten input, a textual message that has been previously entered can also be edited, such as by inserting handwritten characters in the same fashion, if so desired.
- the pen-enabled computing device 100 and associated methods of the present invention provide increased functionality by permitting a seamless editing of the textual or handwritten input with either additional text or handwriting and appropriately spacing the resulting data without requiring extensive input from the user.
- the editing function provided by the pen-enabled computing device and associated method of the present invention will be quite intuitive to a user and will produce a resulting message that is appropriately spaced notwithstanding the insertion and deletion of handwriting and/or textual data.
- the embodiments described herein address instances in which a single computer program product cooperates with the pen enabled computing device to accomplish the functions and processes further described herein.
- the computer program product may operate, communicate, or otherwise interact with other computer program products such as, for example, personal information management (PIM) software or Web-based PIM or organizer software.
- PIM personal information management
- These separate computer program products may be executing on the pen enabled computing device or on an external computing device in communication with the pen enabled computing device.
- PIM personal information management
- a myriad of possible embodiemnts exist for editing, transferring, or otherwise manipulating data related to the nature of a page as identified by the system, computer program products, pen enabled computing device, and associated methods as described herein.
- Electronic ink based personal information management characterized by combinations of any or all of the below items: - electronic ink representations of items (e.g., schedule, to-do, contact) are stored in memory on the device for later transmission to personal information management (PLM) software running on a personal computer or to a web based organizer.
- PLM personal information management
- - electronic ink items may be defined by placing forms on the device and spatially segmenting the handwriting which is produced on them (depends on page identification).
- PIM system for displaying, modifying, augmenting, exporting, synchronizing, insuring consistency with paper items, conditionally translating handwriting to text, and printing mixed electronic ink and text items using PC PIM software or Web based PIM software.
- - items may be changed using personal information management software on the personal computer or Web organizer by standard techniques (e.g., keyboard and mouse editing, speech recognition, etc. - fields of the personal information management software on the personal computer or Web organizer accept and display both electronic ink and text
- the personal information management software displays a message which recommends that the user print a page containing the item for his planner, or change the item on the device using the technique claimed below, in order to synchronize the handwritten version of the item with the electronic version
- an automatic on-line handwriting recognizer may be used to convert the item to a representation more suitable to the external package
- - user configurable options may be set so that importing and/or exporting to and/or from other packages is automatic
- the user may be notified and required to resolve conflicts which may occur as a result of importing and/or exporting to and/or from other packages
- a message which indicates that a particular time slot has been reserved (e.g., "reserved") via handwriting may be exported to an external package.
- Such a message is sufficient in cases where the user wishes to keep the details of his appointments private from other external package users
- the personal information management software supports the printing of electronic ink based items, which may co ⁇ espond to different views of information in the database (e.g., items which correspond to a certain period of time)
- the user may be notified and required to resolve conflicts which may occur as a result of synchronizing the database on the PECS with the PC or Web PIM database - the PC PIM or Web Organizer software may or may not organize and display information in the same way as the paper and PECS software. For example, there may be a page labeled "Al" which is the first page of contacts whose last name begins with the letter "A.” The PC PIM software may group all contacts whose last name begins with "A" together for display purposes. If a change is made on the PC PIM side to handwritten information, the PIM software can warn the user that a change to information written on page "Al" occu ⁇ ed.
- the user can print information from multiple physical pages (e.g., print all of the contacts whose last name begins with "A"). This implies that physical page id is part of the record for a contact.
- - use keyword button on PECS if the user wants to access database by other fields — e.g., company name. could configure software to always try automatically recognizing certain fields.
- the PC PIM software may include a means for assigning a page identifier to a page printed from it.
- the printed page may have as its only electronic page identifier the fields co ⁇ esponding to its identity.
- Electronic page identifiers can consist of cross-hatched boxes. Location of box can be known by table lookup, using record value (if page id is part of item record) or other key value as index.
- the PECS display indicates which item was changed
- the user can be asked to type (using the PC or Web access device) the new information while viewing an image of the handwritten information.
- the user can be asked to confirm that the new information should indeed replace the old information in the PC PLM or Web organizer database.
- the change can be made with buttons on the device or control mechanisms on the paper, for instance a keypad on a preprinted form
- the printing is on the back side
- they may be placed in a ringed binder alongside co ⁇ esponding pages.
- the appointment schedule for a user is printed and placed alongside a schedule page for the same day, the user may make notes about the appointments on the right-hand schedule page which are captured by the PECS and treated accordingly by the PC PLM or Web organizer software, while viewing the details of the appointments on the facing page.
- a system for retrieving and displaying items on the PECS in response to user input - electronic versions of items may be displayed if available on the
- PECS when the user places the pen above or on the item or in response to other activations of control mechanisms.
- Other information associated with the item may optionally be displayed (e.g., date assigned in the case of a to-do item)
- the user may prefer to make a note or check a box or press a button indicating that an electronic version of the information is available for display on the device.
- the stroke that identifies the item to be changed may itself be the note that indicates that an electronic version of the item is available.
- a checkbox may be associated with certain items. Its state may be toggled when the pen is placed down in it (e.g., alarm on, alarm off; recurring to-do item on, recurring to-do item off). - the state of a checkbox may be displayed when the pen is placed over or in it (e.g., alarm is on)
- to-do list item gives information about the to-do list item, e.g. pen on the item shows the date the item was assigned on the LCD - marking an item as completed both electronically and on paper by checking a box associated with the item
- the identity of the to-do list form in use (e.g., "To-do list 11/29/98").
- the identity may be defined by the date and type of form and indicated by the method claimed above - on a to-do list, if it is assumed that the user writes items in sequence and does not skip an item or items, then the condition where the user has failed to co ⁇ ectly identify the page on which he is writing can be detected if the user writes in a slot which is greater than the next expected slot for the cu ⁇ ent software page. If this condition occurs, the user can be warned appropriately. 10.
- - holding the pen over or on some part of a contact list item gives information about the contact list item, e.g., pen on part of the item shows the contact's birthday, which may have been entered using the personal information management software on the personal computer or web organizer software
- - special boxes may be part of a contact list item, and may be used to indicate when certain events related to the contact occur, just by placing strokes in the boxes, because the device date and time stamps each stroke
- the personal information management software can display contacts which fall into date driven categories (e.g., here are the contacts you were in touch with least recently, here are the contacts you were in touch with most recently, here are all of the contacts with whom you were at a certain point in the sales cycle by the given date, etc.) 14.
- the physical paper context may not change as a result of changing the navigation context, that is, all handwriting is associated with the physical paper context.
- the "OK" button could mean "return to (e.g., display) the context where the user was writing.”
- the cu ⁇ ent paper context could be continuously displayed along with the cu ⁇ ent "navigation" context, when the two are different. The default is for the "paper context” and “navigation” contexts to be the same and be indicated by the page change operation.
- the page forward and page backward buttons of the PECS allow navigation through the machine usable items co ⁇ esponding to that page or page type, if the cu ⁇ ent page is not a notes page.
- the page forward and page backward buttons also allow navigation through the items co ⁇ esponding to the navigation context, if it is different from the page context.
- - Aids for navigation through machine usable data of the type co ⁇ esponding to a particular page can be printed on the paper.
- a page of contacts might include a "previous” box, a "next” box, a "name” box, a “phone” box, a "fax” box, and an "e-mail” box.
- Tapping the "next” box with the pen shows the next contact in machine usable form co ⁇ esponding to the current page in alphabetical order.
- Tapping the "fax" and "e-mail” boxes would display the fax number and e-mail address for that contact.
- a system for providing games for user familiarization, training, entertainment and education Operation and State Identification
- games which require the user to touch certain parts of the paper after a prompt can be included.
- a game modeled after "Operation" is one example. After the "Operation" page is identified, the display shows the name of a bone. The user must place the pen in the box co ⁇ esponding to the bone within a certain amount of time or a negative, low frequency sound is heard.
- Another example is a state identification game. A map of the United States is printed on the page.
- the name of a state is displayed and the state must be touched on the map within a certain period of time or a negative, low frequency sound is heard.
- a state capital name can be displayed and the co ⁇ esponding state must be touched.
- Another personal information management application is a trip meter.
- a starting point and a destination could be touched on a map, and the driving distance, driving time, and/or flying time between the two points could be displayed. 17.
- Expenses can be tracked using the PLM system. As long as the user writes down his expenses on the day they occur, they will be organized co ⁇ ectly by date for the expense reporting software automatically, because the device date and time stamps the writing.
- embodiments of the present invention provide a pen enabled computing device having the innate capability of processing or otherwise manipulating recorded pen input data.
- Embodiments of the present invention also provide a pen enabled computing device having the capability of detecting the presence of a page engaged with the device and determining the type of form with a minimum of separate identification devices and required user-initiated operations.
- the user uses the form according to its identified function and does not otherwise have to independently indicate the nature of the form to the pen enabled computing device.
- embodiments of the present invention provide a pen enabled computing device having the capability of determining the type of form engaged therewith without requiring additional components or special modification of the form to indicate the type of form to the device.
- Embodiments of the present invention also provide an associated method of identifying a form to a pen enabled computing device, while indicating the type of form as well as uniquely identifying forms having multiple pages and multiple pages of a single type of form.
- such a method accomplishes the identification of the page to the pen enabled computing system while occupying minimal, if any, space on the writing surface of the page.
- embodiments of the present invention provide a pen enabled computing device capable of specifically identifying the page being used therewith in order to facilitate future data modification, access, and manipulation in an efficient and expedient manner to provide enhanced seamless and automatic operation with minimal inconvenience and enhanced flexibility for the user.
Landscapes
- Engineering & Computer Science (AREA)
- General Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Human Computer Interaction (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- User Interface Of Digital Computer (AREA)
- Position Input By Displaying (AREA)
Abstract
Description
Claims
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
AU40638/00A AU4063800A (en) | 1999-03-31 | 2000-03-31 | System, computer program product, computing device and associated methods for form identification and information manipulation |
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US12719699P | 1999-03-31 | 1999-03-31 | |
US60/127,196 | 1999-03-31 |
Publications (3)
Publication Number | Publication Date |
---|---|
WO2000058821A2 WO2000058821A2 (en) | 2000-10-05 |
WO2000058821A3 WO2000058821A3 (en) | 2001-08-02 |
WO2000058821A9 true WO2000058821A9 (en) | 2002-06-27 |
Family
ID=22428798
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/US2000/008769 WO2000058821A2 (en) | 1999-03-31 | 2000-03-31 | System, computer program product, computing device and associated methods for form identification and information manipulation |
Country Status (2)
Country | Link |
---|---|
AU (1) | AU4063800A (en) |
WO (1) | WO2000058821A2 (en) |
Family Cites Families (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
DE69423296T2 (en) * | 1993-11-30 | 2000-11-30 | Hewlett Packard Co | Arrangement for data entry |
-
2000
- 2000-03-31 AU AU40638/00A patent/AU4063800A/en not_active Abandoned
- 2000-03-31 WO PCT/US2000/008769 patent/WO2000058821A2/en active Application Filing
Also Published As
Publication number | Publication date |
---|---|
AU4063800A (en) | 2000-10-16 |
WO2000058821A3 (en) | 2001-08-02 |
WO2000058821A2 (en) | 2000-10-05 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US7091959B1 (en) | System, computer program product, computing device, and associated methods for form identification and information manipulation | |
EP1014254B1 (en) | Multi-moded scanning pen with feedback | |
US7634718B2 (en) | Handwritten information input apparatus | |
US7353453B1 (en) | Method and system for categorizing data objects with designation tools | |
US6336125B2 (en) | Method and device for generating application data and storage medium containing thereon a program for generating application data | |
US5898434A (en) | User interface system having programmable user interface elements | |
Brandl et al. | Nicebook: supporting natural note taking | |
US7720286B2 (en) | System and method for associating handwritten information with one or more objects via discontinuous regions of a printed pattern | |
US5442795A (en) | System and method for viewing icon contents on a video display | |
US8913832B2 (en) | Method and device for interacting with a map | |
US6697056B1 (en) | Method and system for form recognition | |
CN101258461A (en) | Reading mode for electronic documents | |
Steimle | Pen-and-paper user interfaces: Integrating printed and digital documents | |
WO2007141204A1 (en) | System and method for recalling media | |
JPH1145149A (en) | Device and method for displaying data, recording medium, device and method for transmitting data | |
CN108885530A (en) | Visual user interface control is operated with ink order | |
US5644339A (en) | Electronic information apparatus | |
US20110199343A1 (en) | Method for generating and automatically sending a handwritten electronic message | |
JP6536214B2 (en) | Dictionary terminal, information display control method and information display control program | |
JP2001216311A (en) | Event analyzing device and program device stored with event analyzing program | |
WO2000058821A9 (en) | System, computer program product, computing device and associated methods for form identification and information manipulation | |
JP2008181510A (en) | Handwritten entry information collection and management system using digital pen | |
JP2001101162A (en) | Document processor and storage medium storing document processing program | |
JP6019716B2 (en) | Computer apparatus, program, and information processing system | |
JP2013016079A (en) | Computer device, program, and notebook |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
AK | Designated states |
Kind code of ref document: A2 Designated state(s): AE AG AL AM AT AT AU AZ BA BB BG BR BY CA CH CN CR CU CZ CZ DE DE DK DK DM DZ EE EE ES FI FI GB GD GE GH GM HR HU ID IL IN IS JP KE KG KP KR KZ LC LK LR LS LT LU LV MA MD MG MK MN MW MX NO NZ PL PT RO RU SD SE SG SI SK SK SL TJ TM TR TT TZ UA UG US UZ VN YU ZA ZW |
|
AL | Designated countries for regional patents |
Kind code of ref document: A2 Designated state(s): GH GM KE LS MW SD SL SZ TZ UG ZW AM AZ BY KG KZ MD RU TJ TM AT BE CH CY DE DK ES FI FR GB GR IE IT LU MC NL PT SE BF BJ CF CG CI CM GA GN GW ML MR NE SN TD TG |
|
121 | Ep: the epo has been informed by wipo that ep was designated in this application | ||
DFPE | Request for preliminary examination filed prior to expiration of 19th month from priority date (pct application filed before 20040101) | ||
AK | Designated states |
Kind code of ref document: A3 Designated state(s): AE AG AL AM AT AT AU AZ BA BB BG BR BY CA CH CN CR CU CZ CZ DE DE DK DK DM DZ EE EE ES FI FI GB GD GE GH GM HR HU ID IL IN IS JP KE KG KP KR KZ LC LK LR LS LT LU LV MA MD MG MK MN MW MX NO NZ PL PT RO RU SD SE SG SI SK SK SL TJ TM TR TT TZ UA UG US UZ VN YU ZA ZW |
|
AL | Designated countries for regional patents |
Kind code of ref document: A3 Designated state(s): GH GM KE LS MW SD SL SZ TZ UG ZW AM AZ BY KG KZ MD RU TJ TM AT BE CH CY DE DK ES FI FR GB GR IE IT LU MC NL PT SE BF BJ CF CG CI CM GA GN GW ML MR NE SN TD TG |
|
REG | Reference to national code |
Ref country code: DE Ref legal event code: 8642 |
|
AK | Designated states |
Kind code of ref document: C2 Designated state(s): AE AG AL AM AT AT AU AZ BA BB BG BR BY CA CH CN CR CU CZ CZ DE DE DK DK DM DZ EE EE ES FI FI GB GD GE GH GM HR HU ID IL IN IS JP KE KG KP KR KZ LC LK LR LS LT LU LV MA MD MG MK MN MW MX NO NZ PL PT RO RU SD SE SG SI SK SK SL TJ TM TR TT TZ UA UG US UZ VN YU ZA ZW |
|
AL | Designated countries for regional patents |
Kind code of ref document: C2 Designated state(s): GH GM KE LS MW SD SL SZ TZ UG ZW AM AZ BY KG KZ MD RU TJ TM AT BE CH CY DE DK ES FI FR GB GR IE IT LU MC NL PT SE BF BJ CF CG CI CM GA GN GW ML MR NE SN TD TG |
|
COP | Corrected version of pamphlet |
Free format text: PAGES 1/8-8/8, DRAWINGS, REPLACED BY NEW PAGES 1/8-8/8; DUE TO LATE TRANSMITTAL BY THE RECEIVING OFFICE |
|
122 | Ep: pct app. not ent. europ. phase | ||
NENP | Non-entry into the national phase in: |
Ref country code: JP |