EP1269303A1 - Processing of documents - Google Patents

Processing of documents

Info

Publication number
EP1269303A1
EP1269303A1 EP01918045A EP01918045A EP1269303A1 EP 1269303 A1 EP1269303 A1 EP 1269303A1 EP 01918045 A EP01918045 A EP 01918045A EP 01918045 A EP01918045 A EP 01918045A EP 1269303 A1 EP1269303 A1 EP 1269303A1
Authority
EP
European Patent Office
Prior art keywords
document
information
editing
computer
coding pattern
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Withdrawn
Application number
EP01918045A
Other languages
German (de)
French (fr)
Inventor
Tomas LUNDSTRÖM
Petter Ericson
Christofer Akersten
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Anoto AB
Original Assignee
Anoto AB
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Anoto AB filed Critical Anoto AB
Publication of EP1269303A1 publication Critical patent/EP1269303A1/en
Withdrawn legal-status Critical Current

Links

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/03Arrangements for converting the position or the displacement of a member into a coded form
    • G06F3/033Pointing devices displaced or positioned by the user, e.g. mice, trackballs, pens or joysticks; Accessories therefor
    • G06F3/0354Pointing devices displaced or positioned by the user, e.g. mice, trackballs, pens or joysticks; Accessories therefor with detection of 2D relative movements between the device, or an operating part thereof, and a plane or surface, e.g. 2D mice, trackballs, pens or pucks
    • G06F3/03545Pens or stylus
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/03Arrangements for converting the position or the displacement of a member into a coded form
    • G06F3/0304Detection arrangements using opto-electronic means
    • G06F3/0317Detection arrangements using opto-electronic means in co-operation with a patterned surface, e.g. absolute position or relative movement detection for an optical mouse or pen positioned with respect to a coded surface
    • G06F3/0321Detection arrangements using opto-electronic means in co-operation with a patterned surface, e.g. absolute position or relative movement detection for an optical mouse or pen positioned with respect to a coded surface by optically sensing the absolute position with respect to a regularly patterned surface forming a passive digitiser, e.g. pen optically detecting position indicative tags printed on a paper sheet
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/048Interaction techniques based on graphical user interfaces [GUI]
    • G06F3/0487Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser
    • G06F3/0488Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser using a touch-screen or digitiser, e.g. input of commands through traced gestures
    • G06F3/04883Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser using a touch-screen or digitiser, e.g. input of commands through traced gestures for inputting data by handwriting, e.g. gesture or text
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F40/00Handling natural language data
    • G06F40/10Text processing
    • G06F40/166Editing, e.g. inserting or deleting
    • G06F40/171Editing, e.g. inserting or deleting by use of digital ink
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V30/00Character recognition; Recognising digital ink; Document-oriented image-based pattern recognition
    • G06V30/10Character recognition
    • G06V30/22Character recognition characterised by the type of writing
    • G06V30/228Character recognition characterised by the type of writing of three-dimensional handwriting, e.g. writing in the air

Definitions

  • the invention relates to methods and devices intended for editing electronically stored documents.
  • US Patent Specification US-5,897,648 describes a device and a method for editing electronic documents .
  • An original document is scanned into a computer where an electronic version of the document is stored.
  • the origi- nal is then placed on an X/Y digitizing tablet and the position of the document on the digitizing tablet is correlated with the scanned version of the document.
  • Editing is then carried out on the digitizing tablet with the aid of a digitizing pen coupled to the tablet.
  • the editing is done in the form of markings on the original, the positions of the markings being transferred via the tablet to the computer.
  • the editing markings are interpreted and converted into electronic form in the computer, after which the edited electronic document is displayed.
  • the use of an invention according to US-5,897,648 has the very significant disadvantage that a user is forced to be located at the computer or at least at the digitizing tablet when editing.
  • a general problem is thus how to provide for simple and flexible editing of electronic documents.
  • the invention is characterized m that, on the basis of an electronically stored document, which can be of any type such as text, image, drawing etc., the document is printed on a surface, preferably a sheet of paper, which is provided with a position-coding pattern. Manual editing is then carried out on the printout surface with a device which comprises means for reading the position-coding pattern and also a pen point for marking on the surface.
  • Editing is done by means of a code which is m the form of sym- bols from a predetermined set of symbols, on the sheet of paper. Transferring of the editing information, i.e. the symbols and markings applied to the surface, is done to a storage and processing device, preferably a computer. This transfer can be done directly during the editing or on a later occasion. Interpretation of the editing code is then done, wholly or partially m the computer, m interaction with the document stored m the computer, whereafter changes are made to the stored document m direct dependence on the interpretation.
  • the present invention allows for a plurality of users to edit one and the same document.
  • Fig. 1 schematically shows an embodiment of a product which is provided with a position-coded pattern.
  • Figs 2a-2d schematically show how the symbols can be configured in an embodiment of the invention.
  • Fig. 3 schematically shows an example of 4x4 symbols which are used for coding a position.
  • Fig. 4 schematically shows a device according to the present invention which can be used for position-determination in three dimensions.
  • Fig. 5a shows a printout of a text document with manually drawn editing instructions.
  • Fig. 5b shows a printout of a drawing document with a manually drawn editing instruction.
  • Fig. 5c shows a flow chart of a method according to the invention.
  • Fig. 6 schematically shows an embodiment of a system for editing document information according to the present invention.
  • a coding pattern will be presented with reference to Figs 1, 2a-d and 3.
  • This coding pattern represents position information which can be used in a method according to the invention.
  • a device which is intended to be used in manual editing of a printed document is presented in connection with Fig. 4.
  • the device which is pen-shaped, reads the position-coding pattern and where applicable also text and is provided with a pen point to make editing information on the printed document visible.
  • Figs 5a and 5b how examples of manual editing information are drawn on a printout of a text document (Fig.
  • Fig. 1 shows a part of a product m the form of a sheet of paper 1 which is provided on its surface 2 with an optically readable position-coding pattern 3 enabling position-determination to be carried out.
  • the position- coding pattern consists of symbols 4 which are systematically arranged over the surface 2 so that it has a "patterned" appearance.
  • the sheet of paper has an x coordinate axis and a y coordinate axis. In this case, position determination can be carried out on the entire sur- face of the product. In other cases, the surface which allows position determination may consist of a smaller part of the product.
  • the sheet of paper can be used for producing an electronic representation of information which is written or drawn on the surface. The electronic representation can be produced by continuously determining the position of the pen on the sheet of paper by reading the position-coding pattern while writing on the surface with a pen.
  • the position-coding pattern comprises a virtual raster which is thus neither apparent to the human eye nor can it be detected directly by a device which is to determine positions on the surface, and a plurality of symbols 4 each of which can assume one of four values " l " - “ 4 " as described in the text which follows.
  • the position-coding pattern in Fig. 1 has been greatly enlarged for the sake of clarity. Moreover, it is only shown on a part of the sheet of paper.
  • the position-coding pattern is arranged in such a manner that the position of a partial surface on the writing surface is coded by the symbols on this partial surface.
  • a first and a second partial surface 5a, 5b are shown by dashed lines in Fig. 1.
  • the part of the position-coding pattern (in this case 3 x 3 symbols) , which is located on the first partial surface 5a, codes a first position
  • the part of the position-coding pattern which is located on the second partial surface 5b codes a second position.
  • the position-coding pattern is thus partly common to the adjoining first and second positions.
  • Such a position- coding pattern is designated as "floating" in this application.
  • Figs 2a-d show an embodiment of a symbol which can be used in the position-coding pattern according to the invention.
  • the symbol comprises a virtual raster point 6 which is represented by the intersection between the raster lines, and a marking 7 which has the form of a dot.
  • the value of the symbol depends on where the marking is placed. In the example in Fig. 2, there are four possible placements, one on each one of the raster lines which extend from the raster points . The displacement from the raster point is equal for all values. In the text which follows, the symbol has the value 1 in
  • Each symbol can thus represent four values "1-4".
  • This means that the position-coding pattern can be divided into a first position code for the x coordinate and a second position code for the y coordinate. The dividing is done as follows :
  • each symbol is thus translated into a first digit, in this case bit, for the x code and a second digit, in this case bit, for the y code.
  • the patterns can be combined into a common pattern which is coded graphically with the aid of a plurality of symbols according to Fig. 2.
  • Each position is coded with the aid of a plurality of symbols.
  • 4x4 symbols are used for coding a position in two dimensions, i.e. an x coordinate and a y coordinate .
  • the position code is built up with the aid of a number series of ones and zeros which has the characteristic that no sequence of four bits occurs more than once in the series.
  • the number series is cyclic, which means that the characteristic also applies when the end of the series is coupled together with its beginning. A sequence of four bits thus always has an unambiguously determined position in the number series.
  • the series can have a maximum length of 16 bits if it is to have the characteristic described above for sequences of four bits. In this example, however, only a seven-bit- long series is used as follows:
  • This series contains seven unique sequences of four bits which code a position in the series as follows:
  • the number series is written sequentially in columns over the entire surface which is to be coded.
  • the coding is based on the difference or position displacement between numbers in adjoining columns.
  • the magnitude of the difference is determined by the position in the number series at which one allows the column to begin (i.e. with which sequence) . More specifically, taking the difference modulo seven between, on the one hand, a number which is coded by a four-bit sequence in a first column and which thus can have the value (position) 0-6, and, on the other hand, a corresponding number (i.e. the sequence at the same
  • the x code is structured in such a manner that the differences will be as follows:
  • Each x coordinate is thus coded with two numbers between 3 and 6 and a subsequent number which is 1 or 2. Subtracting three from the high numbers and one from the low one provides a number in mixed base which directly yields a position in the x direction from which the x coordinate can then be determined directly as shown in the example below. Using the principle described above, x coordinates
  • 0,1,2... can thus be coded with the aid of numbers which represent three differences. These differences are coded with a bit pattern which is based on the number series above. Finally, the bit pattern can be coded graphically with the aid of the symbols in Fig. 2.
  • the y coordinates are coded according to the same principle as used for the x coordinates .
  • the cyclic number series is written repeatedly in horizontal rows over the surface which is to be position-coded. Exactly as in the case of the x coordinates, the rows are allowed to begin at different positions, i.e. with different sequences, in the number series. However, for the y coordinates, it is not differences which are used but the coordinates are coded with numbers which are based on the starting position of the number series in each row.
  • the x coordinate for 4x4 symbols it is possible to determine the starting positions in the number series for the rows which are included in the y code in the 4x4 symbols.
  • the most significant digit is determined by allowing it to be the only one which has a value in a specific range.
  • a row of four is allowed to begin at position 0-1 in the number series to indicate that this row relates to the least significant digit in a y coordinate, and the other three begin at position 2-6.
  • a number series according to the following is thus found:
  • Each y coordinate is thus coded with three numbers between 2 and 6 and a subsequent number between 0 and 1.
  • Subtracting 1 from the low number and 2 from the high ones provides, in a corresponding manner as for the x direction, a position in the y direction in mixed base from which the y coordinate can be determined directly.
  • 5 x 5 x 5 x 2 250 positions in the y direction.
  • the starting position for the first number series in the first column can be calculated once the x coordinate has been determined.
  • the above-mentioned seven different starting positions for the first series can code different sheets or writing surfaces on a product .
  • Fig. 3 shows an example of an image with 4x4 symbols which are read by a device for position determination.
  • the vertical x sequences code the following positions in the number series: 2 0 4 6.
  • each position is coded with 4 x 4 symbols and a number series with 7 bits is used.
  • Positions can be coded with more or fewer symbols.
  • the number of symbols need not be the same in both directions.
  • the number series can be of different length and does not need to be binary but may be based on another base. Different number series can be used for coding in the x direction and coding in the y direction.
  • the symbols can have different numbers of values.
  • the marking is a dot.
  • it may have a different appearance.
  • it may consist of a line which begins at the virtual raster point and extends therefrom to a predetermined position.
  • the symbols are used within a square partial surface for coding a position.
  • the partial surface can have a different shape, for example hexagonal.
  • the symbols do not need to be arranged in rows and columns at an angle of 90 degrees to each other either, but can also be arranged in some other manner.
  • the virtual raster For the position codes to be detected, the virtual raster must be determined. This can be done by studying the distance between different markings. The shortest distance between two markings must originate from two adjoining symbols with the value 1 and 3 so that the markings are lying on the same raster line between two raster points. When such a pair of markings has been detected, the associated raster points can be determined with knowledge of the distance between the raster points and the displacement of the markings from the raster points. Once two raster points have been located, additional raster points can be determined with the aid of measured distances to other markings and with knowledge of the relative distance of the raster points.
  • An embodiment of a device for position determination, whose spatial relationship to a surface can be determined, is shown schematically in Fig. 4.
  • the casing mainly contains an optical part, an electronic part and a power supply.
  • the optical part comprises at least one light-emitting diode 13 for illuminating the surface which is to be imaged, and a light-sensitive area sensor 14, for example a CCD or CMOS sensor, for registering a two-dimensional image.
  • the device may also comprise a lens system (not shown) .
  • the power supply for the device is obtained from a battery 15 which is mounted in a separate compartment in the casing.
  • the electronic part contains image-processing means 16 for determining a position on the basis of the image registered by means of the sensor 14 and more specifically a processor unit with a processor which is programmed to read images from the sensor and to perform position determination on the basis of these images.
  • the processor unit may also store an identifier which is unique for each reading device.
  • This device identifier is usually composed of a plurality of digits and/or characters, and can be used in coded form. The identifier is of use when performing the method according to the invention where a user is identified, by the device identifier, as a particular editor of a document.
  • the device comprises a pen point 17, with the aid of which it is possible to write normal pigment-based writing on the surface on which the position determination is to occur.
  • the pen point 17 can be retracted and extended so that the user can control whether or not it is to be used.
  • the pen point 17 may be non-retractable and covered by a cap (not shown) when not in use. In certain applications, the device does not need to have a pen point at all.
  • the device also comprises buttons 18, with the aid of which the device is activated and controlled. It also has a transceiver 19 for wireless transmission, e.g. by means of IR light or radio waves, of information to and from the device.
  • the device can also comprise a display 20 for showing positions or registered information.
  • Applicant's international patent publication WO 98/20446 describes a device for registering text. This device can be used for position determination if it is programmed in a suitable manner. If it is to be used for pigment-based writing, it must also have a pen point.
  • the device can be divided into different physical casings, a first casing containing components which are necessary for obtaining images of the position-coding pattern and for transferring these to components which are located in a second casing and which carry out the position determination on the basis of the image or images registered.
  • the position determination is done by a processor which thus must have software for locating and decoding the symbols in an image and for determining positions from the codes thus obtained.
  • a person skilled in the art can design software on the basis of the above example, which carries out position determination on the basis of an image of a part of a position-coding pattern.
  • the person skilled in the art can design software on the basis of the above description, for printing the position-coding pattern.
  • the raster is a rectangular grid. It can also have other forms, i.e. non-rectangular.
  • the pattern is optically readable and the sensor thus is an optical sensor.
  • other position-coding patterns could be used within the scope of the present invention, for example the pattern described in applicant's international patent publication WO 00/01670, or the prior art position-coding patterns as disclosed in US-A-5 051 736, WO 00/31682 and EP-A-0 469 864.
  • the pattern could be based on another parameter than an optical parameter.
  • the sensor must be of a type which can read the parameter in question. Examples of such parameters are chemical, acoustic or electromagnetic symbols. Also capacitive or inductive symbols can be used.
  • Fig. 5a shows a printout 501 of a text document which is preferably stored in a computer.
  • the printout 501 is suitably done on a sheet of paper, the surface of which is provided with a position-coding pattern as described above in connection with Figs 1-3. For the sake of clarity, however, no such pattern is shown in Fig. 5a.
  • the digital pen of Fig. 4 is used for providing editing information on the printout 501.
  • Such editing information is schematically exemplified m Fig. 5a at reference numerals 502, 503 and 504.
  • An incorrect word “green” has been stricken through with a horizontal line and has been marked m the right-hand margin that it is to De replaced by the word "gold' 503.
  • the words 'fade” and “not” have been marked 504 with an editing symbol which, on interpretation, is to indicate that the words are to change places with one another.
  • Fig. 5b very schematically shows a drawing consisting of simple geometric figures.
  • a cross marking 505 has been drawn on one side of a rectangle m order to indicate that the side is to be erased.
  • FIG. 5c A method for performing editing according to the invention, and with reference to the example of figure 5a, is illustrated m figure 5c.
  • a read and search step 510 the text of the document is analyzed m that each letter is associated with its position and at the same time a search for editing symbols with corresponding positions is performed. Then, m a number of decision steps 511,513,515,517 tne editing symbols are identified and corresponding interpretation steps 512,514,516,518 are performed.
  • the identity of the digital pen is also determined m that a transfer of a unique device identifier is performed from the digital pen to the means for storing the document, typically a memory unit m a computer or server.
  • a particular digital pen may be associated with a particular user, i.e. editor or reviewer of the document. If several editors, each with a digital pen having a unique device identity, make corrections to the text, these corrections may then be associated with the different users.
  • the editing symbol of step 511 is found, a corresponding editing symbol is searched for in the margin area of the printed document as well as any handwritten text close to that editing symbol. Such text is interpreted using ICR-technique and the interpreted text is then inserted at the appropriate location in the stored document.
  • step 513 If the editing symbol of step 513 is found, the text in the stored document which is located at the same position as the editing symbol is removed from the stored document.
  • step 515 a corresponding editing symbol is searched for in the margin area of the printed document as well as any handwritten text close to that editing symbol.
  • Such text is interpreted using ICR-technique and the interpreted text is then inserted at the appropriate location in the stored document, replacing the letter at the position of the editing symbol in the stored text.
  • each editing symbol results in an editing command being generated, either in the pen itself or in a computer unit that communicates with the pen and comprises software for identifying editing symbols.
  • the present invention allows for a plurality of editors to edit one and the same document. By associating editing commands with timestamps and users, e.g. by way of unique identity codes associated with the digital pens, a sequence of editing commands can be interpreted by the computer which holds the electronic version of the document .
  • the editing symbols can be a subset of a larger set of symbols comprising more or less complicated indications of how the subsequent interpretation should be carried out and how the real changes of the stored document are to appear.
  • the corresponding set of editing commands can be determined in advance or be generated by, for example, a user through a suitable learning process according to the prior art technique.
  • Fig. 6 shows a system that can be used m carrying out the method described above.
  • the system includes a plurality of pen-shaped reading devices 601, a central unit 602, a local processing unit 603, and a printer unit 604.
  • the reading devices 601 correspond to the digital pen shown m Fig. 4 and are arranged for wireless communication with the central unit 602, for example directly via a telecommunications network, or indirectly via short-range communication to a modem unit (not shown), such as a mobile phone, a personal computer, a Bluetooth ® node etc, which turn connects to the central unit 602 via a suitable network, for example the Internet, a telecommunications network, a LAN etc.
  • a suitable network for example the Internet, a telecommunications network, a LAN etc.
  • the central unit 602 is arranged for communication with the local processing unit 603, typically a computer device such as a personal computer, m which the electronic document to be annotated is created.
  • the central unit acts, inter alia , as a pattern administration unit.
  • the communication between the local processing unit 603 and the central unit 602 can be handled by a messaging service, such as e-mail, if direct communication is unavailable.
  • the local processing unit 603 is arranged to communicate with a printer unit 604.
  • the author registers the document with the central unit 602.
  • the central unit 602 receives from the local processing unit 603 a file identifier and data on the number of standard pages of the document to be registered.
  • the central unit 602 receives an electronic copy of the document.
  • the pages of the electronic document are assigned a corresponding number of pattern pages, i.e. mutually unique subsets of the above- described position-coding pattern.
  • the author distributes the thus-locked document to different reviewers .
  • the author can send the electronic document from the local processing unit 603 to the printer unit 604 and then physically distribute the printouts, or distribute the document electronically so that the reviewers can print the document on any nearby printer units (not shown) .
  • the different reviewers might themselves download the document from the central unit to a respective computer, from which the document can be sent to a suitable printer unit.
  • each reviewer uses his reading device 601 to write on his printout 605, wherein the reading device 601 electronically registers the positions where it is applied to the printout, thereby producing a digital copy of editing information written on the pages of the printout.
  • the digital editing information i.e. position data read by the reading device 601
  • the central unit 602 together with a reading device identifier, and optionally timestamps indicating when the annotations were made.
  • the synchronization can be effected, for example, by the reviewer checking a send icon (not shown) on the printout 605, the send icon being provided with a specific part of the position-coding pattern.
  • the central unit 602 registers one or more positions coded by this specific part of the pattern, it initiates the synchronization.
  • the reviewer can operate a button on the reading device 601 to initiate the synchronization.
  • the digital editing information can be sent to the central unit 602 in real time, i.e. while the reviewer is actually writing the editing information on the printout 605.
  • the central unit 602 issues a message to the local processing device 603, whereupon the author can connect to the central unit 602 to download the feedback from the reviewers, one or more at a time.
  • the central unit 602 also translates the editing information to local positions on each page of the electronic document, so that the local processing unit 603 can incorporate the editing information into the electronic document to be displayed to the author.
  • the incorporation can be effected by the central unit 602 itself which then transmits an electronic file back to the local processing unit 603 for display to the author .
  • the pattern administration functionality of the central unit is incorporated into the local processing unit, so that one and more reading devices communicate with the local processing unit, typically a personal computer, a PDA or the like.
  • the central unit only serves the function of assigning pattern pages to the electronic document, i.e. the local processing unit receives a file with the pattern pages, or parameters for calculating the same, from the central unit. The local processing unit then communicates with the digital pens and produces an updated document based on the received editing information.
  • the digital pen is capable of interpreting the editing symbols, so that the pen can transmit an editing command to the local processing unit.
  • the pen is also capable of effecting an ICR analysis of relevant parts of the editing information written on the printout.
  • the author and the reviewers have been described as separate individuals, it should be evident that in the method and system described above the author act as a reviewer as well. It is also to be understood that the central unit can communicate with a plurality of local processing units, to thereby act as a hub in an infrastructure for handling information. It should also be noted that the incorporation of the editing information might or might not include interpreting the markings made by the reviewers on the sheet 605 containing the printed text. In some reviewing work, it may be desirable to enable for one or more reviewers to simply annotate the text, in the margins for example, and to associate those annotations with specific positions within the text and to enable an owner of the document to view the text together with the annotations.
  • the reviewer is allowed to write brief comments or draw sketches in the margins of the document to aid the author in his further drafting of the document.
  • the document information of the updated electronic document includes the original text and the handwritten annotations made by the reviewers.
  • the reviewer can be allowed to use both annotations and editing symbols/markings in one and the same document. This can be achieved, for example, by using a specific set of annotation symbols which identifies a piece of editing information as an annotation.
  • the local processing unit 603 suitably includes presentation software which can be arranged to present the annotations in the form of, e.g., pop-up windows or the like overlaid on the original document.
  • presentation software could be set to prompt the author for approval before inserting any interpreted text associated with an editing symbol into the electronic document.
  • This software could be contained in the system or distributed to the system on one or more computer readable media, for example a volatile memory or a nonvolatile memory such as a floppy diskette or CD-ROM.
  • the software could also be distributed on computer readable media in the form of propagated signals, such as the stream of bits that represent Internet transmissions of packets or the carrier waves that are transmitted to satellites.

Abstract

An electronically stored document which can be of any type such as text, image, drawing etc. is printed on a surface, preferably a sheet of paper, which is provided with a position-coding pattern. Manual editing is then carried out on the printout surface with a digital pen which comprises means for reading the position-coding pattern and also a pen point for marking on the surface. Editing is done by means of a code which is in the form of symbols from a predetermined set of symbols, on the sheet of paper. The editing information, i.e. the symbols applied to the surface, is digitally registered by the digital pen and transferred to the storage and processing device, preferably a computer. This transfer can be done directly during the editing or on a later occasion. Interpretation of the editing code is then carried out, wholly or partly in the computer, in interaction with the document stored in the computer, whereupon changes are made to the stored document in direct dependence on the interpretation.

Description

PROCESSING OF DOCUMENTS
TECHNICAL FIELD
The invention relates to methods and devices intended for editing electronically stored documents.
BACKGROUND
US Patent Specification US-5,897,648 describes a device and a method for editing electronic documents . An original document is scanned into a computer where an electronic version of the document is stored. The origi- nal is then placed on an X/Y digitizing tablet and the position of the document on the digitizing tablet is correlated with the scanned version of the document. Editing is then carried out on the digitizing tablet with the aid of a digitizing pen coupled to the tablet. The editing is done in the form of markings on the original, the positions of the markings being transferred via the tablet to the computer. The editing markings are interpreted and converted into electronic form in the computer, after which the edited electronic document is displayed. The use of an invention according to US-5,897,648 has the very significant disadvantage that a user is forced to be located at the computer or at least at the digitizing tablet when editing.
A general problem is thus how to provide for simple and flexible editing of electronic documents.
SUMMARY OF THE INVENTION
It is an object of the invention to address prob- lems associated with the prior art This object aim is achieved by a method and a computer program according to claim 1 and 2 below and a system according to claim 8 and 9. In its most general form, the invention is characterized m that, on the basis of an electronically stored document, which can be of any type such as text, image, drawing etc., the document is printed on a surface, preferably a sheet of paper, which is provided with a position-coding pattern. Manual editing is then carried out on the printout surface with a device which comprises means for reading the position-coding pattern and also a pen point for marking on the surface. Editing is done by means of a code which is m the form of sym- bols from a predetermined set of symbols, on the sheet of paper. Transferring of the editing information, i.e. the symbols and markings applied to the surface, is done to a storage and processing device, preferably a computer. This transfer can be done directly during the editing or on a later occasion. Interpretation of the editing code is then done, wholly or partially m the computer, m interaction with the document stored m the computer, whereafter changes are made to the stored document m direct dependence on the interpretation. Although it is m many cases convenient to print the document information onto a pre-prmted sheet on which position coding pattern already has been provided, it may be desirable to have an extra degree of freedom m that the document information is printed together with the position coding pattern. Such an aspect of the present invention is useful m situations where a user has the capability to calculate a position-coding pattern as will be discussed in some detail below.
The present invention allows for a plurality of users to edit one and the same document. By associating editing commands with timestamps and users, e.g. by way of identity codes associated with the reading devices, a sequence of editing commands can be interpreted by the computer which holds the electronic version of the document .
A number of advantages of the invention, which are associated with simplicity of processing, are evident: it is simple and easy to understand, i.e. it is a question of intuitive processing of documents of the traditional type, i.e. paper printouts. This also results in a low learning threshold for the persons who are to carry out the editing.
Furthermore, it is an advantage that it is simple to edit documents without the person editing needing to be located at a computer where the document is stored, or to be tied to a complicated input device which in the prior art is exemplified by a digitizing tablet. The editing information can thus be advantageously stored in the input device for subsequent transfer to the computer/ storage location.
It is also advantageous to directly obtain a copy of the editing in the form of the manually edited printout as offered by the invention.
BRIEF DESCRIPTION OF THE DRAWINGS
Fig. 1 schematically shows an embodiment of a product which is provided with a position-coded pattern.
Figs 2a-2d schematically show how the symbols can be configured in an embodiment of the invention.
Fig. 3 schematically shows an example of 4x4 symbols which are used for coding a position.
Fig. 4 schematically shows a device according to the present invention which can be used for position-determination in three dimensions.
Fig. 5a shows a printout of a text document with manually drawn editing instructions.
Fig. 5b shows a printout of a drawing document with a manually drawn editing instruction.
Fig. 5c shows a flow chart of a method according to the invention.
Fig. 6 schematically shows an embodiment of a system for editing document information according to the present invention.
PREFERRED EMBODIMENTS
For the sake of clarity, the detailed description of the invention below has been divided into a number of part- descriptions. As an introduction, a coding pattern will be presented with reference to Figs 1, 2a-d and 3. This coding pattern represents position information which can be used in a method according to the invention. After presentation of the coding pattern, a device which is intended to be used in manual editing of a printed document is presented in connection with Fig. 4. The device, which is pen-shaped, reads the position-coding pattern and where applicable also text and is provided with a pen point to make editing information on the printed document visible. After that is shown, with reference to Figs 5a and 5b, how examples of manual editing information are drawn on a printout of a text document (Fig. 5a) and a document (F g. 5b) which contains a drawing figure. Then, a method and system for editing document information is presented with reference to Fig. 5c and Fig. 6. The position-coding pattern and the reading device illustrated m Figs 1-4 below are further described in applicant's international patent publication WO 01/16691, as well as m applicant's international patent applications PCT/SE00/01895 and PCT/SE00/01897 , all of which are incorporated herein by reference.
Fig. 1 shows a part of a product m the form of a sheet of paper 1 which is provided on its surface 2 with an optically readable position-coding pattern 3 enabling position-determination to be carried out. The position- coding pattern consists of symbols 4 which are systematically arranged over the surface 2 so that it has a "patterned" appearance. The sheet of paper has an x coordinate axis and a y coordinate axis. In this case, position determination can be carried out on the entire sur- face of the product. In other cases, the surface which allows position determination may consist of a smaller part of the product. For example, the sheet of paper can be used for producing an electronic representation of information which is written or drawn on the surface. The electronic representation can be produced by continuously determining the position of the pen on the sheet of paper by reading the position-coding pattern while writing on the surface with a pen.
The position-coding pattern comprises a virtual raster which is thus neither apparent to the human eye nor can it be detected directly by a device which is to determine positions on the surface, and a plurality of symbols 4 each of which can assume one of four values " l " - " 4 " as described in the text which follows. In this connection, it should be pointed out that the position-coding pattern in Fig. 1 has been greatly enlarged for the sake of clarity. Moreover, it is only shown on a part of the sheet of paper.
The position-coding pattern is arranged in such a manner that the position of a partial surface on the writing surface is coded by the symbols on this partial surface. A first and a second partial surface 5a, 5b are shown by dashed lines in Fig. 1. The part of the position-coding pattern (in this case 3 x 3 symbols) , which is located on the first partial surface 5a, codes a first position, and the part of the position-coding pattern which is located on the second partial surface 5b codes a second position. The position-coding pattern is thus partly common to the adjoining first and second positions. Such a position- coding pattern is designated as "floating" in this application. Figs 2a-d show an embodiment of a symbol which can be used in the position-coding pattern according to the invention. The symbol comprises a virtual raster point 6 which is represented by the intersection between the raster lines, and a marking 7 which has the form of a dot. The value of the symbol depends on where the marking is placed. In the example in Fig. 2, there are four possible placements, one on each one of the raster lines which extend from the raster points . The displacement from the raster point is equal for all values. In the text which follows, the symbol has the value 1 in
Fig. 2a, the value 2 in Fig. 2b, the value 3 in Fig. 2c and the value 4 in Fig. 2d. In other words, there are four different types of symbols.
Each symbol can thus represent four values "1-4". This means that the position-coding pattern can be divided into a first position code for the x coordinate and a second position code for the y coordinate. The dividing is done as follows :
The value of each symbol is thus translated into a first digit, in this case bit, for the x code and a second digit, in this case bit, for the y code. In this manner, two completely independent bit patterns are obtained. The patterns can be combined into a common pattern which is coded graphically with the aid of a plurality of symbols according to Fig. 2. Each position is coded with the aid of a plurality of symbols. In this example, 4x4 symbols are used for coding a position in two dimensions, i.e. an x coordinate and a y coordinate .
The position code is built up with the aid of a number series of ones and zeros which has the characteristic that no sequence of four bits occurs more than once in the series. The number series is cyclic, which means that the characteristic also applies when the end of the series is coupled together with its beginning. A sequence of four bits thus always has an unambiguously determined position in the number series.
The series can have a maximum length of 16 bits if it is to have the characteristic described above for sequences of four bits. In this example, however, only a seven-bit- long series is used as follows:
"0 0 0 1 0 1 0"
This series contains seven unique sequences of four bits which code a position in the series as follows:
For coding the x coordinate, the number series is written sequentially in columns over the entire surface which is to be coded. The coding is based on the difference or position displacement between numbers in adjoining columns. The magnitude of the difference is determined by the position in the number series at which one allows the column to begin (i.e. with which sequence) . More specifically, taking the difference modulo seven between, on the one hand, a number which is coded by a four-bit sequence in a first column and which thus can have the value (position) 0-6, and, on the other hand, a corresponding number (i.e. the sequence at the same
"level") in an adjoining column, the result will be the same independently of where along the two columns the comparison is made. It is thus possible to code an x coordinate which is constant for all y coordinates with the aid of the difference between two columns.
Since each position on the surface is coded with
4x4 symbols in this example, three differences (having the value 0-6) according to the above are available for coding the x coordinate. The coding is then carried out in such a manner that the three differences will always have the value 1 or 2 and the other two will have values within the range of 3-6. Thus, no differences may be zero in the x code. In other words, the x code is structured in such a manner that the differences will be as follows:
(3-6) (3-6) (1-2) (3-6) (3-6) (1-2) (3-6) (3-6) (1-2)...
Each x coordinate is thus coded with two numbers between 3 and 6 and a subsequent number which is 1 or 2. Subtracting three from the high numbers and one from the low one provides a number in mixed base which directly yields a position in the x direction from which the x coordinate can then be determined directly as shown in the example below. Using the principle described above, x coordinates
0,1,2... can thus be coded with the aid of numbers which represent three differences. These differences are coded with a bit pattern which is based on the number series above. Finally, the bit pattern can be coded graphically with the aid of the symbols in Fig. 2.
In many cases, it will not be possible to obtain a complete number which codes the x coordinate, but parts of two numbers, when reading 4x4 symbols. Since the least significant part of the number is always 1 or 2 , however, a complete number can be simply reconstructed. The y coordinates are coded according to the same principle as used for the x coordinates . The cyclic number series is written repeatedly in horizontal rows over the surface which is to be position-coded. Exactly as in the case of the x coordinates, the rows are allowed to begin at different positions, i.e. with different sequences, in the number series. However, for the y coordinates, it is not differences which are used but the coordinates are coded with numbers which are based on the starting position of the number series in each row. Once the x coordinate for 4x4 symbols has been determined, it is possible to determine the starting positions in the number series for the rows which are included in the y code in the 4x4 symbols. In the y code, the most significant digit is determined by allowing it to be the only one which has a value in a specific range. In this example, a row of four is allowed to begin at position 0-1 in the number series to indicate that this row relates to the least significant digit in a y coordinate, and the other three begin at position 2-6. In the y direction, a number series according to the following is thus found:
(2-6) (2-6) 2-6) (0-1) (2-6) (2-6) (2-6) (0-1) (2-6)...
Each y coordinate is thus coded with three numbers between 2 and 6 and a subsequent number between 0 and 1.
Subtracting 1 from the low number and 2 from the high ones provides, in a corresponding manner as for the x direction, a position in the y direction in mixed base from which the y coordinate can be determined directly. Using the method above, 4 x 4 x 2 = 32 positions can be coded in the x direction. Each such position corresponds to three differences, providing 3 x 32 = 96 positions. Furthermore, it is possible to code 5 x 5 x 5 x 2 = 250 positions in the y direction. Each such position corresponds to 4 rows, providing 4 x 250 = 1000 positions. Altogether, it is thus possible to code 96,000 positions. Since the x coding is based on differences, however, it is possible to select the position at which the first number series begins. Taking into account that this first number series can begin at seven different positions, it is possible to code 7 x 96,000 = 672,000 positions. The starting position for the first number series in the first column can be calculated once the x coordinate has been determined. The above-mentioned seven different starting positions for the first series can code different sheets or writing surfaces on a product .
To further illustrate the invention according to this embodiment, a specific example follows which is based on the embodiment of the position code described.
Fig. 3 shows an example of an image with 4x4 symbols which are read by a device for position determination.
These 4x4 symbols have the following values: 4 4 4 2
3 2 3 4
4 4 2 4 1 3 2 4
These values represent the following binary x code and y code: x- code y- -code
0 0 0 0 0 0 0 1
1 0 1 0 0 1 0 0
0 0 0 0 0 0 1 0 1 1 0 0 1 0 1 0
The vertical x sequences code the following positions in the number series: 2 0 4 6. The differences between the columns will be -2 4 2, which modulo 7 gives: 5 4 2 which, in mixed base, codes position (5-3) x 8 + (4-3) x 2 + (2-1) = 16 + 2 + 1 = 19. Since the first coded x position is position 0, the difference which lies within the range 1-2 and which appears in the 4x4 symbols is the twentieth such difference. Since there are also a total of three columns for such difference and there is one start column, the vertical sequence farthest to the right in the 4x4 x code belongs to the 61st column in the x code (3 x 20 + 1 = 61) and the one farthest to the left belongs to the 58th.
The horizontal y series codes the positions 0 4 1 3 in the number series. Since these series begin in the 58th column, the starting position of the rows is these numbers minus 57 modulo 7, which provides the starting positions 6 3 0 2. Translated into digits in mixed base, this becomes 6-2, 3-2, 0-0, 2-2 = 4 1 0 0, the third digit being the least significant digit in the number in question. The fourth digit is then the most significant digit in the next number. In this case, it must be the same as in the number in question. (The exception is when the number in question consists of the highest possible digits in all positions. It is then evident that the beginning of the next number is one greater than the beginning of the number in question.)
In mixed base, the position of the four-digit number will be 0x50 + 4x10 + 1x2 + 0x1 = 42.
The third row in the y code is thus the 43rd which has the starting position 0 or 1, and since there are a total of four rows on each such row, the third row is number 43x4=172.
Thus, the position of the topmost left corner for the 4x4 symbol group is (58,170) in this example. Since the x sequences in the 4x4 group begin on row 170, the x columns of the entire pattern begin at positions ( (2 0 4 6) -169) modulo 7 = 1 6 3 5 in the number series. Between the last starting positions (5) and the first starting positions, numbers 0-19 are coded in mixed base, and by adding up the representations of the numbers 0-19 in mixed base, the total difference between these columns is obtained. A primitive algorithm for doing this is to generate these twenty numbers and directly add up their digits. The sum obtained is called s. The sheet or writing surface is then given by (5-s) modulo 7.
In the example above, an embodiment has been described where each position is coded with 4 x 4 symbols and a number series with 7 bits is used. Naturally, this is only one example. Positions can be coded with more or fewer symbols. The number of symbols need not be the same in both directions. The number series can be of different length and does not need to be binary but may be based on another base. Different number series can be used for coding in the x direction and coding in the y direction. The symbols can have different numbers of values.
In the example above, furthermore, the marking is a dot. Naturally, it may have a different appearance. For example, it may consist of a line which begins at the virtual raster point and extends therefrom to a predetermined position. In the example above, the symbols are used within a square partial surface for coding a position. The partial surface can have a different shape, for example hexagonal. The symbols do not need to be arranged in rows and columns at an angle of 90 degrees to each other either, but can also be arranged in some other manner.
For the position codes to be detected, the virtual raster must be determined. This can be done by studying the distance between different markings. The shortest distance between two markings must originate from two adjoining symbols with the value 1 and 3 so that the markings are lying on the same raster line between two raster points. When such a pair of markings has been detected, the associated raster points can be determined with knowledge of the distance between the raster points and the displacement of the markings from the raster points. Once two raster points have been located, additional raster points can be determined with the aid of measured distances to other markings and with knowledge of the relative distance of the raster points. An embodiment of a device for position determination, whose spatial relationship to a surface can be determined, is shown schematically in Fig. 4. It comprises a casing 11, which is approximately shaped like a pen. In the short end of the casing, there is an opening 12. The short end is provided to bear against or to be held at a short distance from a surface S on which the position determination is to occur. In the figure, a nor- mal direction v- to the surface S and an axis A extending through the device are indicated. In use of the reading device, axis A can form an angle of inclination θ with the normal direction v.. The casing mainly contains an optical part, an electronic part and a power supply.
The optical part comprises at least one light-emitting diode 13 for illuminating the surface which is to be imaged, and a light-sensitive area sensor 14, for example a CCD or CMOS sensor, for registering a two-dimensional image. The device may also comprise a lens system (not shown) .
The power supply for the device is obtained from a battery 15 which is mounted in a separate compartment in the casing.
The electronic part contains image-processing means 16 for determining a position on the basis of the image registered by means of the sensor 14 and more specifically a processor unit with a processor which is programmed to read images from the sensor and to perform position determination on the basis of these images. Moreover, the processor unit may also store an identifier which is unique for each reading device. This device identifier is usually composed of a plurality of digits and/or characters, and can be used in coded form. The identifier is of use when performing the method according to the invention where a user is identified, by the device identifier, as a particular editor of a document.
In this embodiment also, the device comprises a pen point 17, with the aid of which it is possible to write normal pigment-based writing on the surface on which the position determination is to occur. The pen point 17 can be retracted and extended so that the user can control whether or not it is to be used. Alternatively, the pen point 17 may be non-retractable and covered by a cap (not shown) when not in use. In certain applications, the device does not need to have a pen point at all.
The device also comprises buttons 18, with the aid of which the device is activated and controlled. It also has a transceiver 19 for wireless transmission, e.g. by means of IR light or radio waves, of information to and from the device. The device can also comprise a display 20 for showing positions or registered information.
Applicant's international patent publication WO 98/20446 describes a device for registering text. This device can be used for position determination if it is programmed in a suitable manner. If it is to be used for pigment-based writing, it must also have a pen point.
The device can be divided into different physical casings, a first casing containing components which are necessary for obtaining images of the position-coding pattern and for transferring these to components which are located in a second casing and which carry out the position determination on the basis of the image or images registered. As mentioned, the position determination is done by a processor which thus must have software for locating and decoding the symbols in an image and for determining positions from the codes thus obtained. A person skilled in the art can design software on the basis of the above example, which carries out position determination on the basis of an image of a part of a position-coding pattern. Furthermore, the person skilled in the art can design software on the basis of the above description, for printing the position-coding pattern.
In the embodiment above, the raster is a rectangular grid. It can also have other forms, i.e. non-rectangular.
In the embodiment above, it is not the longest possible cyclic number series which is used. This provides a certain redundancy which can be used, for example, for checking the turning of the read group of symbols. In the embodiment above, the pattern is optically readable and the sensor thus is an optical sensor. It should be emphasized that other position-coding patterns could be used within the scope of the present invention, for example the pattern described in applicant's international patent publication WO 00/01670, or the prior art position-coding patterns as disclosed in US-A-5 051 736, WO 00/31682 and EP-A-0 469 864. Further, the pattern could be based on another parameter than an optical parameter. Naturally, in that case, the sensor must be of a type which can read the parameter in question. Examples of such parameters are chemical, acoustic or electromagnetic symbols. Also capacitive or inductive symbols can be used.
Fig. 5a shows a printout 501 of a text document which is preferably stored in a computer. The printout 501 is suitably done on a sheet of paper, the surface of which is provided with a position-coding pattern as described above in connection with Figs 1-3. For the sake of clarity, however, no such pattern is shown in Fig. 5a. In one embodiment of the invention, the digital pen of Fig. 4 is used for providing editing information on the printout 501. Such editing information is schematically exemplified m Fig. 5a at reference numerals 502, 503 and 504. A misspelt word "two", the wrong letter "w" of which has been provided with a tilted line and has been marked m the right-hand margin that it is to be replaced by the letter "o" 502. An incorrect word "green" has been stricken through with a horizontal line and has been marked m the right-hand margin that it is to De replaced by the word "gold' 503. The words 'fade" and "not" have been marked 504 with an editing symbol which, on interpretation, is to indicate that the words are to change places with one another.
Fig. 5b very schematically shows a drawing consisting of simple geometric figures. A cross marking 505 has been drawn on one side of a rectangle m order to indicate that the side is to be erased.
A method for performing editing according to the invention, and with reference to the example of figure 5a, is illustrated m figure 5c. In a read and search step 510, the text of the document is analyzed m that each letter is associated with its position and at the same time a search for editing symbols with corresponding positions is performed. Then, m a number of decision steps 511,513,515,517 tne editing symbols are identified and corresponding interpretation steps 512,514,516,518 are performed.
In the read and search step 510, the identity of the digital pen is also determined m that a transfer of a unique device identifier is performed from the digital pen to the means for storing the document, typically a memory unit m a computer or server. A particular digital pen may be associated with a particular user, i.e. editor or reviewer of the document. If several editors, each with a digital pen having a unique device identity, make corrections to the text, these corrections may then be associated with the different users. If the editing symbol of step 511 is found, a corresponding editing symbol is searched for in the margin area of the printed document as well as any handwritten text close to that editing symbol. Such text is interpreted using ICR-technique and the interpreted text is then inserted at the appropriate location in the stored document.
If the editing symbol of step 513 is found, the text in the stored document which is located at the same position as the editing symbol is removed from the stored document.
If the editing symbol of step 515 is found, a corresponding editing symbol is searched for in the margin area of the printed document as well as any handwritten text close to that editing symbol. Such text is interpreted using ICR-technique and the interpreted text is then inserted at the appropriate location in the stored document, replacing the letter at the position of the editing symbol in the stored text.
If the editing symbol of step 517 is found, one or more letters at the position of the left part, i.e. within the left "loop", of the editing symbol is identified. Then one or more letters at the position of the right part of the editing symbol is identified, whereupon the identified text from the left part is moved to the identified position at the right part of the editing symbol, and vice versa. From the above it is understood that each editing symbol results in an editing command being generated, either in the pen itself or in a computer unit that communicates with the pen and comprises software for identifying editing symbols.
As discussed above, it is in many cases convenient to print the document information onto a pre-printed sheet on which position-coding pattern already has been provided. However, it may be desirable to have an extra degree of freedom in that the document information is printed together with the position-coding pattern. Such an aspect of the present invention is useful in situations where a user has the capability to calculate a position-coding pattern and apply it to the sheet to be printed. Moreover, as discussed briefly above, the present invention allows for a plurality of editors to edit one and the same document. By associating editing commands with timestamps and users, e.g. by way of unique identity codes associated with the digital pens, a sequence of editing commands can be interpreted by the computer which holds the electronic version of the document .
It is to be understood that the editing symbols and markings shown in Figs 5a, 5b and 5c only should be seen as examples of how editing information can be configured.
The editing symbols can be a subset of a larger set of symbols comprising more or less complicated indications of how the subsequent interpretation should be carried out and how the real changes of the stored document are to appear. The corresponding set of editing commands can be determined in advance or be generated by, for example, a user through a suitable learning process according to the prior art technique.
Fig. 6 shows a system that can be used m carrying out the method described above. The system includes a plurality of pen-shaped reading devices 601, a central unit 602, a local processing unit 603, and a printer unit 604. The reading devices 601 correspond to the digital pen shown m Fig. 4 and are arranged for wireless communication with the central unit 602, for example directly via a telecommunications network, or indirectly via short-range communication to a modem unit (not shown), such as a mobile phone, a personal computer, a Bluetooth® node etc, which turn connects to the central unit 602 via a suitable network, for example the Internet, a telecommunications network, a LAN etc. Likewise, the central unit 602 is arranged for communication with the local processing unit 603, typically a computer device such as a personal computer, m which the electronic document to be annotated is created. As will be further described below, the central unit acts, inter alia , as a pattern administration unit. The communication between the local processing unit 603 and the central unit 602 can be handled by a messaging service, such as e-mail, if direct communication is unavailable. Further, the local processing unit 603 is arranged to communicate with a printer unit 604.
After having created a document the local processing unit 603, the author registers the document with the central unit 602. In this process, the central unit 602 receives from the local processing unit 603 a file identifier and data on the number of standard pages of the document to be registered. Alternatively, the central unit 602 receives an electronic copy of the document. In the central unit 602, the pages of the electronic document are assigned a corresponding number of pattern pages, i.e. mutually unique subsets of the above- described position-coding pattern. After registration, a the original document, or a copy thereof, is "locked" to prevent further manipulation and is stored on the local processing unit 603 or the central unit 602.
Then, the author distributes the thus-locked document to different reviewers . The author can send the electronic document from the local processing unit 603 to the printer unit 604 and then physically distribute the printouts, or distribute the document electronically so that the reviewers can print the document on any nearby printer units (not shown) . Alternatively, the different reviewers might themselves download the document from the central unit to a respective computer, from which the document can be sent to a suitable printer unit.
Thereafter, each reviewer uses his reading device 601 to write on his printout 605, wherein the reading device 601 electronically registers the positions where it is applied to the printout, thereby producing a digital copy of editing information written on the pages of the printout. When a reviewer is finished with his review, he synchronizes his reading device 601 with the central unit 602. In this process, the digital editing information, i.e. position data read by the reading device 601, is sent to the central unit 602 together with a reading device identifier, and optionally timestamps indicating when the annotations were made. The synchronization can be effected, for example, by the reviewer checking a send icon (not shown) on the printout 605, the send icon being provided with a specific part of the position-coding pattern. When the processor unit of the reading device
601 registers one or more positions coded by this specific part of the pattern, it initiates the synchronization. Alternatively, the reviewer can operate a button on the reading device 601 to initiate the synchronization. As a further alternative, the digital editing information can be sent to the central unit 602 in real time, i.e. while the reviewer is actually writing the editing information on the printout 605. Upon completed synchronization with one or more reading devices 601, the central unit 602 issues a message to the local processing device 603, whereupon the author can connect to the central unit 602 to download the feedback from the reviewers, one or more at a time. The central unit 602 also translates the editing information to local positions on each page of the electronic document, so that the local processing unit 603 can incorporate the editing information into the electronic document to be displayed to the author. In the case where a copy of the electronic document has been sent to the central unit 602, the incorporation can be effected by the central unit 602 itself which then transmits an electronic file back to the local processing unit 603 for display to the author . In an alternative embodiment (not shown) , the pattern administration functionality of the central unit is incorporated into the local processing unit, so that one and more reading devices communicate with the local processing unit, typically a personal computer, a PDA or the like.
In another alternative embodiment (not shown) , the central unit only serves the function of assigning pattern pages to the electronic document, i.e. the local processing unit receives a file with the pattern pages, or parameters for calculating the same, from the central unit. The local processing unit then communicates with the digital pens and produces an updated document based on the received editing information.
In a further embodiment, the digital pen is capable of interpreting the editing symbols, so that the pen can transmit an editing command to the local processing unit. Optionally, the pen is also capable of effecting an ICR analysis of relevant parts of the editing information written on the printout.
Although the author and the reviewers have been described as separate individuals, it should be evident that in the method and system described above the author act as a reviewer as well. It is also to be understood that the central unit can communicate with a plurality of local processing units, to thereby act as a hub in an infrastructure for handling information. It should also be noted that the incorporation of the editing information might or might not include interpreting the markings made by the reviewers on the sheet 605 containing the printed text. In some reviewing work, it may be desirable to enable for one or more reviewers to simply annotate the text, in the margins for example, and to associate those annotations with specific positions within the text and to enable an owner of the document to view the text together with the annotations. Thereby, instead of being forced to edit part of the text or graphics on the printout, the reviewer is allowed to write brief comments or draw sketches in the margins of the document to aid the author in his further drafting of the document. In this case, after review, the document information of the updated electronic document includes the original text and the handwritten annotations made by the reviewers. It is to be appreciated that the reviewer can be allowed to use both annotations and editing symbols/markings in one and the same document. This can be achieved, for example, by using a specific set of annotation symbols which identifies a piece of editing information as an annotation. The local processing unit 603 suitably includes presentation software which can be arranged to present the annotations in the form of, e.g., pop-up windows or the like overlaid on the original document. By associating different users, via the device identifier, with the different annotations it is possible to include functions in the presentation software which, e.g., presents the comments of the annotators one at a time.
It should also be noted that the presentation software could be set to prompt the author for approval before inserting any interpreted text associated with an editing symbol into the electronic document.
In the context of the invention, especially when allowing for annotations, it might be of importance to anchor the editing information to specific positions within the document, more specifically to specific portions of the text, such as chapters, paragraphs, lines or single words. When the author starts editing the original document based on the editing information, the overlaid handwritten comments must be maintained in synch with their original position. This is achieved by anchoring the annotations to the text, for example via an agreed set of annotation symbols which associates a comment written in the margin of the document with a portion of the text in the document. Such a symbol may be of any kind, for example as discussed in connection with figures 5a-5c above. The different steps described above is suitably performed by software running on the units 602, 603 and devices 601 of the system. This software could be contained in the system or distributed to the system on one or more computer readable media, for example a volatile memory or a nonvolatile memory such as a floppy diskette or CD-ROM. The software could also be distributed on computer readable media in the form of propagated signals, such as the stream of bits that represent Internet transmissions of packets or the carrier waves that are transmitted to satellites.

Claims

1. A method for editing document information in a computer-stored document comprising:
- transferring the document information of the document to a printing device which is capable of printing document information on a surface which is provided with a position-coding pattern,
- receiving editing information from a reading device capable of reading position information from the position-coded surface,
- interpreting the editing information,
- changing the document information in dependence on the interpretation of the editing information, thereby producing an updated document.
2. A method for editing document information in a computer-stored document comprising:
- transferring position-coding pattern information to a printing device which is capable of printing the position-coding pattern on a surface, - transferring the document information of the document to the printing device which is capable of printing document information on the surface,
- receiving editing information from a reading device capable of reading position information from the position-coded surface,
- interpreting the editing information,
- changing the document information in dependence on the interpretation of the editing information, thereby producing an updated document.
3. A method according to claim 1 or 2 , further comprising the step of receiving device identity information from the reading device, so as to associate the editing information with a user of the reading device.
4. A method according to claim 1, 2 or 3 , wherein the editing information is associated with a plurality of users each of which having generated at least one editing command with a reading device.
5. A method according to claim 4, wherein the editing commands generated by the plurality of users are in an ordered sequence identified by at least a timestamp associated with each editing command.
6. A method according to any one of claims 1-5, wherein the editing information comprises position information related to the position of the reading device on the surface, and wherein the interpretation of the editing information comprises interpretation of the position information.
7. A method according to claim 6, wherein the position information is in the form of sequences of coordinates which form manually generated curves which have correspondence in the form of drawn curves on the printed document.
8. A method according to any one of claims 1-7, further comprising the step of displaying the document information of the updated document to a user.
9. A method according to any one of claims 1-8, wherein the step of changing the document information includes adding editing information in the form of handwritten annotations to the computer-stored document.
10. A method according to claim 9, comprising the step of associating, based on position information included in the editing information, each of said handwritten annotations with a respective portion of the document information of the computer-stored document.
11. A method according to any one of claims 1-10, wherein the step of changing the document information includes reformatting one or more parts of the document information.
12. A method according to claim 11, wherein said reformatting includes adding text or graphics to said document information, removing text or graphics from said document information, or repositioning text or graphics included in said document information.
13. A method according to claim 12, wherein the step of adding text includes converting part of the editing information to machine-readable text.
14. A method according to any one of claims 1-13, comprising the step of initially registering said computer-stored document in a pattern administration unit, wherein the pattern administration unit assigns a unique subset of said position-coding pattern to each page of said document.
15. A computer readable-medium having embodied thereon a computer program which can be read by a computer and which comprises instructions for causing a computer to execute the method according to one any of claims 1-14.
16. A system for editing document information in a document, comprising:
- storage means for storing the document,
- means for transferring the document information of the document to a printing device which is capable of printing document information on a surface which is pro- vided with a position-coding pattern,
- means for receiving editing information from a reading device capable of reading position information from the position-coded surface,
- means for interpreting the editing information,
- means for changing the document information in dependence on the interpretation of the editing information, thereby producing an updated document.
17. A system for editing document information in a document, comprising:
- storage means for storing the document, - means for transferring position-coding pattern information to a printing device which is capable of printing the position-coding pattern on a surface,
- means for transferring the document information of the document to the printing device which is capable of printing document information on the surface,
- means for receiving editing information from a reading device capable of reading position information from the position-coded surface,
- means for interpreting the editing information, - means for changing the document information in dependence on the interpretation of the editing information, thereby producing an updated document.
18. A system according to claim 16 or 17, further comprising means for receiving device identity information from the reading device, so as to associate the editing information with a user of the reading device .
19. A system according to any one of claims 16-18, wherein said storage means is included in a computer device which is arranged to initially register said document in a pattern administration unit comprising a database of said position-coding pattern, said central unit being arranged to assign a unique subset of said position-coding pattern to each page of said document.
20. A system according to claim 19, wherein said means for receiving editing information is included in said pattern administration unit.
21. A system according to claim 19, wherein said means for receiving editing information is included in said local processing unit.
EP01918045A 2000-03-21 2001-03-21 Processing of documents Withdrawn EP1269303A1 (en)

Applications Claiming Priority (3)

Application Number Priority Date Filing Date Title
SE0000954A SE516109C2 (en) 2000-03-21 2000-03-21 Procedure, systems and computer programs for document management using position coding patterns
SE0000954 2000-03-21
PCT/SE2001/000605 WO2001071475A1 (en) 2000-03-21 2001-03-21 Processing of documents

Publications (1)

Publication Number Publication Date
EP1269303A1 true EP1269303A1 (en) 2003-01-02

Family

ID=20278908

Family Applications (1)

Application Number Title Priority Date Filing Date
EP01918045A Withdrawn EP1269303A1 (en) 2000-03-21 2001-03-21 Processing of documents

Country Status (5)

Country Link
EP (1) EP1269303A1 (en)
JP (1) JP2003528388A (en)
AU (1) AU2001244916A1 (en)
SE (1) SE516109C2 (en)
WO (1) WO2001071475A1 (en)

Families Citing this family (31)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US7916124B1 (en) 2001-06-20 2011-03-29 Leapfrog Enterprises, Inc. Interactive apparatus using print media
US7202963B2 (en) 2001-06-28 2007-04-10 Anoto Ab Method for processing information
JP4102105B2 (en) * 2002-05-24 2008-06-18 株式会社日立製作所 Document entry system using electronic pen
US7133563B2 (en) * 2002-10-31 2006-11-07 Microsoft Corporation Passive embedded interaction code
JP2004246433A (en) * 2003-02-12 2004-09-02 Hitachi Ltd Data input system
US7753283B2 (en) * 2003-06-13 2010-07-13 Anoto Aktiebolag (Anoto Ab) On-demand printing of coding patterns
US7826101B2 (en) 2003-06-25 2010-11-02 Ricoh Company, Ltd. Document management method, document management program, recording medium, and document management apparatus
GB0321167D0 (en) * 2003-09-10 2003-10-08 Hewlett Packard Development Co Printing digital documents
SE0303058D0 (en) 2003-11-18 2003-11-18 Anoto Ab Methods and arrangements in an information management system
JP4385744B2 (en) * 2003-11-28 2009-12-16 株式会社日立製作所 Electronic pen usage document entry system with falsification prevention function
US7453447B2 (en) 2004-03-17 2008-11-18 Leapfrog Enterprises, Inc. Interactive apparatus with recording and playback capability usable with encoded writing medium
US7831933B2 (en) 2004-03-17 2010-11-09 Leapfrog Enterprises, Inc. Method and system for implementing a user interface for a device employing written graphical elements
US7853193B2 (en) 2004-03-17 2010-12-14 Leapfrog Enterprises, Inc. Method and device for audibly instructing a user to interact with a function
US20060067576A1 (en) 2004-03-17 2006-03-30 James Marggraff Providing a user interface having interactive elements on a writable surface
US8054495B2 (en) 2004-04-07 2011-11-08 Hewlett-Packard Development Company, L.P. Digital documents, apparatus, methods and software relating to associating an identity of paper printed with digital pattern with equivalent digital documents
US7698640B2 (en) 2004-08-04 2010-04-13 Leapfrog Enterprises, Inc. User interactive journal
US7428990B1 (en) 2004-12-22 2008-09-30 Leapfrog Enterprises, Inc. Capacitive sensing of media information in an interactive media device
US7621441B1 (en) 2004-12-22 2009-11-24 Leapfrog Enterprises Interactive device using capacitive sensor array for joint page identification and page location determination
US7661592B1 (en) 2005-06-08 2010-02-16 Leapfrog Enterprises, Inc. Interactive system including interactive apparatus and game
JP4539479B2 (en) 2005-07-27 2010-09-08 富士ゼロックス株式会社 Medium management system, image forming apparatus, medium management method, and program
US7374087B1 (en) 2005-07-29 2008-05-20 Leapfrog Enterprises, Inc. Method, apparatus and system for conveying cartridge notification
US7549596B1 (en) 2005-07-29 2009-06-23 Nvidia Corporation Image bearing surface
US7922099B1 (en) 2005-07-29 2011-04-12 Leapfrog Enterprises, Inc. System and method for associating content with an image bearing surface
JP4497052B2 (en) 2005-08-12 2010-07-07 富士ゼロックス株式会社 Image processing apparatus and program
US7281664B1 (en) 2005-10-05 2007-10-16 Leapfrog Enterprises, Inc. Method and system for hierarchical management of a plurality of regions of an encoded surface used by a pen computer
US7936339B2 (en) 2005-11-01 2011-05-03 Leapfrog Enterprises, Inc. Method and system for invoking computer functionality by interaction with dynamically generated interface regions of a writing surface
US7562822B1 (en) 2005-12-30 2009-07-21 Leapfrog Enterprises, Inc. Methods and devices for creating and processing content
JP2008021120A (en) * 2006-07-13 2008-01-31 Fuji Xerox Co Ltd Writing information processing system, writing information processing method, and program
US7847942B1 (en) 2006-12-28 2010-12-07 Leapfrog Enterprises, Inc. Peripheral interface device for color recognition
JP4301524B2 (en) * 2007-03-27 2009-07-22 株式会社沖データ Printing system and information processing apparatus
JP2016091464A (en) * 2014-11-10 2016-05-23 大日本印刷株式会社 Information processing unit, information processing system and program

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5661506A (en) 1994-11-10 1997-08-26 Sia Technology Corporation Pen and paper information recording system using an imaging pen
US5852434A (en) 1992-04-03 1998-12-22 Sekendur; Oral F. Absolute optical position determination

Family Cites Families (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US4494682A (en) * 1982-07-07 1985-01-22 Hunt-Wesson Foods, Inc. Pouring fitment with container and closure therefor
JPS5991585A (en) * 1982-11-16 1984-05-26 Nippon Telegr & Teleph Corp <Ntt> Inputting method of character
US4806976A (en) * 1986-07-25 1989-02-21 Minolta Camera Kabushiki Kaisha Copy apparatus with pattern writing capability
US4887128A (en) * 1988-01-04 1989-12-12 Eastman Kodak Company Method and apparatus for reproducing documents with variable information
US4969013A (en) * 1989-06-13 1990-11-06 Eastman Kodak Company Apparatus and method for digitizing a document for selective area treatment
JP3118640B2 (en) * 1989-10-19 2000-12-18 キヤノン株式会社 Processing device and display control method
GB2280329A (en) * 1993-06-29 1995-01-25 Primax Electronics Company Lim Pen type input image scanner connected to printer port of a computer
US5737740A (en) * 1994-06-27 1998-04-07 Numonics Apparatus and method for processing electronic documents
US5652412A (en) * 1994-07-11 1997-07-29 Sia Technology Corp. Pen and paper information recording system
JPH0836452A (en) * 1994-07-21 1996-02-06 Oki Electric Ind Co Ltd Writing pen and writing pen device

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5852434A (en) 1992-04-03 1998-12-22 Sekendur; Oral F. Absolute optical position determination
US5661506A (en) 1994-11-10 1997-08-26 Sia Technology Corporation Pen and paper information recording system using an imaging pen

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
DYMETMAN M; COPPERMAN M: "Intelligent paper", LECTURE NOTES IN COMPUTER SCIENCE, vol. 1375, March 1998 (1998-03-01), pages 392 - 406, XP002328425
See also references of WO0171475A1

Also Published As

Publication number Publication date
WO2001071475A1 (en) 2001-09-27
SE516109C2 (en) 2001-11-19
JP2003528388A (en) 2003-09-24
AU2001244916A1 (en) 2001-10-03
SE0000954L (en) 2001-09-22
SE0000954D0 (en) 2000-03-21

Similar Documents

Publication Publication Date Title
EP1269303A1 (en) Processing of documents
US8418052B2 (en) Processing of documents
EP2056233B1 (en) Information management system
US6722574B2 (en) Business card
US9304605B2 (en) General information management system
KR100735795B1 (en) Method and system for graphic design
US6689966B2 (en) System and method for determining positional information
US7422154B2 (en) Apparatus and methods relating to image coding
US7710408B2 (en) Centralized information management based upon position information
US6586688B2 (en) Information-related devices and methods
EP1579379B1 (en) Information code including redundant information providing copy protection
US20010024193A1 (en) Written command
MXPA02006340A (en) General information management system.
WO2000073887A1 (en) Calendar
EP1913526A1 (en) Method and system for combining a position and information code
EP2474888A1 (en) Information input/output apparatus, information processing apparatus, information input/output system, printing medium, and information input/output method
KR100905705B1 (en) Method for Providing Electronic Note Service by Using Digital Paper and Pen, System and Computer-Readable Recording Medium with Program Therefor
JP5084087B2 (en) Handwriting command
JP2005173672A (en) Form, electronic pen, and program
EP1382006B1 (en) Business card with a position-coding pattern
WO2001048592A1 (en) Charge card purchase
EP1269396A1 (en) Apparatus and methods relating to images
JP5169369B2 (en) Handwriting information processing apparatus and program

Legal Events

Date Code Title Description
PUAI Public reference made under article 153(3) epc to a published international application that has entered the european phase

Free format text: ORIGINAL CODE: 0009012

17P Request for examination filed

Effective date: 20021021

AK Designated contracting states

Kind code of ref document: A1

Designated state(s): AT BE CH CY DE DK ES FI FR GB GR IE IT LI LU MC NL PT SE TR

AX Request for extension of the european patent

Free format text: AL PAYMENT 20021021;LT PAYMENT 20021021;LV PAYMENT 20021021;MK PAYMENT 20021021;RO PAYMENT 20021021;SI PAYMENT 20021021

RIN1 Information on inventor provided before grant (corrected)

Inventor name: LUNDSTROEM, TOMAS

Inventor name: ERICSON, PETTER

Inventor name: AKERSTEN, CHRISTOFER

RAP1 Party data changed (applicant data changed or rights of an application transferred)

Owner name: ANOTO IP LIC HB

111L Licence recorded

Free format text: 0100 LEAPFROG ENTERPRISES INC.

Effective date: 20050530

RAP1 Party data changed (applicant data changed or rights of an application transferred)

Owner name: ANOTO AB

TPAC Observations filed by third parties

Free format text: ORIGINAL CODE: EPIDOSNTIPA

17Q First examination report despatched

Effective date: 20071129

RAP1 Party data changed (applicant data changed or rights of an application transferred)

Owner name: ANOTO AB

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: THE APPLICATION IS DEEMED TO BE WITHDRAWN

18D Application deemed to be withdrawn

Effective date: 20101104