SE516109C2 - Procedure, systems and computer programs for document management using position coding patterns - Google Patents

Abstract

An electronically stored document which can be of any type such as text, image, drawing etc. is printed on a surface, preferably a sheet of paper, which is provided with a position-coding pattern. Manual editing is then carried out on the printout surface with a digital pen which comprises means for reading the position-coding pattern and also a pen point for marking on the surface. Editing is done by means of a code which is in the form of symbols from a predetermined set of symbols, on the sheet of paper. The editing information, i.e. the symbols applied to the surface, is digitally registered by the digital pen and transferred to the storage and processing device, preferably a computer. This transfer can be done directly during the editing or on a later occasion. Interpretation of the editing code is then carried out, wholly or partly in the computer, in interaction with the document stored in the computer, whereupon changes are made to the stored document in direct dependence on the interpretation.

Description

20 25 30 35 516 »109 2 In its most general form, the invention is characterized in that, on the basis of an electronically stored document which can be of any type such as text, image, drawing, etc. The document is printed on a surface, preferably paper , which is provided with a position coding pattern.

Manual editing is then performed on the print surface with a device that includes means for reading the position coding pattern as well as a drawing pen for marking on the surface. The editing is done with a code, which is in the form of symbols from a predetermined set of symbols, on the paper. The editing information, ie the symbols applied to the surface, is transferred to the storage and processing device, preferably a computer. This transfer can take place directly during editing or at a later time. Interpretation of the editing code is then performed, in whole or in part in the computer, in interaction with the document stored in the computer, after which changes are made to the stored document in direct dependence on the interpretation.

A number of advantages of the invention, which are associated with simplicity of handling, are obvious: it is simple and easy to understand, ie it is a matter of intuitive handling of documents of the traditional type, ie paper prints.

Thus, there is also a low learning threshold for the people who will perform the editing.

Furthermore, it is an advantage that it is easy to edit documents without the editor having to be at a computer where the document is stored, or be bound to a complicated input device which in prior art is exemplified by a digitizing table. The editing information can thus advantageously be stored in the loading device for a later transfer to the computer / storage location.

In addition, as the invention provides, it is advantageous to directly obtain a copy of the edit in the form of the manually edited printout. BRIEF DESCRIPTION OF THE DRAWINGS Figure 1 schematically shows an embodiment of a product i which is provided with a position coding pattern. Figures 2a-2d schematically show how the symbols can be formed in an embodiment of the invention.

Figure 3 schematically shows an example of 4x4 symbols used to encode a position.

Figure 4 schematically shows a device in accordance with the present invention which can be used for position determination in three dimensions.

Figure Sa shows a printout of a text document with manually drawn editing instructions_ Figure Sb shows a printout of a drawing document with a manually drawn editing instruction.

PREFERRED EMBODIMENTS For the sake of clarity, the following detailed description of the invention has been divided into a number of sub-descriptions. Initially, with reference to Figures 1, 2a-d and 3, a coding pattern will be presented. This coding pattern represents position information which can be used in a method according to the invention. After the presentation of the coding pattern, a device which is intended for use in manual editing of a printed document is then presented in connection with Figure 4. The device, which is pen-shaped, reads the position coding pattern, and if applicable also text, and is provided with a pen tip to make editing markings visible on the printed document. Then, with reference to Figures 5a and 5b, it is shown how examples of manual editing information are drawn on a printout of a text document (Figure 5a) and a document (Figure 5b) containing a drawing figure.

Figure 1 shows a part of a product in the form of a paper 1, which on its surface 2 is provided with an optically readable position coding pattern 3 which enables positional coding pattern 3 which enables positional determination. The position coding pattern consists of symbols 4, which are systematically arranged over the surface 2, so appearance. that this has a “patterned” The paper has an x-coordinate axis and a y-coordinate axis. In this case, position determination can be performed on the entire surface of the product. In other cases, the surface that allows positioning may be a smaller part of the product. The paper can, for example, be used to provide an electronic representation of information written or drawn on the surface. The electronic representation can be achieved by continuously during writing on the surface with a pen, determining the position of the pen on the paper by reading the position coding pattern.

The position coding pattern comprises a virtual grid, which is thus neither visible to the human eye nor can be detected directly by a device which is to determine a plurality of symbols 4, and one can assume one of four values "1" - "4" positions on the surface, and one which was as described below. In this context, it should be pointed out that the position coding pattern in Figure 1 is greatly enlarged for the sake of clarity. In addition, it appears on only part of the paper.

The position coding pattern is arranged so that the position of a partial surface on the writing surface is coded by the symbols on this partial surface. A first and a second sub-surface 5a, 5b are shown in broken lines in Figure 1. The part of the position coding pattern (here 3 x 3 symbols) present on the first sub-surface 5a encodes a first position, and the part of the position coding pattern which located on the second sub-surface 5b encodes a second position. The position coding pattern is thus partly common to the adjacent first and second positions. Such a position coding pattern is referred to in this application as "floating".

Figures 2a-d show an embodiment of a symbol which can be used in the position coding pattern according to the invention. 10 15 '20 25 516. 109 u ø n. oo 5 The symbol comprises a virtual grid point 6, which is represented by the point of intersection between the grid lines, and a marking 7 which has the shape of a dot. The value of the symbol depends on where the marking is located. In the example in figure 2, there are four possible locations, one on each of the raster lines that start from the raster points. The offset from the raster point is the same for all values. In the following, the symbol in Figure 2a has the value 1, in Figure 2b the value 2, in Figure 2c the value 3 and in Figure 2d the value 4. In other words, there are four different types of symbols.

Each symbol can thus represent four values "1-4".

This means that the position coding pattern can be divided into a first position code for the x-coordinate, and a second position code for the y-coordinate. The division is made as follows: Symbol value x-code y-code 1 1 l 2 0 1 3 1 0 4 0 O The value of each symbol is thus translated into a first digit, here bit, for the x-code and a second digit, here bit, for y code. In this way you get two completely independent bit patterns. The patterns can be combined into a common pattern, which is coded graphically by means of a plurality of symbols according to Figure 2.

Each position is coded using a plurality of symbols. In this example, 4x4 symbols are used to encode a position in two dimensions, ie an x-coordinate and a y-coordinate. 10 15 20 516m 109 n | ø. n ø o u oo o o ø o | g. n u o u av av. n 6 The position code is constructed using a series of numbers of ones and zeros, which has the property that no sequence of four bits occurs more than once in the series. The speech series is cyclical, which means that the property also_covers when you connect the end of the series with its beginning. A four-bit sequence thus always has a uniquely determined position in the speech series.

The series can be a maximum of 16 bits long if it is to have the above-described property of four-bit sequences. In this example, however, only a seven bit long series is used as follows: "0 O 0 l 0 1 0".

This series contains seven unique four-bit sequences that encode a position in the series as follows: Position in the series Sequence 0 0001 l 0010 2 0101 3 1010 4 0100 S 1000 6_ 0000 To encode the x-coordinates, write the number series sequentially in columns above the entire area to be coded. The coding is based on the difference or position shift between numbers in adjacent columns. The size of the difference is determined by in which position (ie with which sequence) in the number series the column is allowed to begin.

More specifically, if the difference modulo seven between on one side takes a number, which is coded by a four-bit sequence in 10 15 20 25 30 35 I 516 1 5 .: âuš. regret. ..._ u ... 7 a first column and which may thus have the value (position) O-6, and on the other hand the corresponding number (ie the sequence at the same "height") in an adjacent column, the result will be the same regardless of where along the two columns make the comparison. With the help of the difference between two columns, you can code an x-coordinate that is constant for all y-coordinates.

Since each position on the surface is coded with 4x4 symbols in this example, you have access to three differences (with the value 0-6) as above to code the x-coordinates. The coding is then done in such a way that of the three differences, one will always have the value 1 or 2 and the other two will have values in the interval 3-6. No differences must therefore be zero in the x-code. In other words, the x-code is constructed so that the differences are as follows: (3-6) (3-6) (1-2) (3-6) (3-6) (1-2) (3-6) ( 3-6) (1-2) ...

Each x-coordinate is thus coded with two numbers between 3 and 6 and a subsequent number which is 1 or 2. If you subtract three from the high numbers and one from the low you get a number in mixed base, which directly gives a position in the x-direction, from which the x-coordinate can then be determined directly, as shown in the example below.

Using the principle described above, you can thus code x-coordinates 0, 1, 2 ..., with the help of numbers that represent three differences. These differences are coded with a bit pattern based on the speech series above. The bit pattern can finally be coded graphically using the symbols in Figure 2.

In many cases, when loading 4x4 symbols, you will not get a complete number that encodes the x-coordinates, but parts of two numbers. However, since the least significant part of the numbers is always 1 or 2, one can easily reconstruct a complete number.

The y-coordinates are coded according to the same principle used for the x-coordinates. The cyclic number series is written up- v vv- 10 15 20 25 30 35 .516 109 n u ~. p n ø u n »u o | u. . o ø ø n o o n -; u u n a o u o v. 8 repeated times in horizontal rows over the surface to be position coded. Just as for the x-coordinates, you let the lines start in different positions, ie with different sequences, in the number series. For the y-coordinates, however, no differences are used, but the coordinates are coded with numbers based on the starting position of the number series on each row. Once you have determined the x-coordinate for 4x4 symbols, you can namely determine the starting positions in the number series for the lines that are included in the y-code in the 4x4 symbols. In the y-code, the most significant number is determined by letting it be the only one that has a value in a particular range. In this example, a row of four is started in position 0-1 in the number series, to indicate that this row refers to the least significant number in a y-coordinate, and the other three start in position 2-6. In the y-direction there is thus a series of numbers as follows: (2-6) (2-6) (2-6) (O-1) (2-6) (2-6) (2-6) (O -1) (2-6) ...

Each y-coordinate is thus coded with three numbers between 2 and 6 and a subsequent number between 0 and 1.

If you subtract 1 from the low number and 2 from the high ones, you get in the same way as for the x-direction a position in the y-direction in a mixed base from which you can directly determine the y-coordinate.

With the method above you can code 4 x 4 x 2 = 32 positions in x-direction. Each such position corresponds to three differences, giving 3 x 32 = 96 positions. Furthermore, you can code 5 x 5 x 5 x 2 = 250 positions in y-direction. Each such position corresponds to 4 rows, giving 4 x 250 1000 positions.

In total, 96000 positions can be coded. However, since the x-coding is based on differences, you can choose in which position the first series of numbers begins. If you take into account that this first series of numbers can start in seven different positions, you can code 7 x 96000 = 672000 positions. The starting position of the first series of numbers in the first column can be calculated when the x-coordinate has been determined. The above seven different starting positions for the first series may encode different sheets or writing surfaces on a product.

To further illustrate the invention according to this embodiment, here follows a specific example which is based on the described embodiment of the position code.

Figure 3 shows an example of an image with 4x4 symbols read by a position determining device.

These 4x4 symbols have the following values: These values represent the following binary x- and y-code: x-code: y-code: 0 0 0 0 0 0 0 1 0 1 0 0 0 0 0 The vertical x-sequences encode the following positions in 2 O 4 6. The differences between the columns become -2 4 2, which modulo 7 gives: 5 4 2, which in mixed base encodes position (5-3) x 8 + (4-3 ) x 2 + (2-1) = 16 f 2 + 1 = 19. Since the first coded x-position is position number series: 0, the difference which lies in the interval l-2 and which is seen in the 4x4 symbols is the twentieth such difference.

Furthermore, since there are a total of three columns on each, the vertical sequence at the far right of the 4x4-x code belongs to that difference and there is a start column, 10 15 20 25 30 35 ø n; | oo o o n n .. u ø o | : v: annu. 516 109 c u n ø u. 10 6th column in the x-code (3 X 20 + 1 = 61) and the far left the 58th.

The horizontal y-sequences encode the positions 0 4 1 3 in the speech series. Since these series start in the 58th column, the starting position of the rows is these numbers minus 57 modulo7, which gives the starting positions 6 3 O 2. Translated to numbers in the mixed base, this becomes 6-2, 3 ~ 2, 0O, least significant the number in the current number. The 2-2 = 4 1 O O, where the third digit is the fourth digit is then the most significant digit in the next number. In this case, it must be the same as in the current number. (The exception case is when the current number consists of the highest possible numbers in all positions.

Then you know that the beginning of the next number is greater than the beginning of the current number.) The position for the four-digit number will be in the mixed base Ox5O + 4x10 + lx2 + Oxl = 42.

The third line in the y-code is thus the 43rd which has a starting position 0 or 1, and since there are four lines in total on each such line, 43X4 = 172. is the third row number In this example, the position of the upper left corner of the 4x4 symbol group is (58, l70).

Since the x-sequences in the 4x4 group start on row 170, the whole X pattern of the pattern starts in the positions of the number series ((2 0 4 6) -169) mode 7 = 1 6 3 5. Between the last start position (5) and the first starting position encodes the numbers 0-19 in the mixed base, and by summing the representations of the numbers O-19 in the mixed base, you get the total difference between these columns. A naive algorithm for doing this is to generate these twenty numbers and directly sum up their numbers. The sum obtained is cold s. The sheet or writing surface is then given by (5-s) modulo7.

In the example above, an embodiment has been described in which each position is coded with 4 x 4 symbols and a series of numbers with 7 10 15 20 25 30 35 1 Q 4 o o n ll bits are used. Of course, this is just one example.

Positions can be coded with more or fewer symbols. There does not have to be as many in both drawings. The number series can have a different length and does not have to be binary, but can be based on a different base. Different speech series can be used for coding in the x-direction and coding in the y-direction.

The symbols may have different numbers of values.

In the example above, the selection is also a point. Of course, it can have a different look. It can, for example, consist of a line that starts at the virtual grid point and extends from it to a specific position.

In the example above, the symbols within a square sub-area are used to encode a position. The sub-surface can have another shape, for example hexagonal. The symbols also do not have to be arranged in rows and columns at a 90 degree angle to each other but can also be arranged in other arrangements.

In order for the position code to be detected, the virtual grid needs to be determined. This can be done by studying the distance between different markings. The shortest distance between two markings must be derived from two adjacent symbols with the value 1 and 3 so that the markings are on the same grid line between two grid points. When such a pair of markings has been detected, the associated grid points can be determined with knowledge of the distance between the grid points and the displacement of the markings from the grid points. Once two raster points have been located, additional raster points can be determined by means of measured distances to other markings and with knowledge of the mutual distances of the raster points.

An embodiment of a position determining device whose spatial relationship to a surface can be determined is shown schematically in Figure 4. It comprises a housing 11, which is shaped approximately like a pencil. In the short end of the housing un-nu 10 15 20 25 30 n n ø. ao | o ø Q .. u c o u o: u u n a en .516 109 12 there is an opening 12. The short end is intended to abut or be kept at a small distance from a surface S on which the position determination is to take place. The figure indicates a normal direction V Z axis A. The axis A forms an angle of inclination 6 with to the surface S and a normal direction V through the device.

Z The housing essentially houses an optical part, an electronics part and a power supply.

The optical part comprises at least one LED 13 for illuminating the surface to be imaged and a light-sensitive area sensor 14, for example a CCD or CMOS sensor, for detecting a two-dimensional image. Optionally, the device may also contain a lens system.

The power supply to the device is obtained from a battery 15 which is mounted in a separate compartment in the housing.

The electronics part contains image processing means 16 for determining a position on the basis of the image registered with the sensor 14 and more particularly a processor unit with a processor which is programmed to read images from the sensor and perform position determination on the basis of these images.

The device also comprises in this embodiment a pen tip 17, with the aid of which one can write ordinary dye-based writing on the surface on which the position determination is to take place. The pen tip 17 can be folded in and out so that the user can control whether it is to be used or not.

In some applications, the device need not have a pen tip at all.

The device further comprises buttons 18 by means of which the device is activated and controlled. It also has a transceiver 19 for wireless transmission, for example with IR light or radio waves, of information to and from the device.

The device may further comprise a display 20 for displaying positions or registered information. 10 15 20 25 30 35 .516 109: jfjfg 13 The applicant's Swedish patent no. 9604008-4 describes a device for registering text. This device can be used for position determination if it is programmed appropriately. If it is to be used for dye-based writing, it must be further supplemented with a pen-A tip.

The device may be divided into different physical envelopes, a first envelope containing components necessary to take pictures of the position coding pattern and to transfer these to components present in a second envelope and which perform the position determination on the basis of the recorded image or images.

The position determination is made as mentioned by a processor which must therefore have software for locating and decoding the symbols in an image and for determining positions from the codes thus obtained. Those skilled in the art can, based on the example above, design software that performs position determination based on the image of a part of a position coding pattern.

Furthermore, the person skilled in the art can, on the basis of the description above, design software for printing the position coding pattern.

In the exemplary embodiment above, the pattern is optically readable and the sensor is thus optical. As mentioned, the pattern may be based on a parameter other than an optical parameter. In such a case, of course, the sensor must be of a type that can read the current parameter.

In the exemplary embodiment above, the grid is a grid. It can also have other forms.

In the embodiment above, the longest possible cyclic speech series is not used. This achieves a certain redundancy that can be used, for example, to control the rotation of the loaded group of symbols.

Figure 5a shows a printout 501 of a text document which is preferably stored in a computer. The printing 501 preferably takes place on a paper whose surface is provided with a position coding pattern as described above in connection with Figures 1-3. For the sake of clarity, however, no such pattern is shown in Figure 5a.

Edit markings are schematically exemplified in Figure 5a by the reference numerals 502,503 and 504. An misspelled word “two” whose incorrect letter “w” has been provided with a dash and in the right margin is marked 502. An incorrect word “green” has be marked with a dash and in that it shall be replaced by the letter "o" the right margin indicates that it shall be replaced by the word "gold" 503. The words "fade" and "note" have been marked 504 with a symbol which in the interpretation shall mean that the words should swap places with each other.

Figure 5b shows very schematically a drawing consisting of simple geometric figures. A cross mark 505 has been drawn on one side of a rectangle to indicate that this side is to be erased.

The symbols and markings shown in Figures 5a and 5b should only be seen as examples of how editing markings can be designed. The symbols can be a subset of a larger set of editing markings including more or less complicated indications of how the subsequent interpretation is to be performed and how. the actual change of the stored document should look like. The set of editing commands can be predetermined or generated by, for example, a user through a suitable prior art learning process. The transmission of editing information from the reading device includes transmitting position data read by the reading device. This transfer can take place at the same time as the user draws the editing symbols on the printed document, or at a later time.

Claims (5)

10 15 20 25 30 15 PATENT REQUIREMENTS A method for editing document information in a computer-stored document, comprising: - transmitting the document information of the document to a printing device capable of printing document information on a surface provided with a position coding pattern, - receiving editing information from one of reading device capable of reading position information from the position coded surface, - interpretation of the editing information, - change of the document information depending on the interpretation of the editing information_ The method of claim 1, wherein the editing information comprises position information related to the position of the reading device at the surface, and wherein the interpretation of the editing information comprises interpretation of the position information. The method of claim 2, wherein the position information is in the form of sequences of coordinates that form manually generated curves having equivalents in the form of drawn curves on the printed document. A computer program product comprising program code which, when loaded into a computer, performs the method according to any one of claims 1-3. Systems for editing document information in a document, comprising: - storage means for storing the document, - means for transmitting the document information of the document to a printing device capable of printing document information on a surface provided with a position coding pattern,. 51.6 109 16 - means for receiving editing information from a reading device capable of reading position information from the position coded surface, - means for interpreting the editing information, - means for changing the document information depending on the interpretation of the editing information.