SE520010C2 - Method for arranging a first and a second coordinate area relative to each other by creating association between first and second coordinate areas, based on direction in which line was drawn over discontinuity in position-coding pattern - Google Patents

Abstract

An electronic representation of a continuous line that has been drawn over the surfaces is received while the position-coding pattern on these was recorded. An association between the coordinate areas is created based on a direction in which the line was drawn over a discontinuity in the position-coding pattern. First and second associations are created if the line was drawn in first and second directions respectively. An Independent claim is included for: (a) a computer program product (b) a device for arranging first and second coordinate area relative to each other

Description

10 15 20 25 30 520 010,,. . . . When entering data using any of the above methods, it may be desirable to link different electronically available data sets, such as, for example, two electronic documents or an electronic document and a meeting item in an electronic calendar. It would also be desirable to be able to arrange the data sets into structures or hierarchies, to sort them or to be able to browse between them in a structured manner with the aid of, for example, a web browser.

An object of the present invention is to provide a method, a computer program product and an apparatus which enables a user to easily arrange a plurality of such documents or amounts of data.

Summary of the Invention The above objects are achieved in whole or in part by a method according to claim 1 and by a computer program product according to claim 16 and a device according to claim 17. The dependent claims indicate embodiments of the present invention.

Thus, the invention comprises a method for mutually arranging a first and a second coordinate area, which on a first and a second surface are encoded by a position coding pattern, which method comprises the steps of receiving an electronic representation of a continuous line drawn over the surfaces. while recording the position coding pattern thereon, and creating an association between said first and second coordinate regions, based on a direction in which the line was drawn over a discontinuity in the position coding pattern, whereby a first association is created if the line was drawn in a first direction, and a second association is created if the line is drawn in a second direction. Through this method, associations between coordinate areas can be created in a simple and intuitive way for the user. The associations can then be used in many different ways, such as, for example, sorting or linking data sets connected with coordinate areas. By using the direction in which the line is drawn, the user can control the direction of the association or the relationship between the coordinate areas. It is also possible to link additional data sets to coordinate arrays after the association is created.

Exactly how the direction of the line controls the relationship between the coordinate areas can be determined by the software that utilizes or interprets the association.

An electronic representation of a physically continuous line may be a series of coordinate pairs, a series of images, an image or any other machine-readable representation of a line known to those skilled in the art. A discontinuity can be, for example, a jump in the position coding pattern with which the physical line was transmitted in electronic form. Such a jump can occur when the physical line is drawn over a substrate so that different areas of the position coding pattern are registered in the same line.

The method described above can be applied either in a user device or in a data processing unit. Furthermore, it can be performed in real time as coordinate pairs are registered or at an appropriate time based on a representation of the line.

The invention further comprises a computer program product comprising programs for mutually arranging a first and a second electronically stored data set. When executed, the program performs the method according to the invention.

The computer program product may comprise a suitable storage medium and be executed either in a user unit or in a data processing unit.

The invention also comprises a device for mutually arranging a first and a second coordinate area, which on a first and a second surface are coded by a positional coding pattern. The device has means for receiving an electronic representation of a continuous line, which is drawn over the surfaces while the position coding pattern on them has been registered, and means for creating an association between said first and second coordinate areas, based on a direction in which the line was drawn over a discontinuity in the position coding pattern, whereby a first association is created if the line is drawn in a first direction, and a second association is created if the line is drawn in a second direction.

The device according to the invention may be a user device or a data processing unit, as described below.

The invention will be described in more detail in the following with reference to the accompanying schematic drawings which, by way of example, show presently preferred embodiments of the invention according to its various aspects.

Brief Description of the Drawings Fig. 1 schematically shows parts of a system in which the present invention can be used.

Fig. 2 schematically shows a user device which can be used in the system in Fig. 1.

Fig. 3 schematically shows a position coding pattern, which can be used in connection with the system in Fig. 1.

Fig. 4 schematically shows application of areas from an imaginary surface (Fig. 4a) to substrates (Figs. 4b, 4c, 4d). Fig. 5 schematically shows an aspect according to a first embodiment of the present invention.

Fig. 6 schematically shows a second embodiment of the present invention.

Fig. 7 schematically shows a third embodiment of the present invention.

Fig. 8 schematically shows a fourth embodiment of the present invention.

Fig. 9 schematically shows a fifth embodiment of the present invention.

Fig. 10 schematically shows a sixth embodiment of the present invention.

Fig. 11 schematically shows a seventh embodiment of the present invention.

Fig. 12 shows a schematic flow chart according to a first aspect of the present invention.

Fig. 13 shows a schematic flow chart according to a second aspect of the present invention.

Fig. 14 shows a schematic sketch of the data processing means of the user device.

Description of preferred embodiments Fig. 1 schematically shows parts with which the invention can be realized. These comprise two or more substrates, also called "surfaces" 2a, 2b, which are provided with position coding pattern P, a reading pen or user device 1 which can register the position coding pattern at the same time as graphic markings are made on the substrate.

Furthermore, there are one or more storage units 6 for storing electronic representations of information on the data and a data processing unit 5 for processing data stored on the storage unit and for communication with the user device. The storage unit 6 is shown in the accompanying drawings as only one storage unit, but can of course consist of several storage units located in different places. In the same way, the data processing unit 5 can be several data processing units, with the same or different functions. In addition, the data processing unit and the storage unit can be integrated into one unit.

With the user device 1, the position coding pattern on the surfaces can be registered, whereby coordinates for positions or points on a surface can be calculated. The coordinates can be analyzed in the user device and / or sent via a communication network, as discussed below, to the storage unit or data processing unit for analysis, storage or for performing services in response to the coordinates registered based on the position coding pattern.

Fig. 2 shows a schematic example of a user device which is suitable for use in connection with the present invention. This user device 1 can be of the type described in WO Ol / 26033.

The user device 1 comprises a housing 11, which is shaped approximately like a pen. In the short end of the housing there is an opening 12. The short end is intended to abut or be kept at a small distance from the surface or the substrate, on which registration of the position coding pattern for position determination is to take place.

The housing mainly houses an optics part, an electronics part and a power supply.

The optical part may comprise at least one LED 13 for illuminating the surface to be imaged, and a light-sensitive area sensor 14, for example a CCD or CMOS sensor, for detecting a two-dimensional image. Optionally, the device may also comprise an optical system, such as a mirror and / or lens system. The LED can be I I 1 x i I | 10 15 20 25 30 520 010 - «..- 7 be an infrared LED, and the sensor may be sensitive to infrared light.

The power supply to the device is obtained from a battery 15, which is mounted in a separate compartment in the housing.

It is also conceivable to provide the power supply via a cable from an external power source (not shown).

The electronics part contains signal processor 16, which comprises a processor 16a with working memory 16b and program memory 16c. Furthermore, the device comprises a storage memory 16d for storing, for example, coordinate data. The processor 16a is programmed to read images from the sensor, to detect the position coding pattern in the images and decode it into positions in the form of coordinate pairs, and to process the information thus registered in electronic form in the manner described in more detail below.

The device further comprises in this embodiment a writing pen 17, with the aid of which one can write ordinary dye-based writing on the surface on which the position determination is to take place. The writing pin 17 can be folded in and out so that the user can control whether it is to be used or not. In some applications, the device does not need a stylus at all.

The dye-based font may be of such a type that it is transparent to infrared light and the position coding pattern is absorbent to infrared light.

By using an LED, which emits infrared light, and a sensor, which is sensitive to infrared light, the pattern is sensed without the above-mentioned writing interfering with the pattern.

The device may further comprise buttons 18, by means of which the device can be activated and controlled. It may also have a transceiver 19 for wireless transmission, for example 10 15 5 25 a 520 010 n a n a n v-u-a 8 with IR light, radio waves or ultrasound, of information to and from the device. The device may further comprise a display 20 for displaying positions or registered information.

The device may be divided into different physical housings, a first housing containing components necessary to take pictures of the position coding pattern and to transmit these to components located in a second housing and which perform the position determination on basis of the registered image or images.

Fig. 3 shows a schematic example of a position coding pattern P which can be used in connection with the present invention and which can be applied, for example, to the substrates 2a and 2b. In the other accompanying drawings, the position coding pattern P is shown only schematically, such as a dotted surface.

This position coding pattern is used to register in electronic form what is written on the data.

Various types of position coding patterns that can be used for this purpose are previously known. For example, US 5,477.012 shows a position coding pattern, where each position is coded with a unique symbol. The position coding pattern can be read with a pen, which optically detects the position code, decodes it and generates coordinate pairs which describe the movement of the pen over the surface. WO OO / 73983 and WO Ol / 26032, which are hereby incorporated by this reference and which are both left to the applicant for the present application, describe another position coding pattern, in which each position is coded by means of a plurality of symbols of a simpler type and where each symbol contributes to the coding of more than one position. WO OO / 73983 uses various large dots to encode ones and zeros in the position coding pattern, which is binary 520 010 9 binary. In WO Ol / 26032, four different offsets of a dot from a nominal position are used to encode four different bit pairs in the position coding pattern. A certain number of dots, for example 6 * 6 dots P1, P2, encode a unique position. The position can be calculated on the basis of the bit values corresponding to the dots.

The position coding patterns in WO OO / 73983 and WO Ol / 26032 can be detected optically with a pen, which decodes the dots and generates a coordinate pair for each set of, for example, 6 * 6 dots. If the position coding pattern is read while the pen is writing on the position coding pattern, a sequence or series of coordinate pairs is obtained, which describes the movement of the pen over the position coding pattern and which thus constitutes an electronic representation of what is written on the paper.

In the following, it is assumed that the substrates 2a and 2b are provided with a position coding pattern of the type described in, for example, WO Ol / 26032.

The position coding pattern in WO OO / 73983 and WO Ol / 26032 can encode coordinates for a very large number of points or positions. These points can together be said to form an imaginary surface, ie a "thought" or "theoretical" surface, which can be described with the help of coordinates, but which does not really exist in the physical sense. Fig. 4a schematically shows such an imaginary surface 4, where three areas A, B and C are marked.

The imaginary surface can be divided into smaller areas by storing coordinate pairs representing the boundaries of each such area in, for example, a database. On the imaginary surface in Fig. 4a, the three areas A, B and C are delimited by coordinate pairs indicating the corner points of each area. 10 15 20 25 30 5:20 0.10 š¿ff: ä; With a position coding pattern, as described above, such coordinates can be coded in machine readable form.

By registering the position coding pattern in a point, the coordinates of the point on the imaginary surface can be calculated.

By applying position coding patterns on a substrate 2, for example paper, coordinates for points on the imaginary surface 4 can be coded on the substrate 2. If an area on the imaginary surface is coded by a position coding pattern applied to a substrate and reserved for this particular substrate, say that the area is “cut out” from the imaginary surface and placed on the substrate 2. This is described in more detail in, for example, WOO1 / 48685, WOOI / 48678 and WOOl / 48591, which are hereby incorporated by this reference. The imaginary surface can be divided into areas and sub-areas by specifying the areas in a database and connecting them to, for example, functions or owners. When an area is assigned a function or an owner, it can be reserved so that it cannot be assigned to other functions or owners at the same time. A list of reserved areas can be kept available, for example via the Internet.

Fig. 4b shows a first example of this, where the areas A and B from the imaginary surface 4 in Fig. 4a are "cut out" and applied next to each other on a piece of paper 2, by the coordinate areas encoding corresponding areas on the imaginary surface being applied to the paper.

Fig. 4c shows how the areas A and C are "cut out" from the imaginary surface 4 and applied next to each other on a paper 2 in the same way as in Fig. 4b.

Fig. 4d shows how the cut-out areas A and C are arranged on different substrates 2a, 2b and how the substrates are then brought together so that a line can be drawn over 10 15 20 25 520 010 i .. I. ll areas A and C whereby registration of the position coding pattern can be done with the user device.

Figs. 4b and 4c thus show data with two coordinate areas A, B and A, C, respectively.

A function can be linked to each area. The coordinates, for example coordinates for the corner points of each area on an area, are then stored in a database, so that it is possible to locate the area to which the coordinate pair belongs with knowledge of a coordinate pair in the database. In this database, each area can then be linked to, for example, addresses to which data marked in an area is to be sent, applications, functions or other areas that are associated or connected to the area.

It is also possible to define subscribers, licensees or owners of areas in the database, so that information marked in a specific area can be linked or sent to a predetermined unit. An example of records in such a database is shown in Table 1.

Table 1: Examples of records in a database containing n areas.

Area Belongs to Function | ._1 x1, y1; x2, y2 Companies Send to (lll.ll 11.1) x3, y3; x4, y4 Companies 2 Send.to (222.22.22.2) xn¿, yn¿; x¿, yn Companies n Send.to (nnn.nn.nn.n) FT For example, when a coordinate in an area according to the next top row in Table 1 is registered by the user device or received in a server, the data that is 10 15 20 25 30 520 010 l2 entered in the relevant area or on the relevant documentation is sent to an IP address (in the example according to table l: lll.ll.ll.l), which belongs to Company l, for further processing.

A database described above can be found both in the pen 1, for the functions that the pen 1 should directly recognize, or in the storage unit 6 or in connection with the data processing unit 5, for the functions that can be performed therefrom. When coordinates encoding the respective area are arranged on substrates, as shown in Figs. 4b and 4c, areas connected to functions or network addresses can be provided on the substrates. When coordinates of an area are registered by a user device 1, this can trigger a function to be executed in the user device or in some unit with which the user device 1 communicates.

Each area consists in each dimension (x, y) of a number of sequential coordinate pairs, so a line drawn and registered within an area (for example area A + B in Fig. 4b) with the user device will consist of a series of coordinate pairs. By a line is meant here a graphic marking made with a user device on a surface without lifting the user device from the surface. The fact that the user device is lifted from the ground can be detected, for example, by connecting the writing device of the user device to a pressure sensor, or by measuring the distance to the position coding pattern optically.

A line that is continuous on a physical surface can be interpreted in several different ways, in addition to being a regular graphic line. By physically continuous is meant that the line is made in one stroke with the user device. However, the electronic representation of the line may be continuous or discontinuous depending on the position coding pattern on the substrate on which the line is registered.

By the electronic representation of the line is continuous is meant that the distance on the imaginary surface between coordinate pairs for two directly registered points in succession is less than a certain, predetermined distance value. A physically continuous line can thus be discontinuous if, when registered in electronic form, it contains coordinate pairs from at least two different, different areas of the position coding pattern.

Thus, if a line is drawn and recorded over two areas (e.g., areas A and C in Fig. 4c), which are not adjacent to each other on the imaginary surface, the recorded line will include a discontinuity.

A discontinuity can be detected, for example, when the value of the distance on the imaginary surface between two points registered directly one after the other is greater than said predetermined distance value. The predetermined distance value can be selected depending on the type of areas registered. A line comprising such a discontinuity is hereinafter referred to as a "hyperline".

Discontinuities can either occur as a result of two areas of the position coding pattern being applied to the same substrate (Fig. 4c), or when two substrates, on which different areas of the position coding pattern are applied (Fig. 4d), are brought together so that a line can be drawn over them.

The hyperline can be given a meaning that relates to a relationship between the different areas over which the line is drawn. This meaning can be to logically connect, or "associate", the areas to each other, so that for example the data set connected to one area is connected to a data set belonging to the other area.

Such a coupling or association may be stored on various 1. u; u v .f; .f - ~> w,,) g: ~: »_ ø. v n. n- n y w p __ ~ ~ 1 I - n u. a. I., V _., - »I. - ß J. u ...,,, u,. | v n w. , _ _ f 4 f »v f. ._,, s 14 ways in a storage unit and thus utilized by applications that have access to the storage unit.

The meaning may, as discussed below, be dependent on the direction of the drawn line, so that, for example, a note page and a business card are associated with each other in a manner which depends on the direction in which the line was drawn. The associations can be handled in the same way as the above-described connections of areas to owners and functions, as described in connection with Table 1 above. This is discussed in greater detail below.

Fig. 1 shows two substrates 2a, 2b which are wholly or partly provided with position coding pattern P. The substrates preferably contain information which has been digitized and is stored and accessible in electronic form 6a, 6b. The information can be arranged on the substrate 2a, 2b by a user writing it down on an area provided with position coding pattern P or by it being created electronically or read with a scanner and reprinted or printed on the substrate 2a, 2b. The electronic form of the information can be stored as electronic documents such as .pdf (Adobe Acrobat®), .doc (MS Word®), .svf (standard vector format) or other formats, but it is also possible to store the information in any form of raw data, such as point, coordinate or vector series which represent information recorded on data by means of a user device. Even more or less processed versions of such raw data, such as text files from OCR software, can be stored and utilized.

The storage can, for example, take place in a personal computer, a PDA, a mobile phone or even in the user device itself. A server comprising a data processing unit 5 and a storage unit 6 can also be used. Different 10 l5 20 25 30 @ ». . Electronic documents 6a, 6b can also be stored on different storage units 6, although this is not shown in Fig. 1.

Furthermore, a communication means 7 can be arranged between the user device 1 and the storage unit 6. This communication means 7 can for instance be a computer, a mobile telephone, a mobile telephone network, a telephone network, a computer network or a combination thereof. Exactly how the communication means works is of secondary importance, since its function is to enable the transfer of data from the user device to the storage unit, in a manner known to those skilled in the art. It is also conceivable to connect the user device directly to a computer network via cable, short-range radio link, IR link, mobile telephone network or in another suitable manner.

Optionally, the storage unit 6 can also be connected to a data processing unit 5, such as for instance a personal computer or other suitable processor-equipped device. The data processing unit 5 may be arranged to read electronic documents 6a, 6b stored on the storage unit and be provided with software for analyzing the documents or files, and for taking measures based on the results of such analyzes. The data processing unit 5 can also act as an intermediary between the storage unit 6 and the communication means 7, as well as communicate directly with the user device 1 and other units. In this way, the data processing unit 5 can receive, analyze and convert information to and from the user device 1. In Fig. 5, as well as in Fig. 1, a first embodiment of the invention is shown where two supports 2a, 2b are used, which contain information also stored in electronic form 6a, 6b on a storage unit 6 and which are provided with position coding pattern P. The information may, as described above, be pre-printed on the base 2a, 2b or recorded by the user. . When this was done is less important, provided that the information is stored and accessible in electronic form in the storage unit 6.

Figures 5-10 show a dashed line which schematically distinguishes the physical aspects of the invention (use of the user device and the substrate) from its logical aspects (processing in the data processing unit / storage unit).

The user now wants to create a reference 3a from a first document 2a to a second document 2b, so that when the user or someone else reads via his computer the corresponding first electronic document 6a, he will automatically receive a reference 3a to the corresponding other electronic document 6b.

According to the invention, the user places the two supports 2a, 2b next to or overlapping each other. With his user device 1, the user draws a line 3 from the first substrate 2a, over the joint between the substrates and a bit into the second substrate 2b.

When the user moves the user device over the data, the user device 1 registers a series of coordinate pairs, which corresponds to line 3. The coordinate pairs are usually stored as raw data in a memory in the user device and are not sent to the storage unit / data processing unit 6/5 until the user device 1 registers an area that indicates that transmission is to be initiated.

At the joint between the substrates 2a, 2b, however, the user device 1 registers a discontinuity in that a registered coordinate pair does not belong to the same area as the previous coordinate pair. Then discounted 20 15 25 25 30 520 010 -. . ,. , u. u l7 nuity is registered, the user device 1 first examines whether any of the areas belong to a predefined functional area, for example a transmission area. If so, this function can be performed, for example, to send the data stored in the storage memory 16d. If none of the areas is a functional area, instead the hyperline 3 can be registered as an association Ba, ie a connection between the areas, as described above. The association can be stored in the storage memory l6d and include references to the areas between which line 3 was drawn.

As an alternative, the series of coordinate pairs can be stored just like the other coordinate pairs, for later analysis in the data processing unit 5, whereby the discontinuity is detected there.

When the user then selects an area that has a send function, the coordinate pairs stored in the memory, including any associations, are sent via the computer network 7 to the storage unit 6. Optionally, only data associated with the send function, the current surface or any registered association is sent.

When the coordinate pairs are sent to the storage unit 6, they can either be stored as they are, for later processing, or immediately processed by the data processing unit 5.

Such processing may include interpretation of recorded data through, for example, OCR or ICR. The processing can also include conversion to other storage formats such as text documents, graphic files, forms, spreadsheets or database records. The processing may also include analysis and storage of registered associations 3a.

As can be seen from Fig. 5, the associations 3a registered by the user device 1 can be analyzed with respect to which areas the line has been crossed. For example, an on-area and an off-area can be determined, based on the time sequence or order in which the coordinates on each side of the discontinuity were recorded.

Figures 5a and 5b show how an association between two substrates 2a and 2b can differ depending on the direction in which the line 3 was drawn with the user device 1.

In Fig. 5a, the substrate 2a is registered before the substrate 2b.

The association may then be that the electronic document 6a refers to the electronic document 6b. In Fig. 5b, on the other hand, the substrate 2b is registered before the substrate 2a, so the association may be that the electronic document 6b refers to the electronic document 6a.

The associations can be stored on the storage unit 6 or on any other suitable storage unit. The storage of the association 3a can take place in various forms. According to a possible embodiment, associations are stored in a database, in the form of relationships between areas, as shown in Table 2. In Table 2, for example, Area 1 can be defined as off-area and Area 2 as on-area, whereby from the area can be the area registered first when the line was drawn and the to area may be the area last registered when the line was drawn. This information can be used, for example, so that Area 1 refers to Area 2, without Area 2 necessarily referring to Area 1. It is of course also possible to reverse the definition of from and to area, so that Area 2 refers to Area l. Which area is defined as access area or off area can also be application-specific, ie be dependent on the software that receives, uses and / or interprets the association.

Table 2: Examples of database of associations. lO 15 20 25 520 010 w. .A 19 Area 1 Area 2 X1, Y1íX2fY2 X3, Y3íX4, Y4 XS / YS / 'XGIYG X7ly7; X8Iy8 Xn-B / yn-Bíxn-Z / Yn-Z Xxvl / Yn-lïxn / yn It is also possible to store associations in other forms than the coordinate sets shown in Table 2, for example as document names, page numbers or other designations designated for specific areas. It is also possible to divide the imaginary surface into sub-areas with, for example, similar function, use or owner. In this way, the exact coordinate pairs do not necessarily have to be specified in the association database in Table 2 for an association to work, but only a sufficiently large part of the coordinate pairs for the area to be identified with sufficient accuracy for the current application. The associations can also be converted to terms that refer directly to the electronic documents that are associated, such as hyperlinks or paths on a storage device or in a computer network.

The associations can be stored in electronic form on a suitable storage medium.

Fig. 6 shows a second embodiment of the present invention, wherein a business card 30 and a calendar 31 are associated with each other. As in the first embodiment, here the business card 30 and the calendar 31 are assumed to be provided with position coding pattern P. Likewise, electronic equivalents 30a, 3la to the business card and the calendar, respectively, are stored and available. 10 l5 20 25 30 520 010 20 The business card 30 and the calendar 31 or the calendar side are placed so that a line 3 can be drawn from one to the other. The association 3a thus created can be made dependent on the order in which the business card 30 and the calendar entry 31, respectively, were registered.

If the position coding pattern P of the business card 30 was registered first and then the calendar entry 31, this may mean that the information on the business card 30 is entered in the calendar 31 and / or that a meeting is booked with the business card owner.

If, on the other hand, the calendar entry 31 is registered before the business card 30, this can be interpreted as meaning that a message is sent to the business card holder, which message may include an invitation to a meeting. The time and date of the meeting may also be determined by where on the calendar page 31 the line begins or ends.

Fig. 7 shows a third embodiment of the present invention, where a base 32 with notes is associated with a calendar spread 31. Depending on the direction in which the line 3 associating the bases to each other is drawn, different effects can be achieved.

If, for example, the documentation with notes 32 is registered first, this may mean that the notes are associated with a certain meeting or a certain calendar time.

Based on this association 3a, a meeting record in, for example, Microsoft Outlook® can then be created or modified so that the notes 32a are attached to the meeting record 3la.

If instead the calendar entry 31 is registered first, this may mean that contents of this, such as notes, dates or the like, are transferred to notes according to the calendar entry 3la. It can also mean that data from the calendar look-up 31, 3la, such as, for example, time, place and participants in a meeting are transferred to the notes 32a.

Fig. 8 shows a fourth embodiment, in which a surface 34, for example a desk surface, a refrigerator or a notice board, is provided with a position coding pattern P and a number of substrates 33, containing information, placed on the surface. The substrates 33 are also provided with position coding patterns. By drawing hyperlines 3 from the respective substrate 33 to the physical surface 34, electronic equivalents to the contents of the substrate 33a are connected to an electronic variant 34a of the physical surface 34, for example the "Desktop" in Windows®.

If the physical surface 34 is instead a refrigerator, for example notebooks 33 which are set up on the refrigerator can be associated with an electronic version of the refrigerator surface and stored there for transfer to, for example, a calendar, a to-do list, a shopping list or a electronic message.

Fig. 9 shows a fifth embodiment of the present invention, in which a sequence between a number of substrates is registered. The documents are provided with position coding patterns and can be prints of, for example, pages in a book, photos in a photo album or transparencies for a presentation.

The user arranges the images in the order in which he / she wants them and then draws hyperlines 3 between pairs of substrates 35, or over all the substrates in a line. By registering the direction of the hyperlines 3, the data are associated with each other and an order can be separated, according to which the electronically stored equivalents 35a to the data can be sorted and presented. Fig. 10 shows an extension of the embodiment described in connection with Fig. 3, where documents 36 were connected to each other by means of lines 3 indicating references from one document 36 to another. 36. By registering the direction of the lines 3 drawn from document to 0 document, it is possible to determine how the documents 36, 36a should relate to each other, so that a hierarchy can be created. For example, this can be used to create a structure of electronic documents 36a, where, by means of printouts of the documents 36, references are formed between them by laying out the documents on a surface and then drawing lines 3 between them in the order in which wishes. The references can advantageously be inserted into the electronic documents in the form of hyperlinks, but they can also be stored as described above, in the form of database records. It is also possible to use associations to create literature references to other documents.

Fig. 11 shows an embodiment of the invention in which notes written on a substrate 32 and registered in electronic form 32a are sent as an e-mail message 38, by drawing a line 3 from the substrate 32 with the notes to a business card 30 which is provided with a position coding pattern and sent to the business card holder, such as e-mail or fax.

This can be made possible by data from the business card being stored in a storage unit 6. By instead drawing line 3 from the business card 30 to the base 32 with notes, another function can be initiated, such as for example information from the business card being added to the notes. .

Fig. 12 shows a flow chart of the input of data to the pen 1. When the user device in step 40 is moved in a line over a substrate, it receives in step 41 a picture 10. . . _,. 5 0 "'~" - .-. “. lf ”.

:. '°' 'I «. n- y; >, ° '^' '1 ~ 1 | 1 s; ~ '"I» 1.

'I 1. - ... ov data in the form of a series of images representing the position coding pattern of the substrate. In addition, in step 4la, time data is received, which indicates when each image in the series is taken. The time resolution of time data is advantageously adapted to the space between the images, for example 1/50 or 1/100 of a second. Time data may in this context be related to the order in which coordinate pairs are registered, rather than the actual times at which registration takes place. Image data is analyzed to calculate coordinate pairs for points on the line from the position coding pattern in step 4lb. The processing of image data and time data can take place in real time, for example so that a coordinate pair is calculated for each image as soon as it is registered.

Thus, the flow chart in Fig. 12 can be interpreted so that the step 41 of receiving image data means that an image is received and that the step 41b identifying coordinate pairs means calculating a coordinate pair in that image etc. The calculated coordinate pairs form a series of coordinate pairs, which may constitute an electronic representation of the line.

However, Fig. 12 can also be interpreted as receiving a whole sequence of image data in step 41. This method can be applied when the identification does not have to take place in real time, such as in the data processing unit 5. Thus, the interpretation of Fig. 12 can be controlled by whether the method is applied in a user device 1 or in a data processing unit 5.

The coordinate pairs are stored in step 42 in a storage memory 16d, 48. The stored coordinate pairs can be provided with a time stamp, which describes the exact time at which each coordinate pair was registered. As the coordinate pairs are registered, an analysis of them can be performed, whereby discontinuities are detected in step 43. When a discontinuity is detected, this can be analyzed to determine in step 44 in step 44 whether the discontinuity leads to an area with a predetermined function, such as, for example, a transmission area. If so, the stored coordinate pairs associated with the transmit function can be sent in step 46 to a storage / data processing unit for further processing. If the discontinuity does not lead to a functional area, the user device in step 45 can create a flag, which points out the discontinuity. Such a flag can take various forms and can advantageously be stored in step 47 in the memory device 166, 16c of the user device or in the storage memory 16d to be later sent to the data processing unit 5 or the storage unit 6. The flag can in practice, by the data processing unit , is interpreted as an association between the designated coordinate areas. For example, it may contain from-area and to-area in any suitable form, or it may contain coordinate pairs for each side of the discontinuity and timestamps for each coordinate pair. It is also possible to store the discontinuity in step 47 in the form of a hyperlink or as some combination of the above. As mentioned above, the step of creating a flag is not necessary, as the series of coordinate pairs can be analyzed afterwards instead. The storage of data in the user device or in the storage unit 6 can advantageously be preceded by compression in a manner known to those skilled in the art.

Fig. 13 shows reception and processing in the data processing unit 5. According to a preferred embodiment of the invention, in step 50 the data processing unit receives data from the user device in the form of series of coordinate pairs or image files, representing an electronic version of the user device's movement over the imaginary surface encoded by the position coding pattern_ Received data 52 can be stored in step 51, in the storage unit 6 before or in connection with them being processed by the data processing unit.

Identification in step 53 of discontinuities can be performed in the same way as in the user device, ie by analyzing the distance between coordinate pairs registered close to each other in time. However, it is also possible to identify the discontinuities based on information provided by the user device, such as flags, etc., which indicate discontinuities already identified by the user device. The analysis in step 54 of the discontinuity can be performed entirely in the user device, completely in the data processing unit or divided between them, so that some analysis is performed in the user device 1 and the rest in the data processing unit 5.

The discontinuity is analyzed, among other things, in order to determine in step 55 whether any of the areas are connected to a function, such as, for example, sending an e-mail message or fax message. If so, in step 56 this function is executed in a manner known to those skilled in the art. If, on the other hand, the discontinuity has nothing to do with a function, it can be considered as an association, whereby the association data 58, as described above, is created in step 57 and stored on a suitable storage unit 6.

Fig. 14 shows a schematic sketch of the data processing structure of the user device 1, which comprises a processor 16a, which is arranged to receive input data, analyze them and generate output data for, for example, working memory 16b, program memory 16c, storage memory 16d or for communication with external units 5, 6. The data processing structure according to a preferred embodiment comprises input means 21 comprising means 21a for receiving image data and means 21b for receiving time data from a time measuring means 24. The data received is analyzed in analysis means 22 and output by output means 23. Above 10 15 20 25 30 5:20 0.10 ši fi ïïäf gg .n ~ .. 26 said means 21, 22, 23, 24 may consist of purpose-adapted customer-specific integrated circuits (eg ASIC) or of some form of programmable integrated circuit ( eg PROM, FPGA) or standard processor with software.

Alternative embodiments It is possible to register in the user device only coordinate pairs, which are stored in the memory of the user device until the user device registers a transmission function. When the transmit function is registered, the coordinate pair in the memory of the user device is sent to the data processing unit / storage unit for further processing.

If the data sent to the data processing unit / storage unit is grouped according to which area of the imaginary surface they are entered on, the data processing unit will not necessarily be able to find two consecutive coordinate pairs, which belong to different coordinate areas. This can also occur if the coordinate pairs are grouped side by side, based on the data on which they were registered.

The data processing unit can then instead analyze the received coordinates and identify the lines that could constitute hyperlines. Examples of such are lines which lie at an edge of a coordinate area or at an edge of an area arranged on a substrate. These lines can then be analyzed as described above, ie with respect to the timestamps of the coordinate pairs included in the line.

It is also possible to analyze each line individually, based on the timestamp of each coordinate pair. This can be done by “following” each line from beginning to end and by lines that are in time close to the start or end point of the analyzed line also being analyzed. l0 15 20 25 30 520 010 .n .- 27 An alternative method of identifying discontinuities is to store in a memory information about where the imaginary surface is divided or "cut up", ie where the boundaries between different areas go. knowledge of this, lines that are registered can be examined to see if they cross a predetermined boundary between different areas.Another way to identify discontinuities is to use a position coding pattern that can directly code sub-areas.Thus it is possible to calculate a coordinate directly from the position coding A difference in area affiliation between two coordinate pairs registered in time directly after each other can thus be an indication of a discontinuity.

Another way of storing representations of the line, from which associations can be derived, is in the form of files in image or vector format, which represent the series of coordinate pairs registered by the user device 1 when lines are drawn on substrates provided with position coding patterns. Such a file may include lines drawn over arbitrary areas from the entire imaginary surface, or only lines drawn within certain sub-areas, for example sub-areas linked to a specific substrate such as a page of a document or the like. Using time data for each coordinate pair in such a series, it is possible to scan or follow lines in such files in chronological order. Through such a method, one can find jumps on the imaginary surface between two coordinate pairs which are registered directly one after the other. Alternatively, a series of coordinate pairs can be analyzed to determine if it includes more than one coordinate area. Optionally, such a file on the image may 10 l5 20 25 30. «. . y v 28 or vector format can be used as the sole storage source for associations.

It is possible to store "keys" for each registered line in a database, where the key points out, for example, a start or end coordinate for each registered _ line. With the help of the keys, the file in image or vector format can be scanned line by line, which can save time compared to scanning the entire file in chronological order. Optionally, knowledge of the approximate time of registration or area affiliation can also be used as supplementary input data.

All methods of storing or representing associations and lines can be used in the user device 1 and the data processing unit / storage unit 5/6, respectively.

The choice between them can be determined by factors such as processor capacity, search speed or storage capacity.

As a further alternative, associations can be stored in or directly adjacent to the documents they associate between. An example of such storage is to provide a document which refers to another document with a hyperlink or the like. Such hyperlinks can be arranged where appropriate, for example directly in the text, in footnotes or in endnotes / bibliographies. Another alternative is to let the association control the physical storage of associated documents, so that each document is stored in one place in a path hierarchy, which place is determined by documents associated with the document.

The invention can also be applied to substrates which have been provided with position coding patterns, but which have not yet been provided with information. In this way, “predetermined associations” can be created, which associate areas on the imaginary surface with each other, so that the information that is later recorded on the data or otherwise linked to the respective area will be included in the association.

The above-described embodiments are to be construed as examples of applications of the present invention.

Of course, it is possible to combine the described applications or parts thereof in a number of ways.

In addition, a very large number of other applications are conceivable and are covered by the appended claims.

Claims (20)

10 15 20 25 30 520 010 30 PATENT REQUIREMENTS A method for mutually arranging a first and a second coordinate area, which on a first and a second surface are encoded by a position coding pattern, which method comprises the steps of receiving an electronic representation of a continuous line, which is drawn over the surfaces below it. that the position coding pattern on them is recorded, and to create an association between said first and second coordinate ranges, based on a direction in which the line was drawn over a discontinuity in the position coding pattern, a first association being created if the line was drawn in a first direction, and a second association is created if the line is drawn in a second direction. The method of any of claims 1, wherein the step of receiving an electronic representation comprises receiving a series of coordinate pairs representing the line and determined from the position coding pattern. The method of any of claims 1 or 2, further comprising the step of detecting a discontinuity by comparing a distance between two coordinate pairs, which are recorded during a predetermined time interval, with a predetermined distance value. A method according to any one of claims 1 or 2, further comprising the step of receiving in electronic form an indication of a discontinuity in the position coding pattern. A method according to any preceding claim, wherein the step of creating an association comprises determining the direction of the line by comparing a time of registration of a first in-line coordinate pair with a time of a second in-line coordinate. 520 010 31 coordinate pair registration, which coordinate pairs are on each side of the discontinuity. The method of claims 1-4, wherein the step of receiving an electronic representation comprises receiving in electronic form an indication of the direction. A method according to any one of the preceding claims, wherein the step of creating an association comprises creating a database record, comprising a reference to said first and second coordinate areas and an indication of a direction between them. A method according to any one of the preceding claims, wherein the step of receiving an electronic representation comprises receiving the electronic representation from a user device. The method of any of claims 1-7, wherein the step of receiving an electronic representation comprises receiving an electronic representation from a sensor. A method according to any one of the preceding claims, wherein said first and second coordinate ranges are linked to first and second sets of data, respectively. The method of claim 10, wherein said amounts of data comprise electronic documents. The method of claim 10 or 11, wherein said association is a reference from one of said data sets to the other. The method of any of claims 10-12, wherein the step of creating an association comprises creating a database record having a reference to said first and second data sets and an indication of a direction between them. A method according to claim 10 or 11, wherein the step of creating an association comprises modifying 10. »A -. u u. a- | , -, v \ ». - - i nn f -H ..,;. . ; . .-. 1 I. - u »1 v | a o 4 »v. m .U H 32 one of said data sets to include a reference to the other of said data sets. The method of claim 14, wherein said reference is a hyperlink. A computer program product comprising programs for mutually arranging a first and a second electronically stored data set, characterized in that the program executes the method according to any one of claims 1-15. 17. A device for mutually arranging a first and a second coordinate area, which is encoded on a first and a second surface by a position coding pattern, characterized by means for receiving an electronic representation of a continuous line, which is drawn over the surfaces while the position coding pattern is recorded thereon, and means for creating an association between said first and second coordinate ranges, based on a direction in which the line was drawn over a discontinuity in the position coding pattern, a first association being created if the line was drawn in a first direction, and a second association is created if the line is drawn in a second direction. Device according to claim 17, characterized by a sensor for reading the position coding pattern on said surface. 19. Device according to claim 17 or 18, characterized in that the device is a user device, which is adapted for digitization of handwriting. Device according to claim 17, characterized by means for communication with a user device.