JP5244386B2 - Data management with electronic pen - Google Patents

Description

  The present invention relates generally to the management of digitally recorded data and in particular to the management of this data with an electronic pen.

The electronic pen can be used to generate information that electronically represents handwritten input on a product having a writing surface. As electronic information can be efficiently transmitted to different destination device for further processing, the information management system it may be desirable to incorporate such an electronic pen.

In US 2003/0061188, US 2003/0046256, and US 2002/0091711, which are hereby incorporated herein by reference, the Applicant can use each product to code multiple absolute positions on the product. position code proposes information management system is that apply to. The electronic pen can electronically record a sequence of positions that reflect the movement of the pen on the product by reading the position code.

The position code on each product is a subset of a much larger abstract position coding pattern. Examples of such abstract patterns, U.S. No. 6,570,104 which is incorporated herein by reference, is shown in US Patent No. 6,330,976, and U.S. No. 6,667,695 to open ing.

Abstract pattern is divided into subsets of a given size, each subset is associated with a unique identifier in the system. Each subset, in the case of each of the physical-friendly page, it is indicated by the pattern page is represented by a unique page address. In such a case, each of the absolute position, the page address may be represented in the local position in the relevant Tagged pattern pages.

Electronic pen may have for example, as described in U.S. No. 2003/0061188 is a publication of the applicant, the knowledge of the abstract position-coding pattern via the so-called template. Templates are pre-stored on the pen and define specific functional areas in the pattern . Pen is based on the function indicated by the template, to process the position recorded, and / or to provide feedback to the pen user by vibration when recording the position, for example, specific functional areas Can do.

By designating different portions of the abstract pattern to different destination devices, electronic information can be automatically directed from the pen to the correct destination device for processing. For example, the system may include a relay server. Relay server receives one or more absolute location from the pen, to identify the relevant lighted network address of the correct operation main (actor), Ru towards the flow of data into this address.

System such as this, the applicant of the previous publication is disclosed in U.S. No. 2003/0055865. In that system, an electronic pen connects to a so-called paper look-up server (PLS) in a first round trip and routes information based on the position data that the pen pulls from the position-coded product. Query it to do. Next, when receiving the routing information from the PLS, a second reciprocating (round-trip), the electronic pen, as indicated by the routing information, connected to the application server. The application server then requests the relevant position data from the pen.

System will benefit by providing a flow of streamlined data transparent it is also a disadvantage that an end-to-end device. Specifically, the system whenever data has to be transferred, not only each of the communication path, the pen, the PLS and each application, require that becomes accessible. The failure in one of the communication chain, possibly leaving the pen user does not properly informed about it, there is a possibility of data loss.

Another disadvantage is that the product cannot be generated independently of other components of the information management system. For example, the pen must store a template for the product, the relay server must know where to direct the recorded information from a particular product, and the application server You have to know what to do.
US 2003/0061188 US 2003/0046256 US 2002/0091711 US 6,570,104 US 6,330,976 US 6,667,695 US 2003/0055865

  The object of the present invention is to overcome the aforementioned problems of the prior art.

In general, the objects of the invention are achieved at least in part by a method, an electronic pen, a computer program product and an information management system according to independent claims, the preferred embodiments being defined by the dependent claims.

  One aspect of the present invention is a method for managing digitally recorded position data in an electronic pen, the method comprising incorporating the position data into a data file object suitable for storage in a file system; and Exposing to an external data handler.

Another aspect of the present invention is Ru electronic pen der for writing on the product. The electronic pen includes an image reading device that records at least one image of a product during writing, an image processor that determines position data indicating writing based on the at least one image, and data suitable for storage in a file system. and file assembler incorporating position data to a file object, Ru and an exposure unit for exposing the file object to the external data handler.

General advantages of these embodiments, the position data recorded by the pen is that can be processed asynchronously in the information management system. There is no need for round trip communication between the pen and other components of the information management system. Position data to be recorded, because it may be exposed by the pen as an autonomous data file objects. Thus, the data file objects may be sent in the system in a sequential chain transfer, the transfer is independent of other transfer. Certain parts of the chain may become temporarily inoperable without location data being lost in the system. The present invention, by designing the other components involved in the external data handler and transfer work properly, allowing the flow of streamlined data transparent.

Exposure data, the data file object is pushed against the external data handler, or by data file object external data handler, suitably from the file system in the pen memory, it is being Eject (Pull that) is the thing so that can be used for, may include. Push exposure requires an external data handler to be active, but any other component of the system may be offline . Exposure for removal (pull) may be achieved when all other components of the system are offline.

In one embodiment of the method , the method converts the first code record on the product to at least one absolute position in the coordinate system; converts the second code record to input data; and processing at least one of an absolute position based on the data further comprises a. Input data is not good to define at least one functional area the coordinate system. The method may further include mapping the at least one absolute position with respect to the input data in order to identify whether at least one of the absolute position is in the range of at least one functional area.

In one embodiment of the electronic pen, the image processor is operable to determine the position data based on at least one image of the first code on the product, location data, at least one of the absolute coordinate system position includes an image processor is also operable to determine the input data based on at least one image of the second code. Furthermore, the pen of the processing logic circuit is operable to process at least one of an absolute position based on the input data, the input data may define at least one functional area of the coordinate system, processing logic The at least one absolute position may be mapped to the input data in order to specify whether the at least one absolute position falls within the range of the at least one functional area.

General advantages of these embodiments, pen, complete description of the functional layout of the product (description), or a partial description, is that it may be dynamically applied. Description, any explanation that has been pre-stored in the pen memory, supplement or may be used to replace. Pen does not need to be pre-stored with such a description for all products. New descriptions and products may be introduced without the product or pen provider having to update the corresponding descriptions in the pen.

If the pen has access to a complete description of the functional layout, it can provide advanced user feedback on any functional area on the product, for example via the pen's MMI. Furthermore, the pen can output data in order to process the position recorded in view of such a complete description, a higher refinement level (refinement level). For example, refinement, and assigning the recorded position in the functional area, and / or, if rules associated with the functional area is indicated by the second code, or included in the second code If it has, the processing the recorded position in accordance with the rules, may include. Increasing the pen's knowledge of the product's functional layout can also reduce the need to distribute functional layout data for pre-storage in other parts of the information management system. Data is because it may be directly inferred from the pen data output.

By combining the aspect of outputting the recorded position data in the data file object with the aspect of coding the functional layout data, the dependency of the pen on the information management system and the information management system of the pen dependency, it is possible to further relax.

Further objects, features, aspects and advantages of the present invention is not only from the drawings, the following detailed disclosure, will be shown from the dependent claims which are appended hereto.

The invention will be described in more detail with reference to the accompanying schematic drawings.

<Abstract pattern>
The following description is based on the use of abstract position-coding pattern that will be divided into a plurality of pages units. Multiple pages units may each individually addressable in page units group hierarchy. 1 shows one example is shown. In this example, the pattern 106 includes a plurality of “segments” 110, the “segment” is divided into a plurality of “shelf” 111, the “shelf” 111 includes a plurality of “books” 112, and “ The “book” 112 is divided into a plurality of “page units” 113. A “page unit” is also called a “pattern page”. Suitably, all pattern pages have the same format within one level of the pattern hierarchy. For example, one shelf consists of A4 format pattern pages and the other shelf consists of A5 format pattern pages . Location of a particular pattern page abstract pattern is somewhat, such as an IP address, may be written in "segment. Shelf. Book. Page" format such as "99.5000.1.1500". For reasons of processing efficiency, may be I different from the internal representation of the page address, for example, it is shown as a predetermined length of the 64-bit integer, and the like.

In one example, each segment consists of pattern pages of more than 26,000,000, each pattern page has a size of about 50 × 50 cm ^2. In one embodiment, at least one segment is divided into 5,175 shelves, each shelf consisting of two books of 2,517 pages.

Each pattern page, as the actual subset of the coding pattern, or may be considered as an absolute position is coded by the subset. Such an absolute position may be expressed as a global position in the overall pattern coordinate system 114, or may be expressed as a logical position, ie a local position of the coordinate system 115 in the pattern page. .

An electronic pen has a global sequence of positions (ie, global pen strokes) on its position- coded product, or a local position (ie, an addressed pen) on the page address and corresponding pattern page. May be recorded as one of the strokes).

The following description is also based on each product that includes a position code corresponding to one or more pattern pages. However, it should be noted that the position code on the product need not conform to the pattern page. One or more subsets from one or more pattern pages may be arbitrarily arranged on the product. Product has embedded functionality, the position code on the product, selectively acts on the electronic pen strokes, including a specific location is associated with one or more functions.

Functionality of the pen, by the user operating the pen in a specific part of the position-coding pattern, is at least partially controlled. For this purpose, the pen stores one or more templates. The template defines how the pen should act on information recorded from different parts (functional areas) of the position coding pattern . In certain embodiments, for example a specific page unit group in the page hierarchy, such as segments or shelf, is effective for all pattern pages within that particular subset is associated with a given template definition. The template defines the size, placement, and function of any functional area that may affect pen behavior. Such functional areas are indicated as “pidgets”. FIG. 4 depicts a template 400 as laid out on a pattern page 402 and shows various pidget 404 arrangements.

In the template, all positions not occupied by pidgets in the pattern page are defined as belonging to the drawing area. The drawing area may be considered an implicit pidget . The position detected in the drawing area is interpreted by the pen and recorded as a pen stroke .

The pidget may indicate a trigger function, a service selection function, a device selection function, a local action function, a data selection function, an interpretation function or a feedback function. The trigger function triggers the pen to expose data as described below. The service selection function specifies a service. The service may convey contextual information (eg, email, fax, SMS) that affects pen processing ( record location processing ) and / or pen selection ( processing location selection ). Content information (for example, pages, books, shelves) that affects the content may be transmitted . Device selection function, the connection device for the pen (e.g., PC, mobile devices, LAN access point) to identify the. The local action function initiates an action that affects the pen memory. The data selection function maps the recorded position to pen resident data. For example, keyboard pin jets, rather than by mapping the position to the character in the pen memory, also, shortcut pin jets, may be to map the position in the communication address in the pen memory. The interpreter may operate to convert one or more strokes in the pidget to machine-coded characters. The feedback function may cause the pen to activate an internal MMI such as a vibrator, display or speaker.

Another type of pidget represents a new instance function. New instance function, by associating each pen stroke instance identifier, operates to separate the pen strokes that potentially overlap. For example, any a new instance function is started, a new instance identifiers, would be generated for relevant parts of the abstract pattern such as pattern page or book, as a result, recorded thereto after The pen stroke may be associated with its instance identifier. Instance identifier, page addresses and appropriately or associated, or incorporated into the page address.

In one embodiment, the template, define each pidget by pidget ID, and define the size and placement of local positions in the pattern page 402 (FIG. 4). The pidget ID is a 16-bit identifier that determines the pidget function. Pidget ID consists pin jet type (bits 8-15) and pidget number (bits 0-7). The pidget type may specify the aforementioned function, and the pidget number may specify a sub- function.

Above the trigger pin jet obtains Ri Do those specific ones or general. Particular trigger pin jet has a pidget ID identifying both the specific service and specific connection apparatus via the pidget number. Pen, when recording the position of such particular trigger pin in the jet, always achieve exposure data. General trigger pin jet has a pidget ID that does not identify a specific connection apparatus and / or a particular service. To trigger the exposure of data, generally of the trigger pin jets, by the pen record the corresponding position to form the trigger pin jets ID to identify both valid and active service access apparatus, device selection Must be used in combination with pidgets and / or service selection pidgets.

<Electronic pen>
FIG. 2 depicts an embodiment of a pen 200. The pen 200 has a pen-like housing or shell 202 that defines a window or opening 204. An image is recorded through the window or opening 204 . The housing is, camera systems, electronic systems and power supply are Nde free.

The camera system 206 comprises at least one of the illumination light source, a lens system, and an optical image reader (not shown in FIG. 1). Light source, suitably a light-emitting diode (LED) or laser diode, illuminates a part of the area seen through window 204 by infrared irradiation. Image of a region observed is projected to the image reading apparatus by the lens device. The image reading apparatus may typically be a two-dimensional CCD or CMOS detector which is triggered to capture images at a rate which is fixed about 70Hz to 100 Hz.

Power for sensor devices may alternatively replaced by a main power supply (not shown), or supplemented a battery 208 that can be.

The electronic system includes a controller 210 connected to the memory block 212. The controller 210 is involved in various functions of the electronic pen, and is a commercially available microprocessor such as a CPU (Central Processing Unit), DSP (Digital Signal Processor), FPGA (Field Programmable Gate Array), or alternative It can be realized by several other programmable logic devices such as ASIC (Application Specific Integrated Circuit), individual analog and digital components, or some combination of the above. Memory block 212 preferably includes various types of memory such as working memory (e.g., RAM) and program code, and a fixed storage memory (a nonvolatile memory such as a flash memory). The associated software is stored in the memory block 212 and executed by the controller 210 to provide a pen control system for electronic pen operation.

The housing 202 also has a nib 214 that allows the user to physically write and draw on the surface with normal dye-based marking ink placed thereon. The marking ink in the nib 214 is suitably transparent to the irradiated light to avoid interference with photoelectron detection with an electronic pen. The contact sensor 216 operates on the nib 214 to detect when the pen is lowered (pen down) and / or lifted (pen up) and possibly to determine the operating force Connected as possible. Based on the output of the contact sensor 216, the camera system 206 is controlled to capture images between a pen down and a pen up. Controller 210, the pen strokes to produce a sequence of temporally coherent positions representing electronically processes these images.

As further described below, the electronic system further includes a communication interface 218 for exposing data to nearby devices or remote devices such as computers, mobile phones, PAs, network servers, and the like. Communication interface 218, such short-range communication wired or wirelessly (e.g., USB, RS232, radio, infrared transmission, ultrasound transmission, inductive coupling, etc.) to the component, and / or, usually computers, telephones, or Components may be provided for wired or wireless telecommunications over a satellite communications network.

The pen may also include an MMI (Man Machine Interface) that is selectively activated for user feedback. The MMI may include a display, an indicator lamp, a vibrator, a speaker, and the like.

Furthermore, the pen may include one or more buttons that it the activation and / or controllable.

<Information management system>
FIG. 3 is an overview of an information management system in which the present invention may be used. As will be described later, the pen 200 captures pen stroke data into a file object 300 that is exposed to other components of the system. In one embodiment, the file object 300, as indicated by the path 300A~300C of 3 is pushed to a given local receiving device 302 or the remote receiving device 304. Path 300 A is set up directly between pen 200 and remote device 304 via communication network 310. The path 300B is set up from the pen 200 through the local device 302 to the remote device 304 via the network 310. Path 300C is set up between the pen 200 and the local apparatus 302. In another embodiment, the file object 300 is exposed for being pulled from the pen 200 by the receiving device 302, 304 along any one of the paths 300A - 300C.

The local receiving device or remote receiving device may include a flow controller 306. The flow controller 306 is an application program that resides either locally on the receiving device or on the remote device, as illustrated by the paths 300C1 , 300C2, on the file object 300 or data retrieved from the file object 300. Relay to 308 . Examples of such flow controllers, Ru Tei shown open in the aforementioned US No. 2003/0047256 and WO 03/056420 Pamphlet and WO 03/105064 pamphlet. Multiple flow controllers may be involved in data transfer from the pen to the application program.

The flow controller 306 includes, for example, HTTP (Hypertext Transfer Protocol) post, SMTP (Simple Mail Transfer Protocol), X. 400, MMS (Multimedia Messaging Service), via the SMS (Short Message Service), database synchronization, etc., may be configured with appropriate data to relay to the destination address. The destination address may be given directly or indirectly by relay data extracted from the file object 300 by the flow controller 306. In one embodiment, the flow controller extracts the explicit destination address from the file object. In another embodiment, the flow controller 306 extracts an address indicator from the file object and accesses a lookup table to derive a destination address based on the address indicator. For example, the address indicator may be a page address, global position, pidget ID, pen resident parameter value, and the like. Also, appropriate data to be relayed may be given either directly or indirectly by the relay data. As an alternative to relaying the file object 300, the flow controller 306 may send a notification to a destination address that the file object 300 can use for retrieval.

File object 300 or receives data drawn from the file object 300, application programs 308, physical product on the transmission side, in other words the product (Figure 3 position data in the file object 300 is generated therefrom, You may access the page description that defines (indicated by the symbol P). The page description defines the layout of the sending product P. The layout includes both the physical layout and the functional layout of the product. Physical layout includes support graphics, that is to instruct the user to control the user, and / or the information can understand human for the purpose of informing the user. The functional layout defines the arrangement and size of so-called user areas on the associated pattern page. The user area is an application-specific function area, and the arrangement, size, and function may be unique to a specific application program 308. In the page description, an identifier of a dedicated processing instruction that can be accessed by the application program may be assigned to each user area. Alternatively, each processing instruction may be included in the page description. Also, the physical layout, explicit instructions for the generation of physical layout, or, either as an indicator of the storage device address of the computer file that defines the physical layout, may be included in the page description.

  The structure, functionality and use of page descriptions are described in US 2002/0040816 and WO 03/105064.

  FIG. 4 depicts the interrelationship between the pattern page 402, the template 400, the page description 410 and the physical product 420. The physical product 420 includes a position code (not shown) that defines a position within one or more pattern pages 402 (only one is shown in FIG. 4). These locations are associated with one or more functions, either predefined in the system (for pidget 404) or application specific (for user area 414). The pidget 404 has a pre-defined placement and size within the pattern page 402, but may be arbitrarily placed on the physical product 402. The page description 410 defines not only the physical layout of the product 420 but also the placement, size and function of any additional user area 414 on the pattern page 402. Product 420 includes one functional layout portion 400 ′ provided by template 400 and one functional layout portion 410 ′ provided by page description 410.

The pen stroke data of the file object 300 (FIG. 3) is independent. That is, the receiving application program 308, the data without requiring any communication with the pen 200 that created the can for accessing and processing data. Thus, in contrast to the prior art, the pen 200 may asynchronously, i.e. without reciprocal (roundtrip) in the information management system, and outputs the data.

The file object 300 may be generated by the pen 200 at various levels of sophistication. For example, the file object 300 may include any one of the following or a combination thereof.
(1) Unprocessed pen stroke data, that is , a temporally coherent arrangement of global or logical positions constituting the pen stroke.
(2) Pen stroke picture, that is , an image that directly represents the recorded position.
3 refined pen stroke data, typically, ID of any pidget being contacted by the pen strokes, and / or templates related information of results of functions associated with pidget.
(4) For example, scrambled and / or encrypted pen strokes to prevent unauthorized processing or distribution of pen generated data of the information management system. In one example, the page address may be handled according to a predetermined algorithm or excluded from the pen stroke.
(5) Display of all of the user area to be contacted by the pen strokes, and / or as a result of the functions associated with the user area.

The file object 300 may include additional pen related data or stroke related data at various levels of refinement. The refinement level is, for example, any one of a pen stroke position and an arbitrary pidget time stamp, a pen stroke position force value (force value) , and an arbitrary pen resident parameter . Time stamps allow application programs or flow controllers to combine strokes and / or pidgets based on their mutual recording times. Each time stamp reflects the time at which the sender image of the position / pidget was captured by the pen camera system 206 (FIG. 2). The force value allows the application program to render an image of a stroke with a line width that depends on the force.

The pen resident parameters are stored in memory block 212 (FIG. 2) to identify the characteristics of the pen itself or the characteristics of the pen owner / user. These parameters may be used by the receiving application program 308 (FIG. 3) and thus may be at least selectively included in the file object 300. Including the pen resident parameter may be determined by the page address of the stroke in the file object and / or by the context information or content information described above. Pen resident parameters may include a pen unique identifier, a language identifier, a name, a street address, an email address, a telephone number, a pager number, a fax number, a credit card number, and the like.

File object 300, the character coding or non-character coding (e.g. binary) are well may be in any suitable format. File formats, rather than by a unique specification, or, for example, XML (Extensible Markup Language), HTML (hypertext mark-up language), XForms, may be standardized, such as PDF (portable document format).

<Pen control system>
The pen 200 is operated by software executed by the control device 210 (FIG. 2). The overall software architecture is described with reference to FIG. In FIG. 5, some of the major software components have him draw.

The pen system software is based on modules. A module is an individual entity in software that has a clean interface. Module, active der Luca by including at least one process, or whether it is passive by containing no process is either. Module, function interface executes a function call, or may have a message interface to receive messages. The active and passive modules are basically structured as a tree where the parent to the module is involved in starting and stopping all its children.

Also, the pen system software implementing the event framework to reduce the dependencies between modules. Each module may expose a predefined set of events that can be signaled. To get notification of a particular event, the module must be registered in the event register. Event register is either notified that must be made as to whether or callback functions must be performed by posting messages, may exhibit.

  The pen system software has the following two basic functions.

Remember what the pen writes and expose the recorded information when requested. The refinement step may or may not be performed prior to exposure .

The combination of software modules implements a so-called coordinate pipeline 502. The coordinate pipeline 502 is a flow of events for storing strokes . The global position string from the image processing module 504 is received by a translator module 506 that converts the global position to a logical position. When the translator module 506 detects a stroke through the pidget, a corresponding pidget event is triggered. Pi jet events, these modules including an event listener to determine the respective relevance of Pi jet event, causes the appropriate action by other software modules. The addressed pen stroke is received by the coordinate manager module 508. Coordinate manager module 508 assembles them into addressed pen strokes that are encoded in a compact representation and stored in memory block 212 (FIG. 2) .

<1. Supervisor>
As shown in FIG. 5, the pen system software includes a supervisor module 509 that activates all other modules directly or indirectly. The supervisor module 509 includes an interface to the event register so that the module can register itself for events that it wishes to receive. In addition, the supervisor module 509 generates a control event that can be used by the listener in other software modules.

<2. Server>
Server module 510 is provided as a coordinator for events at the pen. Server module 510 determines whether a to be what pen based on the state and the next event in the system.

<3. Image processing>
The image processing module 504 will not be described in detail here. The image processing module 504 can be realized, for example, according to the prior publication of the applicant below.

  US 2003/0053699, US 2003/0189664, US 2003/0118233, US 2002/0044138, US 6,667,695, US 6,732,927 No., US 2003/0122855, US 2003/0128194, and references therein.

<4. Location database>
The pen includes a location database 511. The location database 511 resides in the pen memory block 212 (FIG. 2) and can be edited or updated via the communication interface 218 (FIG. 2) . The location database 511 is used by the translator module 506 and includes a segment database 511A. The segment database 511A associates each segment with the template indicated by the template index . The segment database 511A is placed in a RAM memory cache for fast access. The position database 511 includes a template database 511B. Template database 511B associates the template index with a pointer to a template definition that may be stored as a parameter of non-volatile memory in the pen . Template definition, page X size (16 bits), the page Y size (16 bits), the number of pages (16-bit), pidget number (16 bits), may include a pin jet list (pidget number of length). The pidget list is composed of a plurality of pidgets. Each pidget has a pidget ID (16 bits, the most significant 8 is the type, and the least significant 8 is the number), X upper left coordinate (16 bits), Y upper left coordinate (16 bits), width (16 bits), high defined by the (16-bit).

<5. Translator>
Translator module 506 uses an algorithm based on the known pattern formatting data, translate global position to a logical position. The pattern format data defines a hierarchical division of the coding pattern into pattern pages (FIG. 1). Other parts may be dynamically pulled from position database 511 of the pen, but part of the pattern format data may be dynamically stored in the pen. For example, if all segments have a predetermined size , the global location may be processed to identify the current segment at that global location. Knowing the current segment, the data for that split may be derived from its template definition in the location database 511 so that the global location is further processed to the current shelf, book, page, and local location, That is, the complete logical position is specified . If the pen is operated on the same pattern page one for a while, put the boundary of the current pattern page global position in the cache, by checking any new global position for these boundaries, significant speed Up may be achieved. As long as the new global position is within the bounds, the need for page address recalculation is eliminated.

The translator module 506 also detects whether any recorded position falls within the pidget range on the current pattern page by mapping local positions to relevant template definitions given by the position database 511. To do. In one embodiment, only one pidget can be specified per stroke. That is, when multiple pidgets are identified , only the last pidget is considered. Corresponding pins jet event is then generated for notification to other software. If no pidget is specified , it is assumed that the position falls within the drawing area.

For each global position, the translator module 506 provides at least a page address and a local position. Lineups of temporally coherent positions, that is, when the stroke, the translator module 506 may provide pidget ID. Each logical position is also associated with a time stamp provided by the timing circuit of controller 210 (FIG. 2) and a force value provided by contact sensor 216. Each pidget ID is associated with a time stamp. The translator module 506 may also have an interface that allows other modules to look up information in the location database 511 for templates and pidgets.

<6. Coordinate manager>
Coordinate manager module 508 receives logical positions from translator module 506. Before being stored, logical position is temporally coherent sequence, i.e. are grouped strokes. This may alternatively be done by the translator module 506. The coordinate manager module 508 then preprocesses each stroke for compression and stores the result in non-volatile memory. Examples of such compression and storage is shown in U.S. No. 2003/0123745 and U.S. No. 2003/0122802.

The coordinate manager module 508 also includes interfaces for other modules. This interface, for example, to search for stroke stored based on the page address, and, is used to extract the stroke transport format. In one embodiment, the transport format includes, Ri binary der, session identifier, start time of each stroke, the force values of the local positions, and each position in each stroke. Optionally, the transport format may include orientation data calculated from the captured image to show the three-dimensional orientation of the pen in the recording position.

The session identifier indicates the pen working sessions different set of pen strokes is entered. For example, the session identifier may be a 32-bit signed integer value that is incremented each time the pen is switched on. The session identifier may be used by the application program 308 (FIG. 3) to reset any characteristics selected during the previous working session, as further described below.

<7. Parameter storage>
The pen system also includes a parameter storage module 512. All other modules that need to maintain the state after the pen stops call this module to store their state parameters. Furthermore, the parameter storage module 512 processes the pen-resident parameters set in the generation of such a unique pen parameters (pen identifier, etc.) or calibration parameters not only template definition. Each parameter, in the non-volatile memory, the name, are both stored sized and data. Module 512 provides an interface 512 ′ for parameter insertion, retrieval and deletion.

<8. Keyboard>
A keyboard module 514 is provided to enable text input to the pen. Template database 511B includes specific control templates including keyboard pidgets. A control template is defined for a series of pattern pages (eg, segments, shelves, or books), and different pages are assigned different languages to keyboard pidgets. For memory efficiency reasons, the mapping of each keyboard key to a particular character resides in non-volatile memory, and different languages are assigned to different page addresses in the series of pattern pages. Keyboard module, listening (listening) a paper event, at the time of the keyboard pin jet events, and what key is hit is calculated, and load the appropriate character from the memory based on the current page address. Ri by the interface 514 ', other modules, accessing character resulting.

<9. Item selection>
An item selection module 516 is implemented to allow the pen user to select items from the pen memory using the pen. Template database 511B includes one or more dedicated pidgets associated with items in pen memory. Such items, get Ri Oh in the address record of, for example, pen-resident address book. In order to select an item, the user checks the corresponding pidget. Item selection module 516 is implemented to listen for associated pidget events. Upon event detection, the item selection module 516 extracts the associated items and manages them internally in the bit vector. Other modules can access items via the interface 516 '.

<10. Interpretation>
An interpretation module 520 is provided for interpreting pen strokes. Interpretation module 520 is implemented to listen for associated pidget events. Upon detection of an event, the interpretation module 520 performs an interpretation function with a stroke associated with the event to generate an output item. Optionally, a plurality of strokes, for example, based on their position relative to pidget, may be grouped in front of the interpretation process. The interpretation function may, for example, convert strokes into machine- coded characters. Interpretation may be performed in a specific context (eg, email, fax, SMS) given by the aforementioned service identifier. Furthermore, the interpretation function may be supported by the pen resident dictionary such as the aforementioned address book. Interface 520 'allows other modules to access the interpretation results.

<11. Property assignment>
A property assignment module 518 is provided for creating a property table. A property table is a support data structure that includes properties assigned to pen strokes by the user, suitably by the user touching the pen with one or more property selection pidgets . Such properties may affect strokes that are temporally adjacent to the property selection event, i.e., preceding and / or following strokes, depending on the implementation. The property table may include a display of all property values provided by the checked property selection pidget, not just the associated timestamp.

In this way, the property assignment module 518 is implemented to listen (listen) for related pin jet events. Upon event detection, the property assignment module 518 retrieves the associated pidget value from the template database 511B and adds the pidget value to the property table.

A property may usually be associated with an instruction for the processing of a stroke or for the processing procedure itself. Property value, color, line thickness, line character may indicate the display quality of such temporal resolution or spatial resolution. In addition, the property value may indicate the desired interpretation context (letters, numbers, lowercase letters, uppercase letters, email addresses, fax numbers, bank account numbers, etc.). In yet another alternative, the property value indicates a page format parameter such as a page break, page sequence, or page margin. In addition, property values can be used as input data when various personal data stored externally (eg, social security numbers or personal numbers, phone numbers, bank account numbers) is processed when processing a specific set of pen strokes. It may serve as an “access granted” indicator to declare that it may be used.

The property table allows the application program 308 (FIG. 3) to easily extract the specified property value and apply it to the pen stroke. In this way, the application program does not need to match the pen stroke to the template definition to derive the selected property value. Alternatively, the application program may apply the property values in the property table directly by matching the property values to the pen strokes based on each time stamp. This matching may also take into account the session identifier described above, which is included in a stroke as expressed in the transport format described above. Accordingly, the session identifier whenever a change in the stroke data, all properties are reset to their default values. As an alternative or supplement contained in the stroke data, the session identifier may be included as an item which the time stamped in the property table.

<12. File assembly>
The file assembly module 522 is implemented to generate a file object 300 (FIG. 3) that is exposed to a data handler outside the pen. The module 522 is implemented to listen for dedicated events such as trigger pidget events. Upon detection of a dedicated event, the file assembly module 522 retrieves the stroke associated with the trigger pidget in the binary transport format from the coordinate manager module.

The functionality of the file assembly module 522 is dependent on the level of refinement. The above steps are sufficient for elaboration of the Level 1 as defined above.

For elaboration of Level 3, the module 522 may retrieve the data item from the other modules. Other modules are indicated by other pidgets associated with the trigger pidget. Such data items may be retrieved from the keyboard module 514, the item selection module 516, the property assignment module 518, and the interpretation module 520. Further, if necessary, the module 522, such data items, for example, to couple the output items from the character or interpretation module 520, the keyboard module 514. Finally, this module creates a file object 300 (FIG. 3) that can be used by other modules via the interface 522 ′.

Such a file object may have the following structure, for example.
File header (trigger pidget ID, page address, number of pages, number of properties)
Page data header (page address, number of pidgets)
Page data (transport format stroke data)
Pidget data (Pidget ID, time stamp)
Property data (pen resident parameter values, output items from keyboard, output items from item selection, output items from interpretation, property table from property assignment)

The file object allows the application program 308 (FIG. 3), and possibly the flow controller 306 , to identify and extract all or selected portions thereof.

Generation of file objects at refinement levels 2, 4 and 5 is <14. Stroke converter>, <15. Encryptor> and <16. Each will be described later in the section “Importer>.

Selection of the stroke included in the file object may be determined from the content information included in the trigger pidget ID. Such content information may indicate a current pattern page or a book, for example. That is, the selected stroke needs to belong to the same page unit group as the trigger pidget. Clearly, there are alternative ways of selecting the stroke. For example, stroke is dedicated pen strokes may be selected from being defined border region by (i.e., to indicate what should the exposed against the pen, the pen is moved on the product). In another example, all strokes in the pen memory are automatically selected for exposure and suitably one file object is created for each pattern page or book. In some cases, the stroke may be selected with respect to a time reference such as, for example, the time for the previous exposure of the data.

  All such selected strokes may be incorporated as strokes in the file object. Alternatively, only those strokes that are not represented as pidgets and / or property data are incorporated into the file object as actual strokes.

File assembly module 522, but it may also be implemented in order to insert a separate data file object. Independent data, even pen strokes recorded by the pen, not associated with the pen itself. Independent data, the cash despite being inserted, for example, may be associated with the file object by file object contains a pointer to another file object with such data.

The independent data may include any type of non-position data. Non location data may be derived by a sensor in the pen, or may be received by the pen via the communication interface 218 (FIG. 2). In one example, the independent data is input by the same image sensor that captures the image used for position determination. In another example, the independent data is input by an auxiliary image sensor in the pen. Such independent data, one or more images (i.e. still images or moving) rather by the form of, or data extracted from such images (e.g., decoded data, or any of the bar code Other machine-readable codes, or machine-coded text resulting from optical character recognition (OCR) processing. In yet another example, independent data may be voice data recorded by the sound sensor in the pen.

File assembly module 522, the translator module 506 identifies a dedicated pidget in coordinates of the stream, or by or events are generated by the interpretation module 520 to detect one or more pen strokes that form a dedicated command, independent Triggered to connect data to a file object. Alternatively, the event may be generated by a pen system that detects a button 222 (FIG. 2) on the pen being pressed, and a pen system that detects a dedicated voice command (eg, via a microphone in the pen). may be generated by, or communication interface 218 specific parameter values in the data received in (2), is generated by the pen system for detecting the reception of such dedicated flag bit (s) Good.

In certain user scenarios, the user may move the pen across a barcode on the product. Pen software module within the identifies the bar code in accordance with the standard of a suitable foreign, such as EAN or UPC, for example to decode the product identification number. The user may then instruct the pen to create and expose a file object that includes the product identification number and any location data recorded on the product or another product. The file object may be used, for example, to communicate product identification numbers for ordering, purchasing, or retrieving information via an information management system. In yet another alternative, product identification number, in order to identify the position-coded product information management system, that is used to replace the page address of the above or supplements.

In another user scenario, the user commands audio recording via a microphone in the pen or an accessory to the pen, possibly in synchronization with writing on the position-coded product with the pen. . The user then instructs the pen to create and expose a file object that includes the audio recording and location data recorded on the product. File objects, for example, may be used to form a memo records with enhanced audio, or to form a message voice message and document message is bound to be communicated to the recipient via the information management system May be used.

In yet another user scenario, the user commands the recording of a digital photo or video sequence by an image sensor in the pen or in an accessory to the pen. The user may instruct the pen to create and expose a file object that includes a photo or video recording and location data recorded on the product. The file object may be used, for example, to create a multimedia message or electronic postcard that is communicated to a recipient via an information management system.

In yet another variation, the independent data includes at least a portion of the relay data described above. The relay data preferably takes the form of an explicit address of the intended recipient of the information management system, such as the receiving device 304 that hosts the associated application program 308 (see FIG. 3) .

<13. Exposed>
An exposure module 524 is provided for exposing data collected by the data handler outside the pen. This module is implemented to listen for dedicated events such as trigger pidget events. Upon detection of a dedicated event, the exposure module 524 retrieves the file object from the file assembly module 522 and pushes the file to the specific device / port indicated by the device identifier, or the external data handler retrieves the file object from the pen. Either expose the data file object by allowing it to be pulled . For instance, the file object may stored in the file system in the pen memory, file system, it is exposed to the outside data handler. For example, file system, USB (Universal Serial Bus), FTP (File Transfer Protocol), may be exposed for browsing through the HTTP (Hypertext Transfer Protocol) or any other suitable protocol.

Exposure module 524, including all the necessary protocol stack. Further, exposure module 524, Ri by the responsibility to set up a link allocated to the individual modules, also Ri by to work with the device abstraction (device abstraction) is designed to be independent of the transmission means. Responsible for setting up the link is assigned to the link selection module (not shown). The link selection module handles the selection and setup of communication links, for example when the pen is to perform exposure of data file objects.

In one embodiment, exposure module, and an asynchronous mode for exposure of autonomous file object may switched between the conventional synchronous mode for performing online end-to-end communication of data. This mode may be indicated by an event that triggers exposure.

<14. Stroke converter>
For level 2 refinement, the pen system may include a stroke converter module 526 and a picture encoder. The stroke converter module 526 transfers a predetermined set of strokes to a picture buffer in the memory block 212 (FIG. 2) . A picture encoder encodes a picture buffer into a predetermined type of image, such as a bitmap format or a vector graphics format . The stroke converter module may be implemented to apply any properties assigned by the property assignment module 518 to the stroke during the image conversion procedure. For example, stroke converter module may be constructed a table in which the time stamp including the changes in all the display quality that is detected is pressed by the property assignment module 518, the said change while transferring the stroke to the picture buffer May apply to strokes.

<15. Encryptor>
For elaboration of level 4, the pen system may comprise Encryptor module 528. The encryptor module 528 processes all or selected portions of the file object according to a predetermined encryption algorithm to generate an encrypted file object . Alternatively or additionally, the Encryptor module, the page address of all or a subset of stroke may be eliminated or scrambled. For example, the exclude / scramble operation may be applied only to strokes from a specific page address.

<16. Importer>
As described above, the non-location data, via a sensor in the pen, or through the communication interface (218 in FIG. 2), may be imported into the pen. For example, the sensor may be a sound sensor such as a microphone, or may be an image sensor which is either a primary sensor or auxiliary sensors for generating an image for position decision.

The pen system may include one or more importer modules 530. The importer module may be dedicated to specific import means such as image sensor data, audio data or communication interface data. Alternatively or additionally, the importer module may identify data such as machine- coded text, data associated with a bar code, or another specific type of machine-readable code, photo data, video sequence data, etc. May be dedicated to this type of processing.

In the following, a specific type of importer module, the definition importer module , will be described in more detail.

The definition importer module can import definition data into the pen system that defines all or some functional areas on one or more specific pattern pages. Definition data can correspond to the template definition above, or may be a page definition described above for identifying the user area or a user area in any combination or subset thereof on the pattern page.

  As will be described later, the definition data allows the pen not only to provide file objects at refinement level 5, but also to operate with more advanced user feedback / control.

The definition importer module may be configured to import definition data either automatically via a pen communication interface (218 in FIG. 2) or by command. For example, the pen may connect to a local device (302 in FIG. 3) that retrieves definition data from internal memory or from a dedicated network server and provides this definition data to the pen. Alternatively, the definition data kit may be provided to the pen user on a CD-ROM, floppy disk, USB memory, etc. or via download from an Internet site. After being properly installed in a local device that communicates with the pen, the local device can provide the definition data kit, or selected portions thereof, to the pen. In yet another alternative, the pen, not good be connected to a dedicated server on the communication network. Importer module, pull out the definition data from there. It is also conceivable that the definition data is provided in a storage device that is removably installed in a pen accessed by the importer module. The storage device may take the form of any known type of card or cartridge, such as an SD card, CF card, SmartMedia® card, MMC, memory stick, etc. In all of these embodiments, the definition data is typically imported in a specific upgrade session that results in the definition data being stored in pen memory. As a result, the pen can be operated on all pattern pages that are imported and supported by stored definition data.

However, in one particular embodiment, the pen can be inferred such definition data from the recorded image.

As shown in FIG. 6, the product 600 may both are provided defining a position code 602 (which is enlarged by 602A) code 604 (which is enlarged by 604A). Definition code 604 codes definition data that may apply to a particular pattern page or series of such pages, eg, a book or shelf. The definition data suitably includes an application pattern page identifier, for example in the form of one or more page addresses.

In a typical user scenario, the user moves the pen across the definition code to enter an associated template definition or page description for a particular product. The definition code may be provided as an integral part of the product or as a separate product and may be entered into the pen either before, during or after reading the position data from the product .

Using the definition code, a template definition or page description can be added to the pen template database (511B in FIG. 5). Pen, the position database, segment, shelves, the pattern format data defining the size and location of a particular subset of the abstract pattern, such as a book or page, need only be stored in advance. All information about the placement and function of pidgets and user areas on the pattern pages in this subset may be included in the definition data that is imported.

When growing recognition of the user area, the pen can be operated in a more advanced control and / or user feedback. For example, the pen may signal that the location falls within a particular user area via activation of the MMI (220 in FIG. 2). In addition, the pen is instructed to shut off the output of the pen stroke on the pattern page until the pen records the pen stroke in a predetermined set of user areas on the pattern page, suitably by an instruction in the definition data. May be. Pen, it is also possible for the product of a single or multi-page format to verify that it is properly filled out.

Many different large-capacity codes can be used to code discontinuous definition data such as two-dimensional barcodes or matrix codes. Additional examples of suitable codes and methods for their decoding can be found in Applicant's US Provisional Application No. 60 / 582,861, filed June 28, 2004, or prior to Applicants' prior application. Published in US 2001/0038349 , US 2002/0000981.

In order to process the definition code, the pen system may be modified as illustrated in FIG. The image processing module 704, a recorded image can be processed into a stream of the stream or non-location data of the position. The stream of non-position data is analyzed by the translator module 706. The translator module 706 may identify non-position data as definition data and issue a definition data event . Importer module detects an event, to build a template definition or page description from the definition data, not good is implemented in order to store the results. In the template database (511B in Fig. 5), or a dedicated location database to store the page description in which is part page description database (511 of FIG. 5), there is a case associated page address (s ). The importer module may be configured to intermittently process the template database / page description database to free up memory capacity by eliminating records based on usage frequency, time since last use, and the like.

A translator module 706 may be implemented to match the positions recorded in both the template database and the page description database . The translator module 706 may output an event indicating the user area in addition to the pidget.

Returning now to level 5 refinement, the file assembly module (552 in FIG. 5) includes in the file object an identifier for each user region that is hit by a pen stroke on one or more pattern pages. Good. If the imported defined data detailing the functions associated with the user area, the pen system may be designed to handle pen strokes according imported defined data. In such a case, the file assembly module may incorporate the result of such processing into the file object.

The file assembly module may also include a physical layout definition, or a portion thereof , in a file object. Such a definition may include an electronic representation of the physical layout, i.e. a pointer to a storage location within the information management system where the electronic representation can be used.

In certain embodiments, in order to make the file object further platform independent, the pen system renders the physical layout as a collection of pen strokes, and these representations are used to represent the physical layout in the file object. It may be configured to incorporate a fictitious pen stroke. One same algorithm can be used to extract and recreate pen strokes and physical layout from file objects.

Additionally or alternatively, the file assembly module may process the definition data as independent data that is wholly or partially contained within the file object, or attached to the file object.

By using the definition data and the definition code, it is possible to simply and conveniently generate and use a board or product coded by the asynchronous information management system. The user may utilize a computer-based generation tool such as that disclosed in Applicant's previous publication US 2002/0040816 to create a suitable product. As shown in FIG. 8, the tool may be used to define both the physical layout and functional layout of the product (steps 800, 802). Next, the tool generates definition data representing at least the functional layout of the product (step 804) and defines a position code to be printed on the product (step 806). Thereafter, the position code, definition code and physical layout are rendered (steps 808 to 812) for application to the substrate (step 814) by the printer's print engine. All or part of the rendering may be accomplished in the tool or in the printer.

The printed definition data may represent the physical layout of the product either as a background picture or as a complete / original electronic representation of support graphics such as word processing documents, presentation documents, emails, etc. As another alternative, the physical layout may be defined in the form of pointers to information management system storage locations where the physical layout can be used in several forms. In the latter case, the generation tool prompts the user to select or specify a storage location before printing.

Alternatively or additionally, the pen system, the relay data may comprise importer module that is configured to import by automatically or instructions. Imported relay data, the recipient of interest in the information system, for example, the destination address of the associated application program receiving device that hosts (308 in FIG. 3) (304 in FIG. 3) explicitly identify and Good. The destination address may be indicated as a network address such as a URL (Uniform Resource Locator), an e-mail address, an IP (Internet Protocol) address, or the like.

By allowing the pen system to import explicit destination addresses, flow control in the information system may be simplified. In one example, the pen system may push the file object directly to the intended recipient device or application program. In another example, the pen system may use a file to allow any interceptor flow controller (eg, 306 in FIG. 3) to act directly on the address and relay the appropriate data to the intended recipient. An explicit destination address is included in the object. Accordingly, the flow controller is a need to access the look-up table is reduced.

Alternatively or additionally, the pen system may comprise importer module for use restriction data. This data sets limits on the pen system for the use of specific position data recorded by the pen system. For example, the data may define the maximum number of times position data can be recorded from a product, or the maximum number of times position data recorded can be transmitted from a pen. Alternatively, the data may define an expiration date during which the pen system is prohibited from subsequently recording location data from the product or transmitting the recorded location data. Using an expiration date may require that the pen timing circuit be synchronized with a common time reference in the information system.

Providing usage restriction data can control the use of products coded in the information system. Such control can be important for products that represent money, such as checks, evidence, gift certificates, tickets, and the like. Usage control can also be important in time-limited applications such as surveys, tests, polls, surveys, voting, games and gambling. Still further, by restricting the use of the product, the system provider will have to acquire a new product in order to replace the expired product or to make the expired product available again. Future revenue can be secured.

Alternatively or additionally, the pen system may include an importer module for security data. The importer module may be used by the pen system to achieve secure data transfer from the pen to the receiving device .

The security data may signal the pen system that the data must be encrypted. Further, if the encryptor module (528 in FIG. 5) supports different encryption methods, the security data may include an identifier that identifies the particular encryption method used to encrypt the data by the encryptor module.

Not all data in the file object need be encrypted . If the file object needs to be relayed by the flow controller (306 in FIG. 3), at least the relay data must be unencrypted to be able to access the flow controller. Similarly, other data in the file object that is required for decryption at the receiving device must be unencrypted as described below. In one embodiment, the imported security data, at least partially defines the data to be encrypted.

  The encryptor module ensures at least confidentiality (which remains secret while the information is transmitted), preferably authentication of the transmitted information (assuring the receiver that the identity of the sender of the information is not false) ) And / or non-repudiation (no information sender or recipient can deny transmission), either symmetric encryption or asymmetric encryption, or a combination thereof can be used. The integrity of the information (which has not been altered while the information is being transmitted) may suitably be ensured by using a predetermined hash function.

  Triple Data Encryption Standard (3-DES) Algorithm, Advanced Encryption Standard (AES) Algorithm, International Data Encryption Algorithm (IDEA), Diffie-Hellman (DH) Algorithm, and Rivest-Shamir-Adleman (RSA) Algorithm, etc. There are many available standard encryption algorithms that may be used. Additional suitable encryption algorithms and associated methods are disclosed in Applicants' published US 2002/0034300, US 2003/0021419, and US 2003/0074562, which are incorporated herein by reference.

The following describes some embodiments of the encryption process in the pen for illustrative purposes only.
In the first embodiment, pen system, either to extract the key from the imported security data, or, if it works a predetermined algorithm with a predetermined input item, withdrawing the encryption key by either. A predetermined algorithm, such as a hash function, may be preset or indicated in the pen system by security data. Similarly, the predetermined input items are pre-set to the pen system by the security data, or may be shown. That is, in one special embodiment, the pen system can retrieve a cryptographic key without using security data by operating a preset algorithm with a preset input item. Definition data or portions thereof described above may form such an input item.

Next, prior to exposing the file object, the pen system uses an encryption key to encrypt the portion of the data in the file object. Integrity of file objects, by calculating the hash value of the entire content of the file object prior to encryption, and may be guaranteed by including the hash value of that in the non-encrypted portion of the file object .

  In the first embodiment, the receiving device must be able to access the encryption key in order to be able to decrypt the encrypted part of the file object. Thus, the key may be stored on the receiving device. Alternatively, the receiving device may access a database that associates the encryption key with the page address. The aforementioned generation tool then interacts with this database so that appropriate associations are created therein for each coded product that is generated. In this way, when the file object is received, the receiving device extracts the page address (from the unencrypted portion of the file object), derives the appropriate encryption key from the database based on this page address, and encrypts the file object. An encryption key may be used to decrypt the encrypted part. To enhance security, the encryption key must be usable only for decrypting data received from a specific pen or a specific type of pen specified by a pen parameter. Furthermore, any well-known means for authentication and confidentiality may be provided in communication between the receiving device and the database.

In the second embodiment, the pen system extracts the public key of the target receiving device from the imported security data, extracts the random number from the random number generator in the pen, and publishes it to encrypt the random number. including Mel using the key, the encrypted random number in this way in the file object. Further, the pen system calculates a hash value of a random number based on a first predetermined algorithm, and uses this hash value as an encryption key for an encrypted portion of data in the file object. The integrity of the file object, based on the second predetermined algorithm, by calculating the hash value for the entire content of the file before the object of encryption, and the hash value to the unencrypted portion of the file object it may be guaranteed by including Mel.

Upon receiving the file object, receiving device, the encrypted random number extracted from the file object, using its private key to decrypt the encrypted random number, using a first predetermined algorithm The hash value of the random number may be calculated by using the hash value as an encryption key for decrypting the encrypted portion of the file object.

In some embodiments, relay data and / or usage restriction data and / or security data are imported, usually in a preparatory upgrade session, via a pen communication interface or from a storage device plugged into the pen. . In another embodiment, this input data is imported by a pen system that identifies and decodes machine-readable code in one or more images recorded by the pen's primary image sensor.

In this way, the relay data, usage restriction data, and security data may be embedded on the product as one or more machine-readable codes , similar to the definition code 604 of FIG. Such a product may be generated by a computer-based generation tool based on user input, similar to the definition coding product. The user can input the destination address of the receiving device or application program that is encoded as relay data in the product. Similarly, the user may enter the aforementioned maximum number or expiration date. Similarly, the user may enter security data for use by the pen system's encryptor module in encrypting at least a portion of the data exposed in the information system.

  Relay data, usage restriction data, and security data may be applied to a particular pattern page or a series of such pages. Thus, this input data suitably includes the identifier of the corresponding pattern page, for example in the form of one or more page addresses.

  Even if the definition data, relay data, usage restriction data, and security data are encoded in the same machine-readable code or imported in an upgrade session over the communication interface, each has a different page unit hierarchy It should be clear that it is applicable to the level.

In one embodiment, the importer module stores the imported input data (definition data, relay data, usage restriction data, and security data) in a pen memory hierarchical data structure and is stored at one level of the data structure. Data is applicable to both this level and any subordinate levels. The data structure typically reflects the page unit hierarchy (FIG. 1) so that each subordinate level represents a subset of page unit groups associated with its higher level. In this way, the importer module, based on the relevant page address, store any items of input data to the appropriate level of the hierarchy. The pen system may only accumulate input data stored for the associated page unit group at each level of the hierarchy between the top level and a particular page unit . Therefore, such storage may provide efficiency when finding all imported input data used in the processing of the position data from a particular page units. For example, the security data associated with a particular segment is stored in the segment level associated with a particular segment, use restriction data associated with a particular shelf of this segment is stored in the shelf level associated with a particular shelf, relaying data associated with a particular book this shelf is stored in the workbook level associated with a particular book, defined data associated with a particular page of this book is stored in the page level associated with a particular page . In this example, the pen system processes the location belonging to a specific page unit finds the definition data associated with the page level, find the relevant relay data in the book level, find use restriction data associated with the shelf level, Then, relevant security data is found at the segment level.

  The invention has mainly been described above with reference to several embodiments. However, as will be readily appreciated by those skilled in the art, embodiments other than those disclosed above are equally possible within the scope and spirit of the invention as defined and limited only by the appended claims. is there.

  For example, an event may be generated in the pen system by a user pressing a button on the pen or issuing a verbal command recorded by a microphone in the pen. The trigger pidget event may be replaced or supplemented by such an event. Similarly, content and / or context information may be indicated to the pen system by such alternative events.

  It is conceivable that the pen system automatically creates a file object each time a pen stroke is recorded. Each file object may be created to contain a stroke for each pattern page or book. The file object may then be exposed to the external data handler either by command or automatically. Automatic exposure may include exposing the file system in the pen memory via the communication interface.

In the above embodiment, the division of abstract pattern is dynamic in that the logical position is calculated on the basis of the location database in the pen. In alternative embodiments, abstract pattern may be static by being encoded in the pattern. For example, US Pat. No. 6,330,976 discloses a coding pattern in which coding cells are tiled on the product surface, each cell coding both a local location and a page identifier. By decoding the pattern, the pen can thus directly detect its logical position.

Also, the pen, accelerometer, roller ball, may include supplemental device for relative positioning of such triangulation device. In this way , the pen can supplement the absolute position derived from the position code with the relative position indicated by the supplement device. In this case, the position code requires only to code some of the absolute position on the product.

Furthermore, the pen system may be based on the functionality of the distribution between the different software modules. That is, the pen system software need not be configured from individual modules.

  Still further, certain aspects of the present invention are not limited for use with file objects as described herein. In particular, <16. The importer importer functionality may also be incorporated into a pen system designed for end-to-end communication with lookup servers and application servers, as described in the introduction.

It is a diagram depicting a logical division into addressable pages units abstract position-coding pattern. 1 is a cross-sectional view of an electronic pen that implements the principles of the present invention. 1 is a diagram illustrating an overview of an information management system and communication paths in which file objects may be generated in accordance with the present invention and exposed for transfer. FIG. 10 is a diagram depicting the relationship between template definition and page description for a finally position-coded product. FIG. 3 depicts an embodiment of a system of software modules for controlling the operation of the pen of FIG. FIG. 5 is a diagram of position-coded products with definition codes that hold metadata related to the products themselves, and the processing of the positions recorded on the products. FIG. 7 depicts a portion of a pen control system for processing data from the product of FIG. FIG. 7 depicts the steps achieved by the tool for generation of the position- coded product of FIG.

Claims (63)

In an electronic pen used in an information management system including a plurality of external data handlers, a method for managing digitally recorded position data,
The method
Processing to incorporate the position data into a file object suitable for storage in a file system asynchronously with a component other than the electronic pen in the information management system ;
Presenting the location data to at least one of the plurality of external data handlers by pushing the file object to an external data handler or making it available for pulling from an external data handler;
Only including,
The method wherein the file object incorporating the position data is created without communication between the electronic pen and the external data handler before presentation of the position data to the external data handler . Mapping the position data to a position database in the electronic pen;
The method of claim 1, wherein the location database associates location data with processing rules.   The method of claim 2, wherein incorporating the location data is initiated when a dedicated processing rule is identified in the location database based on the recorded location data.   The method of claim 3, further comprising incorporating a rule identifier indicating the dedicated processing rule into the file object.   The method according to claim 2 or 3, further comprising incorporating into the file object a rule identifier indicating each processing rule identified during the mapping.   The method according to claim 4 or 5, further comprising including a time identifier for each rule identifier in the file object. Identifying at least one page unit identifier and a set of location items in the location data;
Grouping at least some of the position items into temporally coherent stroke items and associating each stroke item with its page unit identifier;
Incorporating the stroke item and the page unit identifier into the file object;
The method according to claim 1, further comprising:   At least one of a time identifier for each stroke item, a pen resident parameter value indicating the characteristics of the electronic pen, and a pen resident parameter value indicating the characteristics of the electronic pen owner is stored in the file object. 8. The method of claim 7, further comprising incorporating.   9. A method according to claim 7 or 8, wherein incorporating the stroke items comprises grouping the stroke items by a page unit identifier. Incorporating the location data is initiated when a dedicated processing rule is identified in the location database based on the recorded location data;
The method
Identifying at least one page unit identifier from the dedicated processing rule;
The method of any one of claims 7 to 9, further comprising incorporating a stroke item associated with the identified page unit identifier into the file object.   The method according to any one of claims 7 to 10, wherein each page unit identifier is a global address for that particular page unit in a coding pattern.   The method of claim 11, wherein the location item is a local location within the particular page unit.   The method according to claim 11 or 12, wherein the position data is recorded by the electronic pen from the coding pattern. Mapping the location data to a location database in the electronic pen, the location database associating location data with processing rules;
The position database associates each page unit with a template that defines a functional area in the page unit, and each functional area indicates one of a plurality of the processing rules.
The mapping is
Identifying one of the plurality of page units based on the position data;
Identifying the template associated with the identified page unit;
Identifying at least one functional area by mapping the location item to the identified template;
The method according to claim 11, comprising: Retrieving processing rules from the location database based on the location data;
Applying the retrieved processing rules to at least some of the location data to generate dynamic parameter values;
The method according to any one of claims 2 to 6, further comprising:   The method of claim 15, further comprising incorporating the dynamic parameter into the file object.   The method according to claim 15 or 16, wherein the processing rule includes an instruction of a character recognition process, and the dynamic parameter value is a text item.   18. A method as claimed in any preceding claim, wherein the file object is a data storage object that has no processing instructions and no calls to processing instructions. Detecting that the working session of the electronic pen has changed;
Assigning a unique session identifier to the current working session;
The method according to any one of claims 1 to 18, further comprising:   The method of claim 19, further comprising incorporating the session identifier into the file object.   21. The method according to any one of claims 1 to 20, further comprising retrieving a text item stored in a memory of the electronic pen from a text database based on the position data.   The method of claim 21, further comprising incorporating the text item into the file object.   The method according to claim 21 or 22, wherein the text item is an address for electronic communication.   23. The method of claim 21 or 22, wherein the text database includes keyboard definitions that respectively associate a plurality of sets of location data with a plurality of characters, and wherein the text item includes at least one of the plurality of characters. The position data is a first type of position data representing graphic information reflecting the movement of the electronic pen, and a second representing processing rules applied to the first type of position data outside the electronic pen. Type position data, and
The method
Incorporating the first type of position data into the file object as a temporally coherent stroke item;
Including a time identifier for each stroke item in the file object;
For each processing rule identified from the second type of location data, incorporating a rule identifier into the file object;
For each rule identifier, including a time identifier in the file object;
The method according to any one of claims 1 to 24, further comprising:   26. The method of claim 25, wherein the processing rules are applied to visual graphic characteristics, temporal or spatial resolution, background images, interpretation context, or restricted access display.   27. A method as claimed in any preceding claim, further comprising incorporating non-position data into the file object.   The non-position data is received on the interface for external communication by the electronic pen, barcode data recorded by the electronic pen, voice data recorded by the electronic pen, image data recorded by the electronic pen 28. The method of claim 27, comprising: image data to be processed, character data retrieved by the electronic pen, or character data received on an interface for external communication by the electronic pen. Receiving character data on an interface for external communication;
Converting the character data into position data;
Incorporating the converted character data into the file object;
The method according to any one of claims 1 to 28, further comprising:   30. A method as claimed in any preceding claim, wherein presenting the location data to the external data handler comprises sending the file object to the external data handler. Presenting the position data to the external data handler includes storing the file object in a file system in the electronic pen;
30. A method as claimed in any preceding claim, wherein the file system is accessible to the external data handler. Identifying the selection of location data to be presented;
In accordance with the selection, extracting previously recorded position data from the memory of the electronic pen;
Further including
The method according to any one of claims 1 to 31, wherein the extracted position data is embedded in the file object. Converting the record of the first code on the product into at least one absolute position in the coordinate system;
Converting the record of the second code into input data;
Processing the at least one absolute position based on the input data;
The method according to any one of claims 1 to 32, further comprising:   34. The method of claim 33, wherein the first code record includes an image of a surface portion of the product. The input data defines at least one functional area in the coordinate system;
The method
35. The method according to claim 33 or 34, further comprising: mapping the at least one absolute position to the input data to determine whether the at least one absolute position falls within the range of the at least one functional area. Method.   36. The method of claim 35, further comprising retrieving processing rules for the at least one functional area based on the input data.   37. The method of claim 36, further comprising applying the processing rule to the at least one absolute position included in the functional area.   38. The method according to any one of claims 35 to 37, further comprising operating an MMI of the electronic pen based on specifying the at least one absolute position included in the range of the at least one functional area. The first code defines a unique identifier of the product;
The method
Retrieving the unique identifier from the record of the first code;
Comparing the retrieved identifier with a corresponding identifier included in the input data;
39. The method according to any one of claims 33 to 38, comprising: The input data includes usage restriction data;
40. The method according to any one of claims 33 to 39, wherein the method further comprises restricting processing of the at least one absolute position based on the use restriction data.   The usage restriction data includes a time limit after which the process is prohibited, a first event limit that defines a maximum allowable number of activations of the process, and a maximum allowable activation number for converting the recording of the first code. 41. The method of claim 40, wherein the method indicates at least one of second event deadlines to be defined.   The method according to any one of claims 33 to 41, wherein the input data includes security data used when encrypting the at least one absolute position.   43. The method of claim 42, wherein the security data indicates at least one of an encryption key and an encryption method. 44. A computer program comprising program code that can be loaded into a memory of a processor and that executes the method according to any one of claims 1-43.   An electronic pen comprising a processor configured to perform the method of any one of claims 1-43. Used for writing on the product, an electronic pen that is used by the information management system including a plurality of external data handler,
Means for recording at least one image of the product during the writing;
Means for determining position data indicating the writing based on the at least one image;
Means for performing processing for incorporating the position data into a file object suitable for storage in a file system asynchronously with a component other than the electronic pen in the information management system ;
Means for presenting the location data to at least one of the plurality of external data handlers by pushing the file object to an external data handler or making it available for pulling from an external data handler;
Equipped with a,
Electronic pen to the outside the said electronic pen before presentation to the data handler without external data handler to perform communication, wherein Rukoto the file object that is built with the position data is created in the position data .   The product is provided with at least one subset of a position coding pattern that encodes a continuous sequence of positions in a two-dimensional coordinate system, and the position data can be extracted from the position coding pattern. Item 47. The electronic pen according to Item 46. A memory for storing a position database defining a plurality of page units in the two-dimensional coordinate system;
Processing means for mapping the position data to the position database;
The electronic pen according to claim 47, further comprising: The processing means identifies a page unit identifier indicating one of the plurality of page units by mapping the position data to the position database,
49. The electronic pen of claim 48, wherein means for incorporating the location data is controlled to incorporate the page unit identifier into the file object. The processing means identifies a set of position items belonging to the one page unit based on the position database;
50. The electronic pen of claim 49, wherein the means for incorporating the location data is controlled to incorporate the set of location items into the file object in association with the page unit identifier. The position database associates each page unit with a template that defines a functional area in the page unit, and each functional area indicates a processing rule;
The processing means specifies one of the plurality of page units based on the position data, specifies the template associated with the specified page unit, and adds the position item to the specified template. 51. The electronic pen according to claim 50, wherein at least one functional area is specified by mapping. The means for determining the position data is operable to determine the position data based on at least one image of a first code on the product;
The position data includes at least one absolute position of a coordinate system;
The position data determining means is operable to determine input data based on at least one image of the second code;
52. The electronic pen according to any one of claims 46 to 51, wherein the processing means is operable to process the at least one absolute position based on the input data. The input data defines at least one functional area in the coordinate system;
53. The method according to claim 52, wherein the processing means maps the at least one absolute position to the input data and specifies whether the at least one absolute position is included in a range of the at least one functional area. Electronic pen.   54. The electronic pen according to claim 53, wherein the processing means extracts a processing rule for the at least one functional area based on the input data.   55. The electronic pen according to claim 54, wherein the processing means applies the processing rule to the at least one absolute value included in the functional area. The input data includes usage restriction data;
56. The electronic pen according to any one of claims 52 to 55, wherein the processing means limits processing of the at least one absolute position based on the use restriction data. The input data includes security data;
57. The electronic pen according to any one of claims 52 to 56, wherein the electronic pen further comprises an encryptor operable to encrypt the at least one absolute position based on the security data.   58. The electronic pen according to any one of claims 46 to 57, wherein the means for incorporating position data includes an event listener that starts incorporating the position data into the file object upon receipt of a trigger event.   59. The electronic pen according to any one of claims 46 to 58, wherein the means for presenting the position data to the external data handler includes an event listener that starts presenting the position data when a trigger event is received.   60. The electronic pen according to claim 58 or 59, wherein the processing means generates the trigger event based on the position data. A voice sensor operably connected to the processing means;
60. The electronic pen according to claim 58 or 59, wherein the processing means generates the trigger event based on a verbal command recorded by the voice sensor. A trigger button operably connected to the processing means;
60. The electronic pen according to claim 58 or 59, wherein the processing means generates the trigger event based on detection of a state change of the trigger button. A product comprising at least one subset of the position-coding pattern;
An electronic pen according to any one of claims 45 to 62;
An application processor for obtaining the file object from the external data handler and processing the location data;
An information management system comprising:

Images