SE516739C2 - Notepad for information management system, has activation icon that enables a position code detector to initiate a predetermined operation that utilizes the recorded information - Google Patents

Abstract

The notepad (1) has a writing surface (3) which is provided with a position code. The position code enable electronic recording of information which is being written on the writing surface. An activation icon (7a-7g) enables a position code detector to initiate a predetermined operation that utilizes the recorded information. Independent claims are also included for the following: (a) a device for information management; (b) a computer program for information management; (c) a system for information management; (d) a method for recording and processing information; (e) and a handheld electronic device.

Description

lO 15 20 25 30 516 739 2 with which the user can write or draw on the writing surface.

The pen further has a light source for illuminating the position code and a CCD sensor for receiving the light reflected from the position code. The position information received by the CCD sensor is sent to a computer for processing. This device seems to require a computer to be started and a certain program to be opened before writing on the writing surface can begin.

SUMMARY OF THE INVENTION An object of the present invention is to further simplify the handling of handwritten information.

This object is achieved in whole or in part with a product according to claim 1, a device according to claim 9, a method according to claim 21 and a hand-held electronic device according to claim 22.

More particularly, according to a first aspect, the invention relates to a product having at least one writing surface provided with a position code, which encodes a plurality of positions on the surface for enabling electronic recording of information written on the writing surface, by means of a device which detects the position code. The product further has at least one activation icon which is provided with the position code and which, when detected by the device, causes the device to initiate a predetermined operation which utilizes the information registered by the device.

The product thus has not only a writing surface but also an activation icon with which the user can control the device to process the registered information in a predetermined manner. As a result, the user does not have to press keys or buttons on the device itself, but instead uses the activation icon on the product as a means of controlling the device. You can liken the activation icon to an icon that appears on a computer screen, but instead of placing the computer screen cursor on the icon and clicking with the mouse, the user now gives his command by placing the device so that it can detect the activation icon on the product. The product thus constitutes a new form of user interface for handling information written on a writing surface.

The activation icon can be designed as an image, a symbol, text, numbers or the like, which enables a user to realize and remember which operation is initiated when she uses the icon. The detection of the icon by means of the device is based on a position code which the device detects.

The written information that is registered here is all information that can be provided by hand, such as text, numbers, different types of characters, figures and drawings.

The product thus has, as mentioned, at least one activation icon. However, in a preferred embodiment, the product has a plurality of activation icons for activating various predetermined operations. The user can thus choose between different operations that use the registered information.

The icon and position code can be designed in many different ways. They can be, for example, electrically, chemically or mechanically detectable. They do not have to be detectable with the same principle. In a preferred embodiment, however, both the icon and the position code are optically detectable. This makes them easy to apply to a product because they can be printed on it, for example, and furthermore a device with one or more optical sensors can be used for the detection of the icon and the position code.

The activation icons and position code are passive in this version. They just need to be able to reflect light.

However, the light does not have to be in the visible area.

The activation icon can activate the device to initiate in principle any conceivable operation that utilizes the registered information. The selection is really only limited by the functions of the device that is to use the product. The product may have a standard set of activation icons. It is not necessary that all devices can use all activation icons. In a preferred embodiment, the predetermined operation is an operation from the group: to dial a telephone number included in the information, to send a fax with text included in the information, to send an electronic message with text included in the information, to enter a address information included in the information in an electronic address book, to enter a calendar entry included in the information in an electronic calendar and to enter an entry in an electronic list.

The position code can be of different types. Examples of various known position codes encoding each position with a symbol or symbol group are found in the aforementioned US 5,852,434 and in US 5,051,736. The position code may preferably be of a type which encodes each position with a plurality of symbols, each symbol contributing to the coding of more than one position. Examples of how this type of overlapping or floating code can be realized and detected can be found in the applicant's previous application no. SE 9901954-9 which was submitted on 28 May 1999 and which is incorporated in this application by this reference. The advantage of this position code is that it provides good resolution and is easy to detect because the individual symbols are uncomplicated.

The product can be any product that has a writing surface and at least one activation icon. It can consist of two physical parts, the writing surface with the position code being on one and the activation icon on the other.

Alternatively, it may consist of a continuous part on which both the activation icon and the writing surface are located. The activation icon can then be found on the writing surface or on some other surface. The product can be, for example, a paper with a writing surface with a position code on one part of the paper and an activation icon on another part of the paper. In a preferred embodiment, the product is a notebook with a plurality of writing surfaces.

The invention further relates according to a second aspect to a device for information handling, which device is arranged to electronically register and process information written on a writing surface. The device is further arranged to detect a predetermined activation icon by means of a position code which the activation icon is provided with and to initiate a predetermined operation, which utilizes the electronically registered information, when it detects the activation icon.

An advantage of this device is that it is easy to use because the user does not have to learn different commands and does not have to press different buttons to open a desired program.

As mentioned, the device will at least initiate the predetermined operation when the icon is detected. Depending on the functions available in the device, it can in some cases perform the entire operation. In other cases, the device may, for example, automatically transmit the registered information and information on which operation is to be performed to an external device, for example a PC or mobile telephone, which completes the operation. This transfer can take place directly or at a later time.

By "initialize" is meant here that the device ensures that the operation is performed, even if it does not perform the operation itself, so that the user does not have to give further command to the device or the external device for the operation to be performed. However, the user may need to add additional information and or confirm the operation / information.

The device can register information in different ways. It may include a sensor, such as an accelerometer, which detects the movement of the device when the device is used to write on a writing surface. It may further include a scanner, which scans in the information. In a preferred embodiment, however, the device is arranged to register the information by detecting a position code present on a writing surface on which the information is written by hand. This embodiment is advantageous because the information is registered at the same time as it is written and the same sensor can be used both for registering the written information and for detecting the activation icon. In this embodiment, the position code is continuously detected while the information is being written, whereby a sequence of position indications is obtained which defines how the device has been moved.

The device can, as stated above, use different types of sensors depending on the technology with which the position code and the activation icon are realized. In a preferred embodiment, the device comprises an optical sensor for detecting the activation icon. The device further advantageously comprises an optical sensor which is arranged to record images of the writing surface and a signal processor which is arranged to utilize the position code in the images to provide a digital representation of the information, the predetermined operation performed on the digital representation of the information.

The optical sensors are advantageous in that they enable the information registration and initiation of the predetermined operation to be based on image processing, which is a well-known and well-developed technology.

The device itself can be realized with two sensors of the same or different type, but the device will be cheaper and easier to use if it is realized with a sensor for both registration of information and for detection of the activation icon.

According to a third aspect of the invention, it relates to a method of recording and processing information, comprising the steps of writing information on a writing surface with a pen; record the written information electronically with a device; and causing the device to initiate a predetermined operation of the recorded information by allowing the device to detect an activation icon provided with a position code.

This way enables the user to handle information in a very simple way. She can record information as usual with a device that includes a pen on a piece of paper, but still immediately initiate one or more operations that normally require the start of a particular computer program and subsequent entry of the information into a computer, simply by letting the device detect an activation icon. . It is not necessary according to the invention that the information is written before the activation icon is detected. It is also conceivable to have the device detect the activation icon before writing the note. The technology described above could also be used for general control of handheld computers and other handheld electronic devices, such as mobile phones, PDAs, pens and handheld scanners.

Such devices are controlled by the user, using one or a few buttons, selecting from menus or between icons that appear on the computer display. These devices could be controlled in a simpler way by utilizing activation icons that activate various predetermined operations, which are normally achieved with the push of a button.

To that end, according to a further aspect, the invention relates to a hand-held electronic device which is arranged to perform predetermined operations on the command of a user. This device has an optical sensor for registering at least one activation icon, and a signal processor which is arranged to, in response to the registration of the activation icon, perform one of said predetermined operations. The optical sensor is further arranged to register the activation icon by means of a position code with which the activation icon is provided.

The device may in particular be a C Pen type pen sold by the applicant. Such a reading pen is described, for example, in SE 9604008-4. This device is controlled by means of buttons and menus shown on the display. Control could be simplified by using activation icons.

The activation icon or icons can be found, for example, on a product of the mouse pad type, on a piece of paper or on another product that the user can easily access. The predetermined operation can be any operation that a user normally commands a handheld electronic device to perform using a mouse, buttons or keys on a keyboard.

Examples of such operations are starting or closing a program, and saving or discarding information that has been registered.

What has been said above about the device for information handling naturally applies in applicable parts here as well.

Brief Description of the Figures The present invention will now be described in more detail by way of an embodiment with reference to the accompanying drawings, in which Fig. 1 schematically shows an embodiment of a product according to the invention in the form of a notepad; Fig. 2 schematically shows an embodiment of a device according to the invention.

Detailed Description of a Preferred Embodiment Fig. 1 shows a notepad 1 with a plurality of sheets 2 of paper. The top sheet has a writing surface 3 and a command bar 4.

A position code 5 is printed on the writing surface 3, which is made up of symbols 6 of a first and a second type 6a, 6b and more particularly a dot with a larger radius and a dot with a smaller radius, which corresponds to a "one" and a "zero". For the sake of clarity, the symbols have been enlarged and the position code is displayed on only a small part of the writing surface. In a real embodiment, the position code extends over the entire writing surface and the symbols are smaller so that a better position resolution is obtained.

The position code can be obtained in a number of different ways.

A method of providing a position code where each position is encoded with a symbol is described in US 5,852,434. In the present invention, however, each position is advantageously coded by means of a plurality of symbols, and the coding is such that each symbol in the position code contributes to the coding of more than one position. This means that two adjacent positions have partially common symbols, as shown in Fig. 1 with the dashed areas Sa, 5b. In this way a higher resolution is achieved and the detection becomes easier because the individual symbols can be less complex. This type of overlapping or floating position code can be accomplished using a computer. A special way of generating such a floating position code is described in the applicant's earlier patent application No. 9901954-9 which was filed on May 28, 1999 and which is incorporated into this application by this reference.

In the command field 4, seven different activation icons 7a-g are printed which can be used to control a device for initiating or performing various operations.

The icons 7a-g are so graphically designed that the user understands which operation is activated by the icon. The icons in this example contain images, but they can also contain text and numbers.

Fig. 2 shows a device that can be used to write text and draw figures on the writing surface and to perform operations triggered by the activation icons.

The device comprises a housing 11, which is shaped approximately like a pencil. There is an opening 12 in the short end of the housing. The short end is intended to abut or be kept at a small distance from the writing surface and the command field.

The housing mainly houses an optics part, an electronics part and a power supply. The optical part comprises at least one LED 13 for illuminating the surface to be imaged and a light-sensitive area sensor 14, for example a CCD or CMOS sensor, for detecting a two-dimensional image. The device may also contain a lens system (not shown). 10 l5 20 25 30 516 759 ll The power supply to the device is obtained from a battery 15 which is mounted in a separate compartment in the housing.

The electronics part contains a signal processor 16.

The signal processor is realized with the aid of a microprocessor and it contains software for analyzing images from the sensor for obtaining a digital representation of what is written on the writing surface and for detecting the activation icons. It also contains software for initiating or performing the operations defined by the activation icons. Finally, it can also contain user software, such as address book software for the user to keep track of address data, calendar software for the user to keep track of calendar data, such as meetings, birthdays and other data entered in a calendar or calendar, as well as a to-do list program for the user to be able to enter tasks that she is to perform.

The signal processor also advantageously includes an ICR software that can be used to interpret registered characters so that they can be stored, handled and / or forwarded in character-coded format.

Finally, the electronics part also comprises a mobile telephone unit comprising a mobile telephone transceiver 17, which enables, for example, telephoning, sending faxes, and sending e-mail messages to an external unit in the form of a telephone, a fax machine or a computer.

On the housing of the device there is also a pen tip 18, with the aid of which the user can write ordinary dye-based writing on the writing surface. The pen tip 18 can be folded in and out so that the user can control whether it is to be used or not. The device further comprises buttons 19 by means of which the device can be activated and controlled for operations which are not controlled by the activation icons. It also has a transceiver 20 for wireless transmission, for example with IR light or radio waves, of information over short distances.

This wireless transmission can be used when the device cannot perform the entire initiated operation itself but needs to pass on the registered information and information on which operation is to be performed to an external unit.

The device finally comprises a display 21 for displaying, for example, registered information.

The applicant's Swedish patent no. 9604008-4 describes a device for registering text. This device can, if properly programmed, be used to record information written on a writing surface with position code, to detect activation icons and to initiate / perform operations activated by the activation icons.

In the following, the function of the notepad in Fig. 1 and the device in Fig. 2 are described.

Suppose a user receives a phone call. She writes her notes on notebook 1 using the device's pen tip 18. Suppose, for example, that the caller wants the user to call Kalle on a certain telephone number. The user then writes "Call Kalle 9857299" on notepad 1.

While the user is writing, the optical sensor 14 continuously takes pictures of the writing surface 3 in the current position of the pen. The images contain the position code 5.

The signal processor 16 locates the position code in each image and determines from this the coordinates of the absolute position on the writing surface in which each image is taken. In this way, a description is obtained, in the form of a large number of position indications, of how the pen is moved over the paper. This description is a digital representation of the written information. The position indications can, if necessary, be fed as an input to the ICR software, which determines which characters the positions correspond to, so that the information can be stored in character-coded format instead of as a result of position indications.

When the user has finished their phone call, she puts the device on the top icon that looks like a phone. When the device detects the telephone icon, it starts a program for dialing a telephone number using the mobile telephone transceiver 17. The dialer retrieves the numbers of the telephone number directly from the registered information. The program shows the numbers on the display 21 and waits for the user's confirmation at the touch of a button that the number is correct and that the call is to take place. When this has happened, the number is dialed automatically and the user can talk to Kalle.

The detection of the activation icon can be done in different ways. As an example, it can be done on the basis of the same information that the user uses to decide which icon she should use, ie on the image or text content in the command field 4. As another example, the position code can extend over the command field, whereby the icons are superimposed on the position code or vice versa. The position code will then also be in the image of the icon. Since a certain icon is always in a certain place, the icon detection can then be based on the detection of a certain position. As a further alternative, the icon may be provided with another code, for example a bar code, or symbols or other features which make it possible to identify the activation icon.

Another example of how the notebook and pen can be used is as follows. The user decides that she should pick up the car at 4 pm at the workshop. She then writes a note on her pad "pick up the car at 16". This information is registered in the pen using the position code. The user then places the pen on the calendar program icon 7e, the pen detecting it and activating the calendar program. The program retrieves the registered information. If it is in the form of position indications, it can of course only be stored as position indications in the calendar program. The program can then suggest that the note be entered on the date of the day and ask the user to confirm this with a push of a button on the pen. If, on the other hand, the information is available in character-coded format, the program can interpret "kl" as the time and enter the note on this time and set an alarm. Other activation icons can be used in a corresponding way.

The fax icon 7b can be used if the user wants to send a fax. The user then writes his fax on the writing surface and activates the fax program using the icon 7b. The fax program asks where the fax is to be sent.

Information on this can be retrieved by the user from an electronic address book in the pen or written by hand with the pen on the paper. The fax program then automatically composes the fax and sends it via the mobile telephone transceiver 17 to the intended recipient.

E-mails can be composed and sent in a corresponding manner by activating the e-mail icon 7c.

Data for an address book and a to-do list can be handled in the same way as the data for the calendar program by means of the address book icon 7d and the to-do list icon 7f.

Finally, some notes can be deleted from the pen memory using the trash can icon 7g.

In another embodiment, the device in Fig. 2 can be controlled by means of activation icons which may be designed in the same way as on the notepad in Fig. 1, but refer to operations of the open and close program type and other operations which the user normally commands via the buttons 19 on the device.

Claims (22)

10 15 20 25 30 35 516 739 U 'v. 16 PATENT REQUIREMENTS 1. l. Product having at least one writing surface (3) that is pre- (5), for enabling electronic viewing with a position code encoding a plurality of positions on the writing surface (3) registration of information written on the writing surface, by means of a device which detects the position code, characterized in that the product further has at least one activation icon (7a-g) which is provided with the position code (5) and which, when detected by the device, causes the device to initiate a predetermined operation which utilizes the information recorded by the device. A product according to claim 1, wherein said at least one activation icon is detectable by means of the position code (5) A product according to claim 1 or 2, wherein the product as a predetermined position on the product. includes a plurality of activation icons (7a-g) for activating various predetermined operations. A product according to claim 1, 2 or 3, and the position code (5) wherein said at least one activation icon (7a-g) is optically detectable. A product according to any one of the preceding claims, wherein said at least one activation icon (7a-g) and the position code (5) are detectable by one and the same sensor. A product according to any one of the preceding claims, wherein the predetermined operation is an operation from the group: to call a telephone number included in the information, to send a fax with text included in that information, to send an electronic message with text included in the information, to enter an address entry included in the information in an electronic address book, to enter a calendar entry included in the information in an electronic calendar and to enter an entry included in the information in an electronic list. nu u 10 15 20 25 30 35 516 739 4; »,« <~, 1 .. <- 17 A product according to any one of the preceding claims, wherein the position code comprises a plurality of symbols (6) and wherein each symbol contributes to the coding of more than one position. A product according to any one of the preceding claims, which product is a notepad with a plurality of writing surfaces (3). An information handling apparatus, the apparatus for electronically registering and (3), the apparatus further processing information written on a writing surface characterized by the apparatus for detecting a predetermined activation icon by means of a position code provided with the activation icon and to initiate a predetermined operation, which utilizes the electronically registered information, when it detects the activation icon (7a-g). Device according to claim 9, wherein the device is arranged to electronically register the information written by hand on the writing surface (3) by registering a position code (5) present on the writing surface (3). 11. ll. Device according to claim 9 or 10, which device comprises at least one sensor (14) for the electronic registration of the information written on the writing surface (3) and of the activation icon (7a-g). Device according to claim 10 or 11, wherein the device comprises a single sensor (14) for the registration of the information and the activation icon, which sensor is arranged to effect the registration by registering a position code (5) to which the writing surface and the activation icon are provided. with. Device according to any one of claims 9-12, wherein the device comprises an optical sensor which is arranged to register the activation icon. Device according to any one of claims 9-13, wherein the device comprises an optical sensor, which is arranged to register images of the writing surface and a signal processor which is arranged to use the position code in the car. | 1. 10 15 20 25 30 35 516 739 ~ - -. . ». . . 18 to provide a digital representation of the information, the digital representation of the information being used in the predetermined operation. The apparatus of claim 14, wherein the signal processor comprises a character interpretation function arranged to transmit the recorded information to character encoded format, so that the digital representation of the information has at least partially character encoded format. The device of any of claims 9-15, further comprising a mobile telephone transceiver for transmitting the recorded information from the device to an external device, the predetermined operation being an operation of the group: dialing a telephone number included in the information, sending a fax with text included in the information, and to send an electronic message with text included in the information. whereby Device according to any one of claims 9-16, the device comprises a transceiver (20) for wireless short-range communication. Device according to any one of claims 9-17, wherein the device comprises at least one computer program of the type address book program or calendar program or task list program, wherein the predetermined operation consists of entering a task contained in the registered information in a register for use in any of said computer programs. Device according to any one of claims 9-18, which device is hand-held. Device according to any one of claims 9-19, further comprising a pen tip (18) for writing the information on the writing surface (3). A method of recording and processing information, including the steps of writing information on a writing surface with a pen; record the written information electronically with a device; k n a n e t e c k n a t of the step of causing the device to perform a predetermined. . ». ». 10. . <. x. 516 739 19 operation for the registered information by letting the device detect an activation icon provided with a position code. A handheld electronic device arranged to perform predetermined operations at the command of a user, comprising an optical sensor for recording at least one activation icon, and a signal processor arranged to perform one of said predetermined operations in response to the registration of the activation icon. operations, characterized in that the optical sensor is arranged to register the activation icon by means of a position code with which the activation icon is provided.