GB2354824A

GB2354824A - Paper marking pen with motion sensors and wireless link to a computer

Info

Publication number: GB2354824A
Application number: GB9912097A
Authority: GB
Inventors: Brian Douglas Smith; William Harris Knight; David Michael Malcolm
Original assignee: Pankhurst Design & Development
Current assignee: Pankhurst Design & Development
Priority date: 1999-05-25
Filing date: 1999-05-25
Publication date: 2001-04-04
Also published as: GB9912097D0

Abstract

A hand-held implement, such as a pen, for producing a real and virtual mark on a surface. The pen nib may produce ink marks by use of a roller ball (fig 2) or fibrous tip (fig 3 shown here), and in addition includes sensing means for producing an electric signal in response to movement of the pen nib, said signal representing the mark made by the pen. The sensing means may detect rotational movement (fig 2) or directional strains (fig 3). Preferably, the electrical signal representing the mark is transmitted as a wireless signal to data processing means which can compare the mark made with predetermined marks. The data processing means can understand commands and can perform character recognition possibly for recording and sending e-mail messages.

Description

2354824 PATENT APPLICATION State of the Art Recent years have seen

considerable progress in a number of areas of computing. Two specific areas, wireless communications and body worn computing, particularly interested the applicant.

Overview The purpose of this invention is to provide a wireless means of communication between a pen and a computer, enabling hand - written text, sketches and drawings to be recorded both on a normal piece of paper in ink and communicated via the computer through the medium of email or similar whilst being saved at the same time as an electronic copy on the computer.

Another aspect of the previously described invention provides for the establishment of the working plane to be written upon, by identifying to the receiving computer at least three points in a single plane in space by identifying the extents of the surface area to be written on by electronic signal. This aspect of the application is not limited to the specific context of the invention.

Description of illustrations

Illustration 1 shows the configuration of the pen and includes the writing tip (1) an LED showing the pens status (2) the transmitter to the computer (3) a movement detector (4) an activation switch (5) and a battery (6.) Illustration 2 shows one concept of the pen with a roller ball illustrated along with a typical simple message and working plane points.

1 Illustration 3 shows one concept of the pen with a fibre tip illustrated along with a typical simple message and working plane points.

Background

It is widely recognised that the use of computers whilst very significant to communication and business administration, can be regarded as creatively limiting, particularly in the visual arts and writing industry. One of the purposes of the invention is to enhance user fluidity and originality in the way in which original thought is recorded not via a mechanical device such as a keyboard or mouse or stylus or digitiser but through a fundamental human skill which has been learned the world over, hand-writing on paper.

OTA() Wireless computing (Bluetooth. essentially aflows either products or components of A products to talk to themselves and their recipients within a reasonably close (RT14) proximity. Bluetooth is a joint initiative of five leading telecom and computing A companies Nokia, Ericsson, Intel, IBM and Toshiba within the computer and telecoms industries. It is estimated that before the year 2002 BluetoJ"'wlill'be a built A in feature in more than a 100,000,000 mobile phones and in several million other communication devices ranging from headsets and portable PC's to desktop computers and notebooks. The Bluetoot ffiechnology specifications are available royalty free, making them attractive to many manufacturers wanting to work within this communication protocol.

Bluetooth itself is the protocol name for an open specification for short range wireless connectivity. The technology allows users to make effortless instant connections between a wide range of communication devices, it is based on a radio link for facilitating fast and secure transmissions of both voice and data. Bluetooth operates in a Globally available frequency band ensuring communication CRTW compatibility worldwide, in short Bluetooth is a Global specification for wireless

A connectivity.

2 (araf) Most manufacturers see Bluetooth as a mechanism that allows products to talk to other products, when combined with the concept of body worn computers however the potential of Bluetooth starts to become more apparent.

Body worn computers have been in evidence Jor a number of years. There are a number of examples already available in the commercial market where belts fitted with processors; power suppliers and portable hard disks connect to either PDA's or headsets with internally reflected images of the computer screen. These micro displays are currently state of the art and have been shown in use by companies such as IBM in the USA.

The interface to a body worn computer can either use a hand held mouse type device giving good point dexterity, but little in the way of alpha numeric input or a pen based PDA (personal digital assistant - such as Palm Pilot) type product, neither are ideal for the recording of original thought. One alternative will be the use of digital data recording where your voice digitised and c_Qnverted through an f<Tof appropriate software such as Dragon Naturally speaking or IBM's voice recognition 2% software into ASCII text within say a Microsoft Word document. Whilst this may be suitable for something like 20% of original thought recording for most people that don't work in an isolated private location digital voice recordin'g is not suitable.

One of the most important technical innovations of the modern computer was the mouse. The graphical user interface (GUI) developed by Apple and subsequently Microsoft uses a mouse heavily, and whilst it creates good dexterity for screen cursor manipulation, the mouse generally has to be used with a keyboard as a principal input device for text input etc.

Digitisers are also recognised as a good method of capturing either sketch or written text when combined with character recognition, however digitisers involve the use of either active or inactive pen based products used on PDA based products enabling the user to write on a virtual tablet as the display line appears on the pointer. Whilst character recognition is. now reasonably advanced, the typical applications for products such as this (Palm Pilot) are slow to use and cumbersome for original 3 thought input. Moreover products such as this involve the transport of additional equipment in the form of the digitiser or PDA or transfer port to the computer, and do not offer the freedom of this invention.

Explanation of Invention Whilst most people can learn to type and use a traditional keyboard, not all people learn to type proficiently. Typing can lead to Repetitive Strain Injury and many other conditions due to the nature of the key strokes and the use of the fingers and it is generally recognised that the use of a pen and paper is preferred for the capturing of original thought. This is particularly true when a combination of alphanumeric characters and sketch imagery or drawings are required. Whilst the mouse may be able to work in sketch mode, many software applications generally are not fluent in line transfer quality. It is therefore the purpose of the invention to utilise mans ability to write with an ink-bearing pen on standard paper and combine the movements of this pen with the input instructions to the local computer.

The pen or pencil traditionally has been used in this communication and even in today's high technology society this is still broadly true when conveying original thought.

Learning to write is one of the fundamentals of the developed world's education policy. Part of the education process involves the demonstration of achievement and associated pride with the learning and communication achievement. Whilst to a lesser degree this is also true of the use of computers as one learns, it is particularly true the formal education years of 3-7 where a child learns to write or draw on a medium that can be easily transported for the mother and/or father to see.

This is particularly important when considering the explanation of this invention as it is the purpose of this invention to ensure that the fundamental skills of writing are not superseded or replaced by the computer but potentially enhanced as the ink flowing 4 pen based medium becomes a principle input for the computing technology whilst at the same time creating writing on paper.

Most educated countries encourage people to write from a young age and most educated people historically have developed their own style and technique for writing which leads to completely individual styles and signatures for all people. It is also recognised thatthe use of computers, keyboards, digitisers and mouse etc can restrict creativity particularly when line sketch is used to convey visual message from one person or another.

The invention relies on the wireless protocol of Bluetooth or similar to allow an A (9T#14) intelligent pen to communicate with a remote PC within the field of Bluetooth or A similar radio signal. The computer may be body worn, may be resident on a nearby desk or may be portable in a bag or similar.

Working Description

In its simplest form the pen based product is an ink bearing reservoir pen like product, ideally no larger than a pen or any heavier. It may have a roller ball or a fibrous type (1) fed from a continuous supply reservoir. It will'also feature one of a number of alternate technologies (4) that will enable the movements of the pen t eleT recorded by the computer for recording and onward transmission via the Bluetooth or A similar circuitry (3).

An LED or similar indicating device will be used to show the status of the pen when in use. It is anticipated that this indicator will have a number of conditions, off, on showing red - transmitting, and on showing green operational. The LED and internal circuitry will be powered by a small battery (6).

In one aspect of the invention the product features a roller ball writing tip with a method of reading the relative position of the roller ball enabling the pens movement to be detected. This reading systerri could be achieved in a number of ways, etched lines on the ball read by a laser, contact points as in a mouse read by an optical system or potentially some form of switch part that detects movements in any axis and sends them to the computer processor. Similarly an accelerometer or positional recording system, such as GPS (global positioning system) or an LPS (local positioning system) might be used that allows the movements of the pen to be recorded in space. In the case of the LPS the computer might act as one positional device, the pen as a second and a remote body worn device act as the third to enable triangulation of position.

In another aspect of the invention a fibrous tip could be used to deposit the line. In either case a simple switch will be used that senses pressure when the tip is brought into contact with the writing surface and a line deposited. The movement sensing in the case of the fibre tip might be aghieved using a strain gauge system similar to the Tin) cursor control on an IBM think pad where the pressure from the fibre tip pushes one A of three or four probes or strain gauges so resulting in movement being detected. (See illustration 2) The pen also features a button which is the equivalent of the enter (initiate) or send button. (5) In another aspect of the invention the technology could be applied to any other form of writing implement including fountain pen, gel tip, pencil or any other writing technology that relies on marking paper.

In one or more of these methods a number of simple orientation guides may be used in order to tell the computer where the pen should operate relative to an initial start position. On the initiation command, (which might be activated by pressing the end of the pen to open a new file), orientation in space might involve 'picking'three or four real corners (extents) of the page to define a working plane in space. This might be achieved either by applying pressure via the writing tip or by pressing the end switch on the pen. The receiving software would translate this 'referenced' sized working plane into a work area with pre-set margins and a priority to start in the top left or the initial contact position that the pen registers.

In all cases the distance moved by the pen at each stroke needs to be recorded as either an in-contact or not in-contact movement determined by pressure on the 6 writing part of the device. This is particularly important if sketching or drawing is being undertaken and the line movements are random and not translatable into text based characters.

CRTOO The logic of the product is broadly based on the Microsoft Word97 Intellisense feature, whereby the order of certain characters in alphanumeric form, followed by an V'sign followed by '.' result in an email address being identified. For example briansmith@pdd.co.uk is recognised by the application as an address due to the proximity of the @ and the full stop with no spaces between. The handwriting of the address with these symbols in the right place results in the intelligent application knowing that the writer is about to prepare a letter for sending as an enlail. Similarly this logic could be used to tell the application that once the core of the letter has been written that the use of the words 'regards' or'sincerely' terminates the letter and other than the sigrature identifying the sender, it is complete.

Conceptually by writing the email address and having it recognised and by signing the letter having previously written the word 'regards' ' yours sincerely' or similar, tile application knows the letter has a beginning, an end and content to it, which, on instruction (pressing the button on the pen) will send the instruction from the pen to the CPU to activate the send command within the CPU, via GSM or similar thus allowing the email to be sent.

The main advantage of this invention is the writer has used an ink based pen to write original thoughts on a piece of standard paper whilst at the same time having an option to send a copy of this document as an email and save a coy of the document electronically via a computer that is in close proximity and connected wirelessly whillst writing. (It is assumed the computer sleeps until awakened by the writer).

Details such as changes and corrections can be considered by the use of a simple protocol, similar to the way that Microsoft Word detects the email address in the first instance. So for example crossing out a letiter and putting in another letter beside it would imply a change to the computer.

7 The CPU could use either character based recognition in order to determine the handwriting of the person using it or might repeat the handwriting in the original written form it is prepared in. Whilst the email address is required to be recognised as a traditional set of characters, the use of the product for sketching and recording original drawn ideas would mean that recording and communicating non translated images is important.

In either case it will be necessary that the address and signing text will need to be recognised, hence it is intended that some form of character based recognition software based on existing software would be used whereby in order to function seamlessly the device would have to be educated to the users writing style, which would form an introductory part of the software learning and initiation of the user to the device.

8 9

Claims

I A hand-held implement for producing a real or virtual mark on a surface, the implement including means for producing an electronic signal representing said mark.
2. An implement according to claim I wherein the signal includes a sequence of spatial coordinates of the mark.
3. An implement according to claim 2 wherein the coordinate system is external to the implement.
4. An implement according to claim 3 wherein the coordinate system is provided by a global positioning system.
5. An implement according to claim 3 wherein the coordinate system is a local positioning system.
6. An implement according to claim I wherein the implement includes a member having at least one characteristic that changes as the mark is produced, means being provided to sense changes in said at least one characteristic and to produce said signal therefrom.
7. An implement according to claim 6 wherein said member moves relative to the remainder of said implement as said mark is produced.
8. An implement according to claim 7 wherein said movement is a rotational movement.
9. An implement according to claim 6 wherein the member produces directional strains that are sensed to produce said signal.
10. An implement according to any one of claims I to 9 wherein the implement produces a real mark on said surface.
11. An implement according to claim 10 wherein the implement includes a pen.
12. An implement according to any one of claims I to I I and including means for transmitting the signal in the form of a wireless signal.
13. An implement substantially as hereinbefore described with reference to the accompanying drawings.
14. An implement according to any one of claims I to 12 in combination with data processing means, said data processing means receiving said signal.

11
15. A combination according to claim 14 wherein the data processing means includes a display, said data processing means producing from a received signal a mark on the display corresponding to the mark represented by said received signal.
16. A combination according to claim 14 or claim 15 wherein the data processing means includes data representing a predetermined mark or marks, the data processing means comparing the mark represented by a received signal with said data and producing a correlation signal when the mark represented by the received signal is the mark represented by said data.
17. A combination according to claim 16 wherein said correlation signal causes the data processing means to perform an action.
18. A combination according to claim 17 wherein the correlation signal causes the data processing means to send an electronic message including alpha- numeric characters formed as marks by said implement.
19. A combination acco ing to claim 17 wherein a plurality of correlation signals are produced, said sig als defining the area of the surface on which the implement forms said mark.

12
20. A combination according to any one of claims 14 to 19 wherein the data processing means is a computer.
21. A combination of an implement and data processing means substantially as hereinbefore described with reference to the accompanying drawings.