WO2016040218A1

WO2016040218A1 - Pen-location-determining and transcription method and apparatus with automatic page-flip detection

Info

Publication number: WO2016040218A1
Application number: PCT/US2015/048774
Authority: WO
Inventors: Bradley Richter; Jootae "TJ" KIM
Original assignee: Luidia, Inc.
Priority date: 2014-09-08
Filing date: 2015-09-07
Publication date: 2016-03-17

Abstract

An apparatus for and a method of capturing pen stroke locations, including automatically detecting page-flips. Also a computer-readable medium coded with instructions that when executed carry out the method. The apparatus includes a receiver with at least two acoustic sensors to sense acoustic pulses transmitted from a pen in an active area. The receiver determines and store stroke locations and stroke time information. The receiver is operative to accept or determine a set of one or more indicators of page-flip, and to combine the indicators of the set to automatically determine a page-flip event, including detecting that there is a page- flip and determining the location-in-time of the page-flip.

Description

PEN-LOCATION-DETERMINING AND TRANSCRIPTION METHOD AND APPARATUS WITH AUTOMATIC PAGE-FLIP DETECTION

RELATED APPLICATION

[0001] The present invention claims priority of U.S. Provisional Patent Application No.

US 62/047,082, filed 2014-09-08 to Applicant Luidia, Inc. with inventors Richter, et al. In any jurisdiction where incorporation by reference is permitted, the contents US 62/047,082 are incorporated herein by reference.

FIELD OF THE INVENTION

[0002] The present invention relates to systems for determining and transcribing digital stylus locations using acoustic waves, e.g., ultrasound.

BACKGROUND

[0003] There often is a need to transcribe pen strokes, for example those that occur during writing. There is further a need in the art for a system and method designed to allow a user to create, save, store and retrieve digital handwritten notes on both paper and digital media. [0004] Location determining and transcription systems ("location capture systems") are known for locating and recording locations, e.g., strokes of a pen, pointer, marker, or stylus on an ordinary surface, e.g., on an ordinary whiteboard, on paper, on a projection screen, on a flat- display surface, and so forth. The term pen and stylus are used interchangeably herein to mean any such pen, pointer, marker, or stylus, whether such pen marks the surface or not. [0005] A stylus device is known that includes a transmitter for transmitting acoustic pulses, e.g., ultrasound ("US") pulses and a transmitter for transmitting electromagnetic ("EM") pulses, e.g., infrared ("IR") pulses, and such a device, when used with a capture unit places adjacent to an area, is usable for locating and transcribing locations of the stylus. The capture unit typically includes sensors for receiving the transmitted US and I R pulses. The capture unit in some versions may include a processing engine with memory, and in some of these versions, versions processing and storage may be local. The capture unit includes an interface, e.g., a wireless connection or USB connection, to send information to a remote device. Examples of remote devices include, a so-called "smart" phone that includes a processor, e.g., a phone operating under the IOS (RTM), ANDROID (RTM), or some other mobile device operating system, a tablet, a computer, personal digital assistant (PDA), a projector that includes a processor and memory, a TV or other flat-screen display device that includes a processor and memory, and so forth. Such a system, in combination with the computer, captures the pen strokes on or for remote device, including the color of the pen and any erasing. [0006] As an example, Luidia, Inc., the Applicant of the present invention, makes a system called eBeam (RTM) that provides for recording pen strokes on a surface such as an ordinary whiteboard. The Applicant also markets a product called EQUIL SMARTPEN (TM) which includes a pen device that has a transmitter for transmitting US pulses and one for transmitting IR pulses. These US and IR pulses are usable, when appropriately detected, to determine location. The system also includes a receiver subsystem for receiving the transmitted US and IR pulses. Using relative times of arrival of pulses information, the receiver can determine stroke locations. The receiver includes one or more interfaces (wireless and wired) to send information to a host device, such as a tablet, smart phone, or a computer, e.g., a laptop computer. The receiver is places on an edge, e.g., top edge of a page of paper and the combination provides for determining the position of the pen on the surface in an active area adjacent to the sensors and thus capturing pen strokes.

[0007] Such receivers may be battery operated and may include a memory to record pen strokes without needing an external computer. The stored pen strokes however do, at some stage, need to be loaded to a host device.

[0008] In some situations, the pen can be used as a general purpose pointing stylus. A marking pen may then be undesirable. The marking on the surface would be "virtual" in that no ink or marking material need be used. The term pen as used herein covers such a non-marking stylus device. [0009] For a system designed for taking notes, there typically is a button or a portion on the active area for indicating a new page (a new empty slate). However, a user might forget to indicate that a new page has been started (such an event called a "page-flip" event herein). After a series of strokes and relative times for said strokes for a plurality of pages are collected, it is typically difficult to determine where a new page break as all pen strokes appear to be on a single page. Thus there would be an advantage to automatically detecting page-flip events in such a transcription system.

BRIEF DESCRIPTION OF THE DRAWINGS

[0010] FIG. 1 shows a simplified diagram of an example system that is connectable to a host device, and that includes aspects of the present invention. [0011] FIG. 2 shows a block diagram of elements that can be included in the system of FIG. 1.

[0012] FIG. 3A shows in simplified form a switch usable to detect a page-flip in some embodiments of the invention, and FIG. 3B shows a proximity sensor used to detect a page-flip in some embodiments of the invention. [0013] FIG. 4 shows a receiver, a pen, and a charging bracket with pen-stroke determining capability, into which elements described herein may be included such that the combination operates as an embodiment of the invention.

[0014] FIG. 5 shows a flowchart of a method embodiment of the invention. DETAILED DESCRIPTION

Overview

[0015] Described herein is an apparatus for capturing pen stroke locations, including automatically detecting page-flips, a method for operating such an apparatus that includes automatically detecting page-flips, and a computer-readable medium on which are stored instructions that when executed on one or more processors an apparatus for capturing pen stroke locations, carry out a method of capturing pen strokes that includes automatically detecting page-flips.

[0016] Particular embodiments of the invention include an apparatus configured to capture pen strokes, including automatically detecting a page-flip. The apparatus includes a receiver configured to be placed on or close to an edge of a surface. The receiver includes at least two acoustic sensors located at pre-defined locations relative to the surface and arranged to sense acoustic pulses transmitted from a pen in an active area of the surface adjacent to the predefined locations. The pen includes at least one transmitter to transmit acoustic signals detectable by the acoustic sensors. [0017] The receiver is arranged to determine and store pen-tip locations (stroke locations) and stroke time information of the pen in an active area on the surface adjacent to the predefined sensor locations.

[0018] The receiver alone, or in combination with a host device of the apparatus, is operative to accept or determine a set of one or more indicators of page-flip, and, for versions in which there is a plurality of indicators, to combine the indicators of the set to automatically determine a page-flip event, including detecting that there is a page-flip and determining the location-in-time of the page-flip.

[0019] In some embodiments, the receiver includes an interface to couple the receiver with a host device. The host device includes a host processor and a memory. The receiver is operative, when coupled to the host device, to send determined stroke locations and corresponding time information to the host device.

[0020] In a first set of apparatus embodiments, the receiver comprises one or more physical sensors to determine respective one or more physical-sensor-derived page-flip indicators, such that the set of indicators includes the one or more physical-sensor-derived page-flip indicators. In different embodiments of the first set of apparatus embodiments, the one or more physical sensors include at least one member of the set consisting of: an accelerometer; a mechanical press-switch placed on a bottom area of a housing of the receiver; a proximity sensor; and a set if one or two light sensors. The switch requires very light force to press, e.g., less than the weight of the receiver, and located at the bottom of the housing of the receiver; [0021] In a second set of apparatus embodiments which might overlap with the first set, the set of one or more indicators of page-flip includes one or more content-based page-flip indicators that each detects a page-flip event by calculating unintended content overlap or unintended time gaps from the determined positions and time information. In different ones of the second set of apparatus embodiments, the one or more content-based page-flip indicators include at least one member of the set consisting of: a time-slicing based indicator based on determining from the content long time gaps between two pen strokes; a stroke overlap indicator based on determining a measure of overlap of strokes, wherein an overlap is defined as two non-consecutive strokes that intersect occurring with a relatively long time gap there between; and a region overlap indicator based on grouping sets of strokes by non-overlapping time periods, determining a region for each set, and determining whether the determined regions overlap.

[0022] In some apparatus embodiments, the receiver comprises a processing engine that includes a processor and a memory for firmware. The firmware in the memory includes instructions that when executed cause: a) the determining and the storing of the locations and time information, b) the sending of the determined locations and time information to the host device, c) the receiving and/or determining of the plurality of indicators that are accepted and/or determined by the receiver, and 4) any of the combining carried out in the receiver.

[0023] Particular embodiments of the invention include a method of operating an apparatus that captures pen strokes, the method comprising: [0024] · receiving acoustic pulses at a plurality of acoustic sensors housed in a housing of the receiver that is placed on or close to an edge of a surface, the sensors located at pre-defined locations relative to the surface to sense acoustic pulses transmitted from a pen in an active area of the surface adjacent to the pre-defined locations, the pen comprising a transmitter of acoustic pulses; [0025] · determining locations of the pen in the active area on the surface adjacent to the pre-defined locations and determining time information for the determined locations;

[0026] · storing the determined locations and time information;

[0027] · accepting or determining a set of one or more indicators of page-flip; and [0028] · in the case the set includes a plurality of page-flip indicators, combining the indicators to automatically determine a page-flip event, including detecting that there is a page-flip and determining the location-in-time of the page-flip.

[0029] One method embodiment includes sending the determined coordinates and timing information to a host device;

[0030] In a first set of method embodiments for operating the apparatus, the apparatus comprises one or more physical sensors to determine respective one or more physical-sensor- derived page-flip indicators, the set of one or more page-flip indicators of the method includes the one or more physical-sensor-derived page-flip indicators. The housing of the receiver includes a bottom portion that makes contact with the surface when the method determines locations. In different ones of the first set of method embodiments, the one or more physical sensors include at least one member of the set consisting of: a press switch requiring a relatively light force to activate and located at the bottom of the housing of the receiver; an accelerometer; a proximity sensor; and a set if one or more light sensors. [0031] In some embodiments, the method further comprises calculating one or more content- based page-flip indicators that each are indicative of a page-flip. The calculating includes calculating from the determined positions and time information unintended content overlap or time gaps that might be indicative of an intended page-flip. In such embodiments, the set of one or more indicators of page-flip includes at least one of the one or more content-based page-flip indicators. The one or more content-based page-flip indicators include at least one member of the set consisting of: a time-slicing based indicator based on determining from the content long time gaps between two pen strokes; a stroke overlap indicator based on determining a measure of overlap of strokes, wherein an overlap is defined as two non-consecutive strokes that intersect occurring with a relatively long time gap there between; and a region overlap indicator based on grouping sets of strokes by non-overlapping time periods, determining a region for each set, and determining whether the determined regions overlap.

[0032] Particular embodiments of the invention include a non-transitory machine-readable medium coded with instructions, that when executed by a processing system, carry out any one of the above summarized methods. [0033] Particular embodiments may provide all, some, or none of these aspects, features, or advantages. Particular embodiments may provide one or more other aspects, features, or advantages, one or more of which may be readily apparent to a person skilled in the art from the figures, descriptions, and claims herein. System comprising a pen and a receiver

[0034] One embodiment includes a receiver comprising electronics configured for determining and capturing the locations of an electronic pen or pointer ("pen"), to provide for a user the ability to create, save, store and retrieve sets of pen strokes, e.g., handwritten notes, either on paper or electronic media.

[0035] FIG. 1 shows a simplified diagram of an example location determining system 100 that includes aspects of the present invention and that may be coupled to a host device 150, e.g., a smart phone 150. The system 100 is for writing on a writing surface 105 which may be a sheet of paper on which one can write with a marking pen, or some surface on which one can electronically write with an electronic pen. The system 100 includes a receiver 1 1 1 that is located at the periphery of the surface 105. The receiver includes at least two US sensors with a spatial relationship there between. FIG. 1 shows a receiver 1 1 1 with two US sensors 1 13, 1 15, and one IR sensor 1 17. There is a known spatial relationship between the US sensors 1 13, 1 15, and also between the sensor locations on the receiver, and an edge, e.g., the top edge of the writing surface 105. The receiver forms an active area 107 on a substantially planar surface, which in this embodiment is the surface 105. The receiver 1 1 1 includes a processing engine 131 coupled to the sensors 1 13, 1 15, 1 17 of the receiver 1 1 1. The processing engine is operative to determine the position of a pen 103 in the active area 107, in particular of the tip 127 of the pen 103 from which US pulse are emitted (and in some versions, reflected off the page) towards the receiver 1 1 1 when the pen 103 is on the page.

[0036] One version of the receiver 1 1 1 includes a rechargeable battery 145 and a mechanism for connecting the receiver to a charger. A charging station 140 is provided that includes a receptacle 141 into which the receiver may be inserted, and when so inserted, has its battery charged. [0037] In one embodiment of the pen 103, the tip 127 is a marking tip. This embodiment includes a pen body, the tip 127, and a transmitter 123 of US pulses close to the tip arranged to transmit US pulses detectable by the receiver 1 1 1 when the pen is in the active area 107 and pressed into a sheet of paper. In one version, the pen 103 includes an electronics module 121 to drive the US transmitter 123 to transmit a set of US pulses detectable by the US sensors 1 13, 1 15 when the tip 127 is pressed against the paper surface 105, using, for example, a switch in the body of the pen 103. The pen embodiment 103 shown in FIG 1 also includes an IR transmitter 125 driven by the electronics module 121 and that transmits IR pulses detectable by the IR sensor 1 17 of the receiver 1 1 1 when the tip 127 is pressed against the paper surface 105. The IR pulses are synchronized with the US pulses. [0038] One embodiment of the pen 103 further includes one or more buttons (not shown) each of which activates a respective switch. When a button is depressed and the pen 103 is in an active state, the US and IR transmitters 123, 125 transmit energy in a particular form related to which button was depressed. In some embodiments of the invention, the function of the buttons may be programmed, e.g., to be the left or right buttons of a mouse device. Thus, an aspect of some embodiments is that a button of the pen 103 can provide various optional functions, in the same manner as can the different buttons of a mouse or other stylus or pointing device, e.g., for a computer.

[0039] In some embodiments, the pen 103 also includes a rechargeable battery 147. In this description, each of the pen 103 and receiver 1 1 1 has a receiver battery 145 and pen battery 147, respectively. A charging station 140 is included for charging the receiver and pen batteries. The charging station includes a receptacle 143 (a pen cradle) designed to hold the pen 103 and recharge the pen battery 147, and the receiver charger receptacle 141 designed to receive the receiver 1 1 1 and charge its receiver battery 145. The coupling between the pen 103 and/or the receiver 1 1 1 in some embodiments is via a respective set or sets of contacts on the pen 103 and/or the respective receptacle in station 140. Alternative embodiments use contact-less inductive coupling.

[0040] The pulses transmitted by the US and IR transmitters 123, 125 of the pen 103 may be modulated or digitally coded using the electronics module 121 to identify a particular pen function, e.g., that the pen 103 represents a marking device of one color or another, or that the pen 103 represents an eraser, or whether the pen 103 represents a marking device drawing a thin line or a thick line, or whether the button(s), in the pen 103 are functionally the same as the left or right buttons of a mouse, and so forth. One version of the pen 103 includes a pressure detector, so that the more pressure is applied to the pen 103, the thicker the pen strokes determined and recorded by the receiver 1 1 1.

[0041] One version of the pen 103 has a low-power state that hardly draws any power.

Invoking any of the buttons moves the state from the low-power state to an active state and further provides an indication of which button was invoked.

[0042] The receiver 1 1 1 includes a clamp with a barrier to enable the receiver to be placed at the known distance from the top edge of the page surface 105, for example, after the receiver is removed to perform a physical page-flip. In some embodiments, the receiver 1 1 1 includes one or two notches that enable one to mark location so that in a new page these location marks can be aligned by looking through the notches.

[0043] When operational, the location determining system 100 is able to determine the position of the pen tip 127 in the active area 107, and thus capture the pen strokes written on the surface 105. [0044] FIG. 2 shows a more detailed, but still simplified block diagram of the system 100 that includes an embodiment of the invention. System 100 includes the pen 103, the receiver 1 1 1 and the host device 150. The system 100 includes an embodiment of the invention and further includes a communication link between the receiver 1 1 1 and the host device 150. The US sensors 1 13,1 15, and IR sensor 1 17 are coupled to a processing engine 131 that includes at least one processor 203 that in one embodiment is a DSP device. A single processor will be assumed for purpose of explanation. The processor 203 includes or is coupled to a memory 205 for firmware, coupled to processing elements of the processor 203, e.g., via a bus subsystem. Note that while the various processing elements, e.g., multiply-add units, general purpose logic units, and so forth, are shown as a single processor 203 in FIG. 2, those in the art will understand that this does not imply that there is only a single processing element in processing engine 131.

[0045] The receiver 1 1 1 includes one or more input/output (I/O) interfaces, e.g., one or more of a USB interface, a Bluetooth wireless interface, and a Wi-Fi wireless interface. Other I/O interfaces of course may be included in different embodiments. The I/O interface(s) are shown as a single block 213 in FIG. 2. [0046] In one embodiment, aspects of the present invention include methods that are implemented by sets of firmware instructions in the firmware memory 205. Furthermore, while embodiments of the invention use a DSP device, it would be clear to those in the art that any processor with sufficient processing power, e.g., a microprocessor or microcontroller, may be substituted for the DSP device, or alternately, that one or more programmable logic devices, or even hardwired logic may be used, e.g., as an application specific integrated circuits (ASIC) or custom chip.

[0047] When an IR pulse is detected via the IR sensor 1 17, a switch connects the output of the US sensors 1 13, 1 15 such that by the time the US pulses arrive from the pen 103, the received US pulses are input via respective serial ports to the processing engine 131. [0048] A timing generator 215 sends time information to the processing engine 131 . The received US pulses together with time information and any information on the state of any buttons on the pen 103 are sent to the processing engine 131 to determine the times of arrival of the US pulses relative to the times of arrival of the IR pulse. The together with information on the state of any buttons on the pen 103 is further processed by processing engine 131 . [0049] The firmware memory 205 coupled to the processor 203, which, if a DSP device, may include built-in DSP firmware memory (regarded herein as all or part of firmware memory 205), and more memory, e.g., additional static RAM, such RAM regarded herein as all or part of a memory 21 1 . Memory 21 1 includes a portion 253 for one or more parameters, such as probabilities for a set of page-flip indicators, threshold for page-flip indicators, and so forth, in order to enable the system to be easily configurable in the field, as well as a portion 251 for pen- stroke locations and timing information determined by the receiver. [0050] In one embodiment the firmware memory stores instructions 206 that when carried out by processor 203 to determine the locations of the pen 103, timing information, and one or more other indications.

[0051] The pulses transmitted by the IR transmitter in the pen 103 are assumed to travel much faster than the US pulses, e.g., "instantaneously" compared to the speed of sound. The IR pulses received by the IR sensor 1 17 and the US pulses received by the US sensors 1 13, 1 15 are recorded in the location determining system's receiver 1 1 1 . In one embodiment, the operation of the location determining of receiver 1 1 1 includes determining the times of arrival of the pulses. The location determining of receiver 1 1 1 calculates positions of the pen tip 127 based on the arrival times at the two US detector positions relative to the time of arrival of the IR pulse, which is a time reference generated by the IR sensor 1 17. In one embodiment, the calculations rely on accurate recording of waveforms of the received pulses, so that accurate matching can be used to get improved time accuracy, which in turn leads to greater accuracy of location of the pen tip 127. [0052] The locations, timing information, and other indicators in one embodiment are stored as 251 in the memory 21 1 of receiver 1 1 1. One version of the receiver 1 1 1 and pen 103 combination is usable stand-alone without being connected to a host device such as the host device 150. The receiver and pen combination is also usable while connected to a host device 150, in which case the locations (as coordinates), timing information, and indicators are transmitted to the host device 150 in real time.

[0053] In the stand-alone use case, at some stage, communication is established with the host device 150, and the locations (as coordinates), timing information, and indicators 251 from memory are transmitted to the host device 150

[0054] The host device 150 is also shown in FIG. 1 and in more detail, in FIG. 2, and includes standard components such as at least one processor 221 , memory 223, a USB interface 227, a display 225, one or more wireless interfaces 231 , a touch screen 233, a (virtual or physical) keyboard, a battery, and so forth. Aspects of the present invention are implemented, in one embodiment, as instructions 229 in the host device 150, shown in FIG. 2 as in memory 223. The memory 223 also includes such instructions for such functionality as character recognition, etc., and of course, the usual functions of the host device 150, e.g., as a telephone etc.

[0055] In one embodiment, the receiver 1 1 1 includes a new page button 1 19 to provide the user with the ability to indicate when a new sheet of paper is used for surface 105, such an event being one example of a page-flip.

[0056] In one embodiment, the receiver sends the host device location information in the form of (A, B) un-normalized coordinate pairs, together with signals about the type of pen, e.g., color, line thickness, and so forth. Calibration may be carried out using the pen 103 and receiver 1 1 1 using known methods.

[0057] The processing engine 131 of location capture system's receiver 1 1 1 further accepts input from the one or more buttons 137, and one or more sensors usable for page-flip detection. In one embodiment, the information sent to the host device is in the form of (A,B) un-normalized coordinate pairs, and signals about the type of pen, e.g., color. Calibration may be separately carried out, for example, in the host device to convert the un-normalized (A,B) coordinate pairs to (x,y) coordinate pairs in the active area, e.g. to strokes in the active area. Timing information also is determined. In addition, events such as those that signal pen-up and pen-down are sent. In one version, such events are provided in as (penup, timestamp) pairs, where penup is the pen-up event and timestamp is an indication of the time that the event occurred. (A,B) coordinate pairs are provided in the form of ((A,B), pentype, any_error), where the pentype indicates the color or whether an eraser, and so forth. Furthermore, an eraser is regarded as a special erasing pen that erases an area around its coordinate, such that erasure regions also are transmitted. Also events such as one or more buttons on the pen being pressed are sent. Thus, the host device, after calibration, accepts a pen-down event and a pen-up event with a stream of coordinates in between that represents a contiguous line.

[0058] In some versions, the system 100 uses a firmware program 206 installed in the receiver 1 1 1 to determine and to store the pen strokes that the device captures and to implement the actions of the programmable buttons.

Automatic Page-flip detection

[0059] Aspects of the invention include the automatic detection of page-flips. While the receiver 1 1 1 includes a button 1 19 for indicating a new page to the system, it may happen that a user forgets to press the page-flip button. As already described, the receiver 1 1 1 determines a time sequence of locations (pen strokes) together with an indication of times that such locations for strokes occurred. After locations of two or more pages have been recorded, without a new page indication, it is very difficult to determine where in time the new page should be. For this reason embodiments of the present invention include automatic page-flip detection.

[0060] In one embodiment, the receiver 1 1 1 includes one or more physical sensors whose respective outputs include information that changes during a page-flip, such that the outputs of the physical receivers form indicators of page-flip. The processing engine 131 accepts the output(s) of the included physical sensor(s), and such outputs, called physical-sensor-derived page-flip indicators herein, are used by processing engine 131 , alone or in combinations with the host device 150 to automatically detect page-flips and the locations thereof. [0061] In some embodiments, a set of one or more content-based indicators are calculated (in processing engine 131 and/or host device 150) from the location and time information (jointly "content") and other indications calculated (and locally stored) by the processing engine 131 (and sent to the host device 150).

[0062] While one embodiment uses a single page-flip indicator for automatic page flip detection, embodiments of the invention carry out automatic page detection using a plurality of page-flip indicators, wherein

[0063] · All the page-flip indicators in the plurality are derived from the outputs of physical sensors,

[0064] · All the page-flip indicators in the plurality are content-based indicators determined by calculation from position and timing content, or

[0065] · The plurality includes or more physical-sensor-derived page-flip indicators and one or more content-based indicators.

[0066] Embodiments of the invention use the plurality of page-flip indicators to automatically determine a page-flip event, including detecting that there is a page-flip and determining the position, e.g., the location-in-time of the page-flip. Compared to using a single page flip indicator, the likelihood of the determined page-flip event being a true positive is increased, and furthermore, the result does not depend on the functioning of a single page-flip indicator.

[0067] Some of the page-flip indicators are determined in the receiver 1 1 1 , e.g., by the processing engine 131 executing instructions 207 stored as firmware in the firmware memory 205.

[0068] One aspect of the invention is the assigning to each (physical-sensor-derived or content-based) indicator a likelihood or probability measure (or confidence level) that the indicator is correct and/or incorrect, and applying a statistical decision method to determine the detection decision (of whether there was a page-flip). One method assigns to each indicator not only a probability measure of a true positive, but also a probability of a false positive, i.e., that the indicator indicates a page-flip when no page-flip occurs, and/or a probability of a false negative, that is, an indication that no page-flip is indicated when a page-flip occurred, with probabilities of correct positive and negative probabilities. Furthermore, each assigned probability may incorporate a "cost," "bias" or weighting of the decision. For example, in most situations, the cost of a false negative, i.e., there was a page-flip but the indication is that it did not occur is much higher than the cost of a false positive— an indication that there was a page flip when in fact there was not. A false positive, i.e., page-flip marker wrongly placed at a position in a time-sequence of locations can readily and easily be corrected by a user by simply removing the page-flip. Furthermore, such a situation is easily manually detected, whereas a situation wherein a time-sequence of locations with no page-flip is obtained as result of missing one or more actual page-flips, i.e., one or more false negatives, is difficult to re-analyze to determine the correct placement of page-flip(s). Therefore, in embodiments of the invention, the assigning of probability measures and the combining of the plurality of page-flip indicators are biased to prefer false positives to false negatives.

[0069] Many methods are known in diverse fields including statistics, electrical engineering, operations research, military science, etc. on methods of automatically making a (binary, i.e., yes/no) decision based on pre-assigned probabilities of several indicators. Only one simple method is described herein, and of course, the invention is not limited to such a simple method.

[0070] Many decision methods have their origin signal detection methodologies that originated in such electrical engineering areas as radar detection, and detecting binary signals in communications. Many are based on statistical methods. Some are based on pattern recognition and machine learning. It is not a goal of the invention to invent new decision theory, but rather to provide a simple example, with the understanding that such a method does not limit the invention, and different embodiments of the invention may be readily and straightforwardly modified by one skilled in the art to incorporate many different decision methods without undue experimentation.

[0071] In some embodiments, the assigning of probability measures and the statistical detection method are selected to minimize the probability of false negatives

[0072] Furthermore, improved embodiments can use data to improve the initially assigned probabilities by classifying the indications made as one of false positive, false negative, true positive, or true negative, and thus adapting the probabilities. Furthermore, the overall process can be improved by classifying the decisions made as one of false positive, false negative, such that the decision is improved as more data is written.

[0073] Furthermore, in the method of detecting page-flip events that depends on assigning probabilities to one or more indicators, some of the indicators include determining if a measure has exceeded a threshold. The values of such thresholds are easily and readily determined by simple measurements (simple experiments), without undue experimentation, by one skilled in the art. For example, consider a sensor whose output is used as an indication. Even a single measurement of the values of the output as a function of time provides sufficient information for a practitioner to assign value to the threshold used for the indicator. Therefore, the description herein is regarded as sufficient to practice the invention without undue experimentation.

[0074] In some embodiments, some of the page-flip detection functionality is carried out by the host device 150. In some versions, the firmware memory 205 or receiver memory 21 1 includes one or more sets of instructions 208 that may be transferred automatically to the host device 150. Several versions for typical host devices may be included. Such a set of instructions is stored in the memory 223 of the host device 150. One example is the set of instructions 229 used to carry out the host-device portions of the page-flip indication method described herein. [0075] Some embodiments of the invention store more than one set of pre-assigned probabilities in the firmware memory 205 and/or in parameter portion 253 of receiver memory 21 1 to allow for different sets of indicator probabilities to be used . Similarly, some embodiments of the invention provide for varying the sensitivity of one or more of the page-flip indicators, e.g., by having more than one pre-assigned threshold values for the page-flip indicator. For this, the these different parameters for the page-flip indicators may be stored in the firmware memory 205 and/or in parameter portion 253 of receiver memory 21 1 to allow for different sets of page- flip indicator parameters to be used.

Physical sensors for page-flip indication

[0076] The receiver may include one or more physical sensors that provide indicators of a page-flip. The sensors are used to detect relative motion between the receiver 1 1 1 and the surface 105, or motion in the combination of the receiver 1 1 1 and the surface 105.

Accelerometer

[0077] Some embodiments may include an accelerometer 1 16 or other similar sensor of change of motion. All such motion detectors are called accelerometers herein . Accelerometers are inexpensive and reliable, because of their use in laptop computers, tablet devices, and especially smart phones, of which more than a billion have been made.

[0078] The receiver 1 1 1 in one such embodiment is configured, via firmware, to monitor the output of accelerometer 1 16. If the accelerometer output was nearly still for some amount of time, and has high acceleration (above a threshold that depends on the particular

accelerometer), there is high likelihood that the user has picked up the receiver 1 1 1 in order to move it or turn the page on surface 105 beneath the receiver, thus indicating a page-flip.

Furthermore, if this is followed by the accelerometer 1 16 sensing a deceleration followed by a stop, indicating the receiver 1 1 1 is now at rest, the receiver 1 1 1 determines that a page-flip occurred. In an improved embodiment, an 3-axis (x,y,z) accelerometer is used, and estimation is made to ascertain whether or not the receiver 1 1 1 has returned to a similar location in x, y, z space as the pre-motion location. This further increases the confidence that the user removed the receiver, flipped the page, and placed the receiver 1 1 1 in a similar position on a new page.

[0079] In one embodiment, the accelerometer is an ANALOG DEVICES (RTM) ADXL344 three-axis, digital-output MEMS accelerometer, made by Analog Devices, Inc., of Norwood, MA, USA.

[0080] In one embodiment, probabilities of a true positive are assigned to each page-flip indicator and used as relative weightings for determining an overall probability measure when combining the page-flip indicators. In one embodiment, a probability of 0.9 is assigned for the probability of a true positive for the accelerometer-based page-flip indicator. [0081] Improved probabilities can be determined by simple experiments measuring patterns of what happens when different users lift the receiver and place it again on a page. In one embodiment, a relatively large vertical acceleration in the up direction (z direction) is used. One example uses a threshold of +1 g, where g is the acceleration of gravity (9.8 m/s). Patterns can be detected of dropping the paper page, e.g., paper pad and receiver combination, and such patterns may be excluded from those that lead to an indication of detection. One such to be excluded acceleration pattern is that the z-direction acceleration is g (downwards).

[0082] One field-configurable embodiment includes more than one criterion, stored as parameters in the portion 253 of memory 21 1. Contact indicator using a switch

[0083] Some embodiments include a contact indicator 1 18 of whether or not the bottom of the receiver is making contact with the page surface 105. An example of a simple bottom contact indicator 1 18 is a switch 1 18, e.g., one with with spring-loaded button requiring a force less than the weight of the receiver to press, such a force easily determined by tests. While a press- switch is one form of contact indicators, many different types of contact sensors are known, e.g., using capacitance, induction, and so forth, and a different type contact indicator may be used alone or in addition to the press-switch in different embodiments of the invention. In this specification, switch 1 18 is meant to represent any type of contact indicator, so that the invention is not limited to such a press-switch. FIG. 3A shows a partial perspective view 300 of one end and of part of the bottom 301 of the housing of the receiver 1 1 1 , with the switch 1 18 positioned along the bottom surface of the receiver. When the receiver 1 1 1 is placed on (and in some versions, clamped to) the page, the button of switch 1 18 is depressed, and the button becomes flush with the bottom 301. When a user moves the receiver 1 1 1 away from the page in order to flip the page, the switch provides a page-flip indication. [0084] In one embodiment, the switch 1 18 is an ALPS (RTM) model SPPW812302 one-way operation detector switch made by Alps Electric Co., Ltd., Tokyo, Japan. This switch has a maximum operating force of 0.3N. A threshold can be set of between 0.1 N and 0.3 N.

[0085] In one embodiment, the bottom contact page-flip indicator 1 18 is assigned a probability of 0.9 of making a correct page-flip indication. Proximity sensor

[0086] Some embodiments use a proximity sensor 231 positioned to sense the proximity of the receiver to the surface which contains the active area (the page). A threshold distance is selected to detect the receiver 1 1 1 being lifted off the page and provide an indication a page- flip. Many technologies are available for detecting proximity, and, some because they are used in smart phones to detect proximity to one's face to avoid a touch to the screen being

registered, are relatively inexpensive because of the large number of smart phones being made. In one embodiment, the proximity sensor is a SHARP (RTM) model GP2Y0A21YK infrared proximity sensor sold by SHARP CORPORATION of Osaka, Japan. The sensor has an output voltage range of 5 cm to 80 cm (the lower the output, the lower the voltage). In one

embodiment, the page-flip indicator indicates a page-flip when the distance between the receiver 1 1 1 and the area 105 is more than 15cm, which corresponds to an output less than 1 .5V. In one embodiment, the proximity-sensor-based indicator 231 is given a probability of 0.9 of making a correct indication.

Light sensor (also indicative of proximity)

[0087] Some embodiments of receiver 1 1 1 include a first light sensor 233 on the bottom of the receiver, for example as shown in FIG. 3B that shows a partial perspective view 350 of one end and the bottom of the housing of the receiver 1 1 1 . The first light sensor 233 is positioned and operative to provide an indication of light from the surface containing the active area. The output of the first light sensor 233 increases as the receiver 1 1 1 is lifted away from the paper, and thus provides a measure of proximity. The first-light-sensor output can thus trigger an indicator when visible light is increased beyond a threshold. This sensor 233 is listed separately from proximity sensor 231 because it may not be as reliable because its accuracy depends on the ambient light. The first light sensor 231 acting alone, for example, might have problems in dimly lit environments. Therefore, in one embodiment, the light-sensor-based indicator is given a probability of 0.4 of making a correct indication (which would give this a relatively low weighting in combining with other page-flip indicators.

[0088] In an improved embodiment of using the first light sensor 233, a second light sensor 315 is included and arranged to give a measure of ambient light. In one embodiment, the second light sensor 315 is placed on top or side of the housing of the receiver 1 1 1 . The combination of the first and second provides an indication of the difference between light from the surface containing the active area and the ambient light surrounding the receiver (1 1 1 ), and provides a light-sensor-based page-flip based on detecting when the difference between the ambient light measure from the second light sensor 315 and the light measure from first light sensor 233 goes below a predefined threshold. This improved-embodiment light-sensor-based indicator is given a probability of 0.7 of making a correct indication. Note that the performance of this "differential light" page-flip indicator may suffer, for example, if the user covers up the second light sensor 315 when lifting the receiver 1 1 1 . Yet a better differential-light page-flip indicator includes at least a third light sensor also providing a measure of ambient light.

[0089] Note that with all of these indicators, as the user becomes more familiar with the product, and writes on multiple sheets of paper, the user typically learns how to increase the likelihood of successful page-flip detection, and the probability increases that the indicator correctly indicates a page-flip. For example, in the case of the two-sensor differential-light- sensor-based indicator, the user typically would learn to avoid covering up the second light sensor when removing the receiver.

Content-based indicators

[0090] The content-based indicators detect by calculating unintended content overlap or time gaps. Different embodiments of the invention include calculating a different set of one or more content-based indicators, and such indicators may be combined with each other and/or with the one or more physical-sensor-based indications to further increase the likelihood of correctly detecting a page-flip.

[0091] In some such embodiments, the positions and timing information ("content") transmitted to the host device 150 is stored in the memory 223, and is analyzed by the host device (executing page-flip software stored in the memory 223) to determine the content-based indicators of a page-flip. In alternate versions, the processing engine 131 analyzes the position and timing information 251 stored in the memory 21 1 in the receiver 1 1 1 to determine one or more content-based indicators of a page-flip. In yet another embodiment that includes using more than one content-based indicator, one or more content-based indicators are determined in the receiver 1 1 1 , and one or more content-based indicators are determined in the host device 150.

[0092] Content based indicators used in some embodiments of the invention include one or more of: Time-slicing based indicator

[0093] A time-slicing based indicator is based on determining from the content long time gaps between two pen strokes. In one version, if the time gap between two pen strokes exceeds a threshold of a few seconds (or some other time corresponding to the minimum time required to complete a page-flip and commence drawing or writing, the indicator indicates a page-flip. This method is not a reliable indicator of page-flip, and thus has little value alone. For this reason, the time-slicing based indicator is assigned a relatively low probability of correctly detecting a page- flip, e.g., 0.3. However, this indicator is very effective when combined with other methods.

Stroke overlap indicator

[0094] One embodiment includes determining a measure of overlap of strokes. An overlap is defined as two (non-consecutive) strokes that intersect occurring with a relatively long time gap there between, e.g., a time gap representing the time period to write a page. A page-flip takes at least a few seconds, and each page of notes typically requires in the order of tens of seconds, or at least a minute. One version includes determining time separation between strokes for e region in a page, and determining the relative number of stroke overlaps. An overlap of two strokes occurs when two strokes with long time gaps are crossing over each other at least one point in the strokes. A long time gap is in the order of seconds, e.g., 3 seconds. Too long a time period may produce too high a false negative rate, which is undesirable.

[0095] One embodiment of determining the stroke-overlap indicator includes setting a threshold for the number of time-separated strokes are overlapping and identified. The probability of an intended new-page increases as more overlapping stroked are detected. In one version, the strokes are first time-separated and then page-separated.

[0096] In one embodiment, the probability of correctly indicating a page break using stroke overlap is 0.75. In another improved embodiment, different probabilities are assigned for different relative numbers of overlaps, e.g., different probabilities in the range 0.7 to 0.9. One version has three such probabilities, stored as parameters, e.g., stored in the parameter portion 253 of receiver memory 21 1 .

Region overlap indicator

[0097] A bounding box is the smallest rectangle in which a set of strokes of a page exist. One method of determining a content-based indicator is to group sets of strokes by non-overlapping time periods, and determining a region, e.g., the bounding box for each such time-grouped set. In one example, the bounding box of a set is specified by the coordinates of diagonally opposite corners of the rectangle, e.g., the top-most left and bottom-most right coordinates of a time- grouped stroke set. If two such sets overlap, the probability is high that a full page, unintended overlap is occurring. In one embodiment, a threshold of 50% overlap triggers the indicator indicating a page-flip, and in another version, a different percentage.

[0098] In one embodiment, the probability of correctly indicating a page break using bounding box overlap is assigned to be 0.75. In another improved embodiment, different probabilities are assigned for different relative percentage overlaps, e.g. , different probabilities in the range 0.7 to 0.9. One version has three such probabilities, stored as parameters, e.g., stored in the parameter portion 253 of receiver memory 21 1 .

Combining the indications

[0099] In one embodiment, the overall probability measure of a page is determined as a weighted average of the all indications used, where the weightings are the individual probability measures assigned to the indicator. [00100] Consider the case of N page-flip indications being used, N≥2. Each indication has two values, 0 or 1 , 1 indicative of the indicator indicating a page-flip. Denote the N indicators by D1 , D2, D3, DN, and let P1 , P2, PN, respectively be the assigned probability of the indicator being correct.

[00101] According to one embodiment, The overall probability measure, denoted P of a page- flip occurring at a location-in-time is the weighted average of the indications: ∑₌₁ Pi*Di

[00102] P =

∑[L_{l P}i

where ^* indicated multiplication.

[00103] One embodiment places a page-flip at the location-in-time when the weighted average P is above a probability threshold. In one embodiment, the probability threshold is selected to be low, e.g., 0.4 in order to avoid false negatives. While this may increase the number of false positives, false positives are not as serious as false negatives, because erroneously placed page flips can easily be deleted long after the fact, while it may be difficult to determine page flips after the saving of the stroke data. In some embodiments, alternate values for the probability threshold may be stored, e.g., in the parameter portion 253 of receiver memory 21 1 . [00104] An alternate embodiment uses a Bayesian decision method to determine page-flip events from the plurality of page-flip indicators. The method includes pre-assigning each (physical-sensor-derived and/or content-based) indicator a likelihood (or confidence level) that the indicator is correct, i.e., the probability of correct positive and negative indications, and assigning to each indicator a probabilities of a false positive and a false negative. The method includes, for each assigned probability, assigning a "cost" "bias" or weighting of the decision. Many other methods are known for classifying (into two classes) using a plurality of indicators;

[00105] By combining two or more page-flip indicators, the confidence for an assumed new- page can increase. By combining three or more indicators, including at least one sensor-based indicator, it may be possible to be nearly 100% confident that a new page was desired but forgotten.

[00106] FIG. 4 shows a receiver 1 1 1 , a pen 103, and a charging bracket with pen-stroke determining capability, into which elements described herein below may be included such that the combination operates as an embodiment of the invention.

[00107] While the description above is for a device/system that includes an interactive pen- stroke capture system that uses active pens, the inventive method and system can easily be applied to interactive pen capture systems as well as interactive tablets with styluses, touch based tablets and other non-keyboard character based methods of input. Additional data such as voice recording can be captured and synced to the pen data stream.

[00108] One embodiment includes handwriting recognition of the strokes once entered. Many methods of recognizing hand-written characters are known in the art, and any such method can be used. Character recognition has been studied for more than 40 years, and several commercial software packages are available. For a review of some modern techniques, see, for example, Aini Najwa Azmi, Dewi Nasien, Siti Mariyam Shamsuddin, "A review on handwritten character and numeral recognition for Roman, Arabic, Chinese and Indian scripts," International Journal of advanced studies in Computers, Science & Engineering (IJASCSE), Volume 2, Issue 4, 2013, available online on 2015-09-04 at

arxiv~dot~org/ftp/arxiv/papers/1308/1308~dot~4902~dot~pdfm where "~dot~" is used to denote the period (".") character in the actual URL. Summarizing the method

[00109] Thus, one embodiment of the invention is a method of operating the system (the apparatus) 100 that captures pen strokes. FIG. 5 shows a flowchart of the method. In step 503, the method includes receiving acoustic signals in the at least two acoustic sensors 1 13, 1 15 of the receiver 1 1 1 placed on or close to the edge of the surface such that the sensors are located at the pre-defined locations relative to the surface and operative to sense signals transmitted from the pen 103 that includes an acoustic-energy transmitter 123 that transmits the acoustic signals when the pen is in an active area 107 of the surface adjacent to the pre-defined locations. In step 505, the method includes determining from the received acoustic signals locations of the pen in the active area on the surface adjacent to the pre-defined locations and determining time information for the determined locations. In step 507, the method includes storing the determined locations and time information. In step 509, the method includes accepting or determining 509 a plurality of indicators of page-flip, and in step 51 1 , the method includes combining 51 1 the plurality of indicators to automatically determine a page-flip event, including detecting that there is a page-flip and determining the location-in-time of the page-flip to form page-flip information for each detected page-flip. The method further includes, in step 513, storing the page-flip information 513.

[00110] Some versions of the method further include step 515 of sending the determined locations, time information, and page-flip information to the host device 150.

General

[00111] Unless specifically stated otherwise, as apparent from the following discussions, it is appreciated that throughout the specification discussions utilizing terms such as "processing," "computing," "calculating," "determining" or the like, refer to the action and/or processes of a host device or computing system, or similar electronic computing device, that manipulate and/or transform data represented as physical, such as electronic, quantities into other data similarly represented as physical quantities.

[00112] In a similar manner, the term "processor" may refer to any device or portion of a device that processes electronic data, e.g., from registers and/or memory to transform that electronic data into other electronic data that, e.g., may be stored in registers and/or memory.

[00113] The methodologies described herein are, in one embodiment, performable by one or more processors that accept machine-readable instructions, e.g., as firmware or as software, that when executed by one or more of the processors carry out at least one of the methods described herein. In such embodiments, any processor capable of executing a set of

instructions (sequential or otherwise) that specify actions to be taken may be included. Thus, one example is a programmable DSP device. Another is the CPU of a microprocessor or other computer-device, or the processing part of a larger ASIC. A processing system may include a memory subsystem including main RAM and/or a static RAM, and/or ROM. A bus subsystem may be included for communicating between the components. The processing system further may be a distributed processing system with processors coupled wirelessly or otherwise, e.g., by a network. If the processing system requires a display, such a display may be included. The processing system in some configurations may include a sound input device, a sound output device, and a network interface device. The memory subsystem thus includes a machine- readable non-transitory medium that is coded with, i.e., has stored therein a set of instructions to cause performing, when executed by one or more processors, one of more of the methods described herein. Note that when the method includes several elements, e.g., several steps, no ordering of such elements is implied, unless specifically stated. The instructions may reside in the hard disk, or may also reside, completely or at least partially, within the RAM and/or other elements within the processor during execution thereof by the system. Thus, the memory and the processor also constitute the non-transitory machine-readable medium with the instructions.

[00114] Furthermore, a non-transitory machine-readable medium may form a software product. For example, it may be that the instructions to carry out some of the methods, and thus form all or some elements of the inventive system or apparatus, may be stored as firmware. A software product may be available that contains the firmware, and that may be used to "flash" the firmware.

[00115] Note that while some diagram(s) only show(s) a single processor and a single memory that stores the machine-readable instructions, those in the art will understand that many of the components described above are included, but not explicitly shown or described in order not to obscure the inventive aspect. For example, while only a single machine is illustrated, the term "machine" shall also be taken to include any collection of machines that individually or jointly execute a set (or multiple sets) of instructions to perform any one or more of the methodologies discussed herein.

[00116] Thus, one embodiment of each of the methods described herein is in the form of a non-transitory machine-readable medium coded with, i.e., having stored therein a set of instructions for execution on one or more processors, e.g., one or more processors that are part of the receiver forming a pen stroke capture system. [00117] Note that, as is understood in the art, a machine with application-specific firmware for carrying out one or more aspects of the invention becomes a special purpose machine that is modified by the firmware to carry out one or more aspects of the invention. This is different than a general purpose processing system using software, as the machine is especially configured to carry out the one or more aspects. Furthermore, as would be known to one skilled in the art, if the number the units to be produced justifies the cost, any set of instructions in combination with elements such as the processor may be readily converted into a special purpose ASIC or custom integrated circuit. Methodologies and software have existed for years that accept the set of instructions and particulars of, for example, the processing engine 131 , and automatically or mostly automatically great a design of special-purpose hardware, e.g., generate instructions to modify a gate array or similar programmable logic, or that generate an integrated circuit to carry out the functionality previously carried out by the set of instructions. Thus, as will be appreciated by those skilled in the art, embodiments of the present invention may be embodied as a method, an apparatus such as a special purpose apparatus, an apparatus such as a data DSP device plus firmware, or a non-transitory machine-readable medium. The machine-readable carrier medium carries host device readable code including a set of instructions that when executed on one or more processors cause the processor or processors to implement a method.

Accordingly, aspects of the present invention may take the form of a method, an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form a computer program product on a non-transitory machine-readable storage medium encoded with machine- executable instructions. [00118] Reference throughout this specification to "one embodiment" or "an embodiment" means that a particular feature, structure or characteristic described in connection with the embodiment is included in at least one embodiment of the present invention. Thus,

appearances of the phrases "in one embodiment" or "in an embodiment" in various places throughout this specification are not necessarily all referring to the same embodiment, but may. Furthermore, the particular features, structures or characteristics may be combined in any suitable manner, as would be apparent to one of ordinary skill in the art from this disclosure, in one or more embodiments.

[00119] Similarly it should be appreciated that in the above description of example

embodiments of the invention, various features of the invention are sometimes grouped together in a single embodiment, figure, or description thereof for the purpose of streamlining the disclosure and aiding in the understanding of one or more of the various inventive aspects. This method of disclosure, however, is not to be interpreted as reflecting an intention that the claimed invention requires more features than are expressly recited in each claim. Rather, as the following claims reflect, inventive aspects lie in less than all features of a single foregoing disclosed embodiment. Thus, the claims following the Detailed Description are hereby expressly incorporated into this Detailed Description, with each claim standing on its own as a separate embodiment of this invention. [00120] Furthermore, while some embodiments described herein include some but not other features included in other embodiments, combinations of features of different embodiments are meant to be within the scope of the invention, and form different embodiments, as would be understood by those in the art. For example, in the following claims, any of the claimed embodiments can be used in any combination.

[00121] Furthermore, some of the embodiments are described herein as a method or combination of elements of a method that can be implemented by a processor of a host device system or by other means of carrying out the function. Thus, a processor with the necessary instructions for carrying out such a method or element of a method forms a means for carrying out the method or element of a method. Furthermore, an element described herein of an apparatus embodiment is an example of a means for carrying out the function performed by the element for the purpose of carrying out the invention.

[00122] In the description provided herein, numerous specific details are set forth. However, it is understood that embodiments of the invention may be practiced without these specific details. In other instances, well-known methods, structures and techniques have not been shown in detail in order not to obscure an understanding of this description.

[00123] As used herein, unless otherwise specified the use of the ordinal adjectives "first", "second", "third", etc., to describe a common object, merely indicate that different instances of like objects are being referred to, and are not intended to imply that the objects so described must be in a given sequence, either temporally, spatially, in ranking, or in any other manner.

[00124] All publications, patents, and patent applications cited herein are hereby incorporated by reference.

[00125] Any discussion of prior art in this specification should in no way be considered an admission that such prior art is widely known, is publicly known, or forms part of the general knowledge in the field.

[00126] In the claims below and the description herein, any one of the terms comprising, comprised of or which comprises is an open term that means including at least the

elements/features that follow, but not excluding others. Thus, the term comprising, when used in the claims, should not be interpreted as being limitative to the means or elements or steps listed thereafter. For example, the scope of the expression a device comprising A and B should not be limited to devices consisting only of elements A and B. Any one of the terms including or which includes or that includes as used herein is also an open term that also means including at least the elements/features that follow the term, but not excluding others. Thus, including is synonymous with and means comprising. [00127] Similarly, it is to be noticed that the term coupled, when used in the claims, should not be interpreted as being limitative to direct connections only. The terms "coupled" and

"connected," along with their derivatives, may be used. It should be understood that these terms are not intended as synonyms for each other. Thus, the scope of the expression a device A coupled to a device B should not be limited to devices or systems wherein an output of device A is directly connected to an input of device B. It means that there exists a path between an output of A and an input of B which may be a path including other devices or means. "Coupled" may mean that two or more elements are either in direct physical or electrical contact, or that two or more elements are not in direct contact with each other but yet still co-operate or interact with each other.

[00128] Thus, while there has been described what are believed to be the preferred

embodiments of the invention, those skilled in the art will recognize that other and further modifications may be made thereto without departing from the spirit of the invention, and it is intended to claim all such changes and modifications as fall within the scope of the invention. For example, any formulas given above are merely representative of procedures that may be used. Functionality may be added or deleted from the block diagrams and operations may be interchanged among functional blocks. Steps may be added or deleted to methods described within the scope of the present invention.

[00129] Note that the claims attached to this description form part of the description, so are incorporated by reference into the description, each claim forming a different set of one or more embodiments.

A List of Some of the Embodiments

[00130] Different embodiments of the invention include the following sets of embodiments:

[00131] Embodiment set 1 is an apparatus (100) configured to capture pen strokes, including automatically detecting a page-flip, the apparatus comprising:

[00132] a receiver (1 1 1 ) configured to be placed on or close to an edge of a surface (105), the receiver including:

[00133] at least two acoustic sensors (1 13, 1 15) located at pre-defined locations relative to the surface and arranged to sense acoustic signals transmitted from a pen (103) in an active area (107) of the surface adjacent to the pre-defined locations, the pen including at least one acoustic-energy transmitter (123) configured to transmit acoustic signals detectable by the acoustic sensors; and

[00134] a processing engine (131 ) coupled to the acoustic sensors and configured to determine and store locations and time information of the pen in the active area an active area on the surface adjacent to the pre-defined locations; [00135] wherein the receiver alone, or in combination with a host device (150) to which the receiver is coupled, is operative to:

[00136] accept or determine a plurality of indicators of page-flip, and

[00137] combine the plurality of page-flip indicators to automatically determine a page-flip event, determining a page-flip event including detecting that there is a page-flip and determining the location-in-time of the page-flip to form page-flip information for each detected page-flip; and

[00138] store the page-flip information.

[00139] Embodiment set 2 is an apparatus as recited in Embodiment set 1 , further comprising:

[00140] an interface to couple the receiver with the host device (150), the host device including a host processor (221 ) and a memory (223),

[00141] wherein the receiver is operative, when coupled to the host device using the

interface, to send determined locations, time information, and page-flip information to the host device.

[00142] Embodiment set 3 is the set of embodiments each an apparatus as recited in any one of Embodiment sets 1 to 2, wherein the receiver comprises one or more physical sensors to determine respective one or more physical-sensor-derived page-flip indicators, and wherein the plurality of indicators includes the one or more physical-sensor-derived page-flip indicators.

[00143] Embodiment set 4 is the set of embodiments each an apparatus as recited in any one of Embodiment sets 1 to 3, wherein the processing engine of the receiver includes a processor and a memory, wherein the memory includes instructions that when executed cause the determining and the storing of the locations and time information, the receiving and/or determining of the plurality of page-flip indicators that are accepted and/or determined by the receiver, and, alone or in combination with the host device, and any steps of the combining carried out in the receiver.

[00144] Embodiment set 5 is the set of embodiments each an apparatus as recited in any one of Embodiment sets 1 to 4, wherein the pen includes an electromagnetic energy transmitter configured to transmit electromagnetic pulses, wherein the acoustic energy transmitter is operative to transmit acoustic pulses, wherein the receiver includes a sensor of electromagnetic pulses, and wherein the transmitting of the electromagnetic pulses is synchronized with the transmitting of the acoustic pulses. [00145] Embodiment set 6 is the set of embodiments each an apparatus as recited in

Embodiment set 5, wherein the electromagnetic pulses are infrared pulses and the acoustic pulses are ultrasound pulses.

[00146] Embodiment set 7 is the set of embodiments each an apparatus as recited in any one of Embodiment sets 1 to 6, wherein each page flip indicator is assigned a probability measure of correctly indicating a page-flip, and wherein the combining of the plurality of page-flip indicators uses the respective probability measures as weightings to determine a weighted average probability measure, and wherein the detecting that there is a page-flip and determining the location-in-time of the page-flip includes ascertaining whether the determined average probability measure exceeds a threshold.

[00147] Embodiment set 8 is the set of embodiments each an apparatus as recited in any one of Embodiment sets 1 to 6, wherein each page flip indicator is assigned a probability measure of being correct, and wherein the combining of the plurality of page-flip indicators uses the respective probability measures to perform a statistical detection method. [00148] Embodiment set 9 is the set of embodiments each an apparatus as recited in any one of Embodiment sets 7 to 8, wherein the assigning of probability measures and the combining of the plurality of page-flip indicators are biased to prefer false positives to false negatives.

[00149] Embodiment set 10 is an apparatus as recited in Embodiment set 8, wherein the assigning of probability measures and the statistical detection method are selected to minimize the probability of false negatives.

[00150] Embodiment set 11 is a method of operating an apparatus (100) that captures pen strokes, the method comprising:

[00151] receiving (503) acoustic signals in at least two acoustic sensors (1 13, 1 15) of a

receiver (1 1 1 ) placed on or close to an edge of a surface such that the sensors are located at pre-defined locations relative to the surface and operative to sense signals transmitted from a pen (103) that includes an acoustic-energy transmitter (123) that transmits the acoustic signals when the pen is in an active area (107) of the surface adjacent to the pre-defined locations;

[00152] determining (505) from the received acoustic signals locations of the pen in the

active area on the surface adjacent to the pre-defined locations and determining time information for the determined locations;

[00153] storing (507) the determined locations and time information;

[00154] accepting or determining (509) a plurality of indicators of page-flip; [00155] combining (51 1 ) the plurality of indicators to automatically determine a page-flip event, including detecting that there is a page-flip and determining the location-in-time of the page-flip to form page-flip information for each detected page-flip; and

[00156] storing (513) the page-flip information. [00157] Embodiment set 12 is a method as recited in Embodiment set 1 1 , further comprising sending (515) the determined locations, time information, and page-flip information to a host device (150) of the apparatus (100).

[00158] Embodiment set 13 is the set of embodiments each a method as recited in any one of Embodiment sets 1 1 to 12, wherein the receiver comprises one or more physical sensors to determine respective one or more physical-sensor-derived page-flip indicators, and wherein the plurality of indicators includes the one or more physical-sensor-derived page-flip indicators.

[00159] Embodiment set 14 is the set of embodiments each a method as recited in

Embodiment set 13, wherein the one or more physical sensors include an accelerometer (1 16).

[00160] Embodiment set 15 is the set of embodiments each a method as recited in any one of Embodiment sets 13 to 14, wherein the one or more physical sensors include a switch (1 18) requiring a force less than the weight of the receiver force to press, and located at a bottom area (301 ) of a housing for the receiver, the bottom area (301 ) making contact with the surface when the apparatus is operative to receive the acoustic signals and determine the locations.

[00161] Embodiment set 16 is the set of embodiments each a method as recited in any one of Embodiment sets 13 to 15, wherein the one or more physical sensors include a proximity sensor (231 ) sensing the proximity of the receiver to the surface containing the active area.

[00162] Embodiment set 17 is the set of embodiments each a method as recited in any one of Embodiment sets 13 to 16, wherein the one or more physical sensors include a set of one or two light sensors (233, 315) positioned and operative to provide an indication of light from the surface containing the active area or to provide an indication of the difference between light from the surface containing the active area and the ambient light surrounding the receiver.

[00163] Embodiment set 18 is the set of embodiments each a method as recited in any one of Embodiment sets 1 1 to 17, further comprising calculating unintended content overlap or time gaps from the determined positions and time information to determine one or more content- based page-flip indicators that each detects a page-flip event and is included in the plurality of indicators of page-flip.

[00164] Embodiment set 19 is the set of embodiments each a method as recited in

Embodiment set 18, wherein the determining of one or more content-based page-flip indicators includes determining a time-slicing based indicator based on determining from the content long time gaps between two pen strokes.

[00165] Embodiment set 20 is the set of embodiments each a method as recited in any one of Embodiment sets 18 to 19, wherein the wherein the determining of one or more content-based page-flip indicators includes determining a stroke overlap indicator based on determining a measure of overlap of strokes, wherein an overlap is defined as two non-consecutive strokes that intersect and that occur with a relatively long time gap therebetween.

[00166] Embodiment set 21 is the set of embodiments each a method as recited in any one of Embodiment sets 18 to 20, wherein the determining of one or more content-based page-flip indicators includes determining a region overlap indicator based on grouping sets of strokes by non-overlapping time periods, determining a region for each set, and determining whether the determined regions overlap.

[00167] Embodiment set 22 is a method as recited in any one of Embodiment sets 1 1 to 21 , wherein the pen includes an electromagnetic energy transmitter configured to transmit electromagnetic pulses, wherein the acoustic energy transmitter is operative to transmit acoustic pulses, wherein the receiver includes a sensor of electromagnetic pulses, and wherein the transmitting of the electromagnetic pulses is synchronized with the transmitting of the acoustic pulses.

[00168] Embodiment set 23 is the set of embodiments each a method as recited in

Embodiment set 22, wherein the electromagnetic pulses are infrared pulses and the acoustic pulses are ultrasound pulses.

[00169] Embodiment set 24 is the set of embodiments each a method as recited in any one of Embodiment sets 1 1 to 23, further comprising assigning to each page flip indicator a probability measure of correctly indicating a page-flip, [00170] wherein the combining of the plurality of page-flip indicators uses the respective probability measures as weightings to determine a weighted average probability measure, and wherein the detecting that there is a page-flip and determining the location-in-time of the page-flip includes ascertaining whether the determined average probability measure exceeds a threshold. [00171] Embodiment set 25 is a method as recited in any one of Embodiment sets 1 1 to 23, further comprising assigning to each page flip indicator a probability measure of correctly indicating a page-flip, wherein the combining of the plurality of page-flip indicators uses the respective probability measures to perform a statistical detection method. [00172] Embodiment set 26 is the set of embodiments each a method as recited in any one of Embodiment sets 24 to 25, wherein the assigning of probability measures and the combining of the plurality of page-flip indicators are biased to prefer false positives to false negatives.

[00173] Embodiment set 27 is the set of embodiments each a method as recited in

Embodiment set 25, wherein the assigning of probability measures and the statistical detection method are selected to minimize the probability of false negatives.

[00174] Embodiment set 28 is the set of embodiments each a non-transitory machine- readable medium coded with instructions, that when execute, carry out a method as recited in any one of Embodiment sets 1 1 to 23.

Claims

CLAIMS What is claimed is:

1 . An apparatus (100) configured to capture pen strokes, including automatically detecting a page-flip, comprising: a receiver (1 1 1 ) configured to be placed on or close to an edge of a surface (105), the receiver including: at least two acoustic sensors (1 13, 1 15) located at pre-defined locations relative to the surface and arranged to sense acoustic signals transmitted from a pen (103) in an active area (107) of the surface adjacent to the pre-defined locations, the pen including at least one acoustic-energy transmitter (123) configured to transmit acoustic signals detectable by the acoustic sensors; and a processing engine (131 ) coupled to the acoustic sensors and configured to determine and store locations and time information of the pen in the active area an active area on the surface adjacent to the pre-defined locations, wherein the receiver alone, or in combination with a host device (150) to which the receiver is coupled, is operative to: accept or determine a plurality of indicators of page-flip, and combine the plurality of page-flip indicators to automatically determine a page-flip event, determining a page-flip event including detecting that there is a page-flip and determining the location-in-time of the page-flip to form page-flip information for each detected page-flip; and store the page-flip information.

2. The apparatus of claim 1 , further comprising: an interface to couple the receiver with the host device (150), the host device including a host processor (221 ) and a memory (223), wherein the receiver is operative, when coupled to the host device using the interface, to send determined locations, time information, and page-flip information to the host device.

3. The apparatus of any one of claims 1 to 2, wherein the receiver comprises one or more physical sensors to determine respective one or more physical-sensor-derived page-flip indicators, and wherein the plurality of indicators includes the one or more physical-sensor- derived page-flip indicators.

4. The apparatus of any one of claims 1 to 3, wherein the processing engine of the receiver includes a processor and a memory, wherein the memory includes instructions that when executed cause the determining and the storing of the locations and time information, the receiving and/or determining of the plurality of page-flip indicators that are accepted and/or determined by the receiver, and, alone or in combination with the host device, and any steps of the combining carried out in the receiver.

5. The apparatus of any one of claims 1 to 4, wherein the pen includes an electromagnetic energy transmitter configured to transmit electromagnetic pulses, wherein the acoustic energy transmitter is operative to transmit acoustic pulses, wherein the receiver includes a sensor of electromagnetic pulses, and wherein the transmitting of the electromagnetic pulses is synchronized with the transmitting of the acoustic pulses.

6. The apparatus of claim 5, wherein the electromagnetic pulses are infrared pulses and the acoustic pulses are ultrasound pulses.

7. The apparatus of any one of claims 1 to 6, wherein each page flip indicator is assigned a probability measure of correctly indicating a page-flip, and wherein the combining of the plurality of page-flip indicators uses the respective probability measures as weightings to determine a weighted-average probability measure, and wherein the detecting that there is a page-flip and determining the location-in-time of the page-flip includes ascertaining whether the determined weighted-average probability measure exceeds a threshold.

8. The apparatus of any one of claims 1 to 6, wherein each page flip indicator is assigned a probability measure of being correct, and wherein the combining of the plurality of page-flip indicators uses the respective probability measures to perform a statistical detection method.

9. The apparatus of any one of claims 7 to 8, wherein the assigning of probability measures and the combining of the plurality of page-flip indicators are biased to prefer false positives to false negatives.

10. The apparatus of claim 8, wherein the assigning of probability measures and the statistical detection method are selected to minimize the probability of false negatives.

1 1. A method of operating an apparatus (100) that captures pen strokes, the method

comprising: receiving (503) acoustic signals in at least two acoustic sensors (1 13, 1 15) of a receiver (1 1 1 ) placed on or close to an edge of a surface such that the sensors are located at pre-defined locations relative to the surface and operative to sense signals transmitted from a pen (103) that includes an acoustic-energy transmitter (123) that transmits the acoustic signals when the pen is in an active area (107) of the surface adjacent to the pre-defined locations; determining (505) from the received acoustic signals locations of the pen in the active area on the surface adjacent to the pre-defined locations and determining time information for the determined locations; storing (507) the determined locations and time information; accepting or determining (509) a plurality of indicators of page-flip; combining (51 1 ) the plurality of indicators to automatically determine a page-flip event, including detecting that there is a page-flip and determining the location-in-time of the page-flip to form page-flip information for each detected page-flip; and storing (513) the page-flip information.

12. The method of claim 1 1 , further comprising sending (515) the determined locations, time information, and page-flip information to a host device (150) of the apparatus (100).

13. The method of any one of claims 1 1 to 12, wherein the receiver comprises one or more physical sensors to determine respective one or more physical-sensor-derived page-flip indicators, and wherein the plurality of indicators includes the one or more physical-sensor- derived page-flip indicators.

14. The method of claim 13, wherein the one or more physical sensors include an

accelerometer (1 16).

15. The method of any one of claims 13 to 14, wherein the one or more physical sensors

include a switch (1 18) requiring a force less than the weight of the receiver force to press, and located at a bottom area (301 ) of a housing for the receiver, the bottom area (301 ) making contact with the surface when the apparatus is operative to receive the acoustic signals and determine the locations.

16. The method of any one of claims 13 to 15, wherein the one or more physical sensors

include a proximity sensor (231 ) sensing the proximity of the receiver to the surface containing the active area.

17. The method of any one of claims 13 to 16, wherein the one or more physical sensors

include a set of one or two light sensors (233, 315) positioned and operative to provide an indication of light from the surface containing the active area or to provide an indication of the difference between light from the surface containing the active area and the ambient light surrounding the receiver.

18. The method of any one of claims 1 1 to 17, further comprising calculating unintended content overlap or time gaps from the determined positions and time information to determine one or more content-based page-flip indicators that each detects a page-flip event and is included in the plurality of indicators of page-flip.

19. The method of claim 18, wherein the determining of one or more content-based page-flip indicators includes determining a time-slicing based indicator based on determining from the content long time gaps between two pen strokes.

20. The method of any one of claims 18 to 19, wherein the wherein the determining of one or more content-based page-flip indicators includes determining a stroke overlap indicator based on determining a measure of overlap of strokes, wherein an overlap is defined as two non-consecutive strokes that intersect and that occur with a relatively long time gap therebetween.

21. The method of any one of claims 18 to 20, wherein the determining of one or more content- based page-flip indicators includes determining a region overlap indicator based on grouping sets of strokes by non-overlapping time periods, determining a region for each set, and determining whether the determined regions overlap.

22. The method of any one of claims 1 1 to 21 , wherein the pen includes an electromagnetic energy transmitter configured to transmit electromagnetic pulses, wherein the acoustic energy transmitter is operative to transmit acoustic pulses, wherein the receiver includes a sensor of electromagnetic pulses, and wherein the transmitting of the electromagnetic pulses is synchronized with the transmitting of the acoustic pulses.

23. The method of claim 22, wherein the electromagnetic pulses are infrared pulses and the acoustic pulses are ultrasound pulses.

24. The method of any one of claims 1 1 to 23, further comprising assigning to each page flip indicator a probability measure of correctly indicating a page-flip, wherein the combining of the plurality of page-flip indicators uses the respective probability measures as weightings to determine a weighted average probability measure, and wherein the detecting that there is a page-flip and determining the location-in-time of the page-flip includes ascertaining whether the determined average probability measure exceeds a threshold.

25. The method of any one of claims 1 1 to 23, further comprising assigning to each page flip indicator a probability measure of correctly indicating a page-flip, wherein the combining of the plurality of page-flip indicators uses the respective probability measures to perform a statistical detection method.

26. The method of any one of claims 24 to 25, wherein the assigning of probability measures and the combining of the plurality of page-flip indicators are biased to prefer false positives to false negatives.

27. The method of claim 25, wherein the assigning of probability measures and the statistical detection method are selected to minimize the probability of false negatives.

28. A non-transitory machine-readable medium coded with instructions, that when execute, carry out a method as recited in any one of method claims 1 1 to 23.