CN101116123A - Hand-held sensor devices for use with printed material - Google Patents

Abstract

Interactive printed material and sensor apparatus for use therewith is described. The sensor responds to characteristics of printing on the printed material, for example the degree of infrared reflectance or absorption. The printed material has intelligible components and non-intelligible components. By configuring the printed material and/or the internal programming in the sensor device appropriately, the sensor device can be calibrated to the printed material automatically as the sensor device is applied to the printed material as part of a structured intellectually mediated interaction between the two. In one example, the sensor device is run across a sequence of pre-printed areas specifically designed to enable the values of the property to be presented in a specific order enabling sophisticated calibration and programming functions to be implemented.

Description

Handheld sensor device for use with printed materials

Technical Field

The present invention relates to a hand-held sensor device for use with printed material.

Background

In the field of education and entertainment, there have been many proposals to provide books and the like and pen, stick and the like devices associated therewith for use in conjunction with books, and some commercial products have been successfully produced and put on the market. By having the pen or wand detect the nature of the page, an appropriate interaction can be established between them, in which regard such an interaction can be of entertainment or educational value.

WO-A-88/05951 and WO-A-83/02842 describe systems of this type. While the disclosure states that other interaction mechanisms may be used, the description specifically discloses a hand-held sensor device configured to detect infrared reflectance or absorbance of a portion of a printed material proximate to a head of the sensor device.

For such a combination of sensor device and printed material to work effectively with each other, considerable care must be taken in the printing of the printed material and in the calibration of the sensor device. Sensor devices that are only suitable for working with a single sheet of printed material are completely commercially unsatisfactory. Indeed, what is needed is a sensor device that can handle many different sheets of printed material that have indeed been printed at different times and in different places, even with different printing methods. This is difficult to do in practice, especially in the nature and range of the interaction activity, which can be considered to be between the sensor device and the printed material, resulting in considerable limitations, since the number of different values of the properties of the printed material, for example the reflectivity of infrared radiation, is limited to very small numbers, typically 3 or 4.

WO 2005/013237-a relates to improvements in the use of interactive printed materials with sensor devices and discloses that constituent sensor devices can be operated in a calibration mode, even in a recalibration mode, to account for variations between different printed materials. Although in this earlier application some details are disclosed as to how this calibration or recalibration may be carried out, it is pointed out that it relies on the user interacting the sensor device with the printed material to consciously apply the sensor device to a number of differently marked areas in sequence. This is a disadvantage, since the prior task of consciously calibrating the sensor device itself, without irritation or fun to the user, may be completely overlooked, which may not immediately produce a harmful result, but may lead to instability, errors, and the system eventually failing to function properly as designed, for example, during play or when a mental test paper is gradually run.

Disclosure of Invention

According to the invention, the printed material and/or the internal programming of the sensor device is arranged to ensure that the sensor device is calibrated to the printed material, which calibration can be done automatically as the sensor device is placed on the printed material as part of a structural, intelligently transmitted interaction between the sensor device and the printed material. As noted with reference to the above published disclosure, the interacting sensor devices and printed materials may be configured in a myriad of ways to provide educational and entertainment materials. In all these ways, the printed material comprises intelligent print, i.e. text and/or pictorial matter, which is readable and viewable by the user, and meaningful and non-intelligent material, the presence or absence of which, or the degree to which the latter appears, can be picked up by the sensor. In short, the printed page contains information that is visible and intelligent to a human observer, while other printed information is "invisible".

One way in which the present invention can be implemented is to design an interactive game that requires the user to place the sensors on a given number of areas of the printed material that have different responses (i.e., different responses involving only the sensors). For example, if the sensor is programmed to discriminate between six different infrared absorption levels, the game may be configured to encourage the user to apply the sensor to six different infrared-reflective regions in succession.

One form of attractive game is a track or maze that must be traversed by the user. If it contains a recognizable "start" and the initial instruction is, for example, "start the game, step up one step at a time", then the steps may be printed with different ir reflectivities. Another is to provide some form of track that indicates that the user is driven to sequentially pass the sensor head over different reflective areas as indicated graphically, for example by passing the sensor head over a bridge or crosswalk as indicated graphically into a "playyard".

Further, it is possible to provide sensor units that do not operate in a product-interacting manner, for example, that do not react to printed speech bubbles (which may or may not have any text printed therein) until the step of applying the sensor to different reflection areas has been completed. This may include, for example, simply moving the sensor randomly around the page.

Another solution, as will be described below, is to make such a device quite advantageous, in particular for its operation in complex game play or for use in quiz book or repeated test practice applications, that is to say to program the sensor device to record different values of the sensed properties of the printed material and then to process the records of the different values to extract information therefrom about the different values themselves.

Thus according to a particular feature of the invention there is provided an interaction device comprising printed material and sensor means, the sensor means being adapted to respond to a characteristic of the printing on the printed material, wherein the printing on the printed material has a intellectual component and a non-intellectual component, the latter being sensed by the sensor means, and wherein the sensor means senses different values of a property on the printed non-intellectual image, and the device includes a processing unit adapted to follow the sensor means over a sufficiently large number of different (non-intellectual) printed areas of the printed material, applied in succession in sequence, to identify a maximum value and a minimum value of the sensed property.

Operating in this manner, the specific levels of maximum and minimum values correspond to actual values that are all different and that can be set by the sensor device itself as the specific levels, after which the sensor device is programmed for identification, i.e. if the sensed property is on or within a preset tolerance band of the levels determined by the above maximum and minimum analysis, the sensor will identify the print as having the property on the specific level.

The level detection and self-calibration or recalibration thus obtainable in the sensor device is valuable as it enables the device to adapt to the printed material and thus to the printed material produced by different producers and at different times accordingly. Such recalibration can occur regardless of the sequence of the various levels detected by the sensor device. However, as the discussion below states, it is particularly preferable to give those levels in a limited fixed sequence, since those levels enable the sensor device not only to calibrate itself, but more importantly to change the mode of operation from a default mode or some earlier mode of operation to a new mode of operation. Having read the code, it can also say the word or phrase corresponding to the chart indication.

The use of complex areas of interacting printed material and sensor means for mental testing or early learning books, in which the rules of interaction between the sensor pen and the printed material vary, for example, from page to page, or from one printed book or loose-leaf practice question to the next (works set). In a sensor pen of conventional construction comprising various different programmed operation modes, each of which can be selected for driving the pen's mode of operation, it is straightforward to provide a very realistic amount of stored program material.

The application of the pen to the areas of the printed material in the desired sequence can be ensured by the above-indicated technique, for example by the form of some graphically or graphically indicated driven tracks with individual printed areas in the appropriate sequence, to which the user applies the pen according to instructions or questions and answers on the page by providing all the sequences buried in the sequence area.

As generally described above, sequential input of different values may allow the sensor device to self-calibrate. However, there are games that do not require calibration at all. These games work by comparison only. For use in such games, the sensor module is preset with a threshold value which can therefore distinguish between "correct" and "incorrect". It is factory set at a level in the software that has taken into account all readings and other tolerances-between 5% and 10% soot. Any area with carbon black content greater than this threshold is considered "correct" and recorded. Then, 4 or 5 different levels are used for correct answers, with the various games to be played including sets (sets), sorting, matching, tracking, and maze, which analyze simple sequences and give appropriate responses. A similar scheme (a threshold between "correct" and "incorrect") can be used at the start of a repeat test, but this time the correct answer is calibrated after the entire set of answers has been collected.

Drawings

Specific examples of how the invention can be practiced will now be described with reference to the accompanying drawings. In these figures:

FIG. 1 is a representation of a play board and a frog for playing a game;

FIG. 2 is a schematic representation of a track printed on a game board;

FIG. 3 is a graphical representation of the sensed value response as the sensor passes along the track of FIG. 2;

FIG. 4 is a block circuit diagram illustrating circuitry useful in the sensor device of the present invention; and

fig. 5 is a circuit diagram illustrating a preferred form of circuit for use in the sensor device of the present invention.

Detailed Description

Referring now first to fig. 1, this figure shows a frog on one side of a printed image of 5 water lily leafs in a pond. The sensor unit is encapsulated in a plastic frog that gives an audio signal when the frog is placed on any part of the printing plate. The frog was placed in different areas of the panel with different infrared absorption levels, producing different signals. Internally, the frog can be programmed to play a game, as explained below.

As shown in FIG. 1, 5 water lily leafs, which have been printed with 5 different shades of infrared absorbing ink, are also distinguished by colors or numbers that make them appear different to the player. The player places the frog on the board, activates the switch, and switches on the internal electronic circuit, so that the frog says "jump to any floating leaf", and the game starts. The frog is then lifted from the board by the player and returned to the board, preferably on the water lily lobes (the background is of the same order as one of the water lily lobes). The sensor module in the frog records the level reading and confirms "landing" with positive audio/visual feedback. The frog then says "jump to another lobe". If the frog is now placed on a new float, the new landing is confirmed as before and the new level is again recorded. However, if the frog jumps again onto the same lobe, the level reading will be the same, giving a negative feedback, followed by saying "jump onto another lobe". Finally, the infrared absorption levels on all 5 water lily leafs will be recorded. The sensor module now assigns these levels with predetermined attributes-i.e. the highest level with e.g. "yellow", the next highest with "green", etc. The game continues with the frog speaking a color randomly aloud. The frog must now land on the designated floating leaf. The sensor module compares the last reading with the recorded grade and makes corresponding feedback. The game is challenged by speeding up and by recording the number of hits within certain time limits.

Turning now to fig. 2 and 3, fig. 2 is a typical printed code that can appear on printed paper and includes a sequence of regions of infrared absorbing ink printed at different densities. The infrared absorption that can be detected by the sensor will vary depending on which area the sensor is looking at. Of course, this technique can be applied to properties other than infrared absorption.

It is evident from figure 2 that successive zones will give different levels as the sensor passes over the printed code and it can be seen that it will in fact produce an output as schematically shown in figure 3.

The particular actual measurement of absorption is not critical. It is clear from fig. 3 that as the sensor passes over the area shown in fig. 2 from left to right, the infrared absorption starts at zero (when the sensor is on white paper), rises at a to a level arbitrarily noted 5, then falls at B to an arbitrarily lower level designated as value 1, then rises again at C to level 4, falls at D to level 2, and rises at E to level 3.

This process produces a sensor sequence 51423 in terms of output. The absolute level of absorption can be set in a completely arbitrary manner, but the relative level must be set to allow sufficient differentiation, which obviously can be done without great difficulty.

These level readings in the sequence can be used either in the method of sliding the sensor over a series of areas as shown in fig. 2, or, for example, the sensor can be applied to these areas in succession.

As explained above, if the sequence is maintained, then secondly, the sensor device is also made self-calibrating according to the respective levels (which the sensor can detect by means of the maxima and minima clearly visible in fig. 3), the sequence in which these maxima and minima occur being in fact capable of representing a code which can be decoded and applied to change the operating mode of the sensor device.

As will be seen from the discussion below, the sensor device can thus function in both a self-calibrating and decoding manner. There is a code, but here the code is in a way that also allows the sensor device to learn the absolute printing grade, without requiring the sensor device to have any prior knowledge of the absolute grade. However, the encoding mechanism not only allows for unique identification of each printing level, but also generates a unique code sequence that can be used when decoding to pass the sensor through operation, such as operation in a particular programming mode.

Turning now to the specific example shown in figure 2 and the example producing the code sequence shown in figure 3 as 5-1-4-2-3, it will be seen that it will serve to identify a particular mode of operation for the appropriate interaction between the sensor device and the printed material bearing the indicia shown in figure 2.

With the readily available printing scheme, printing with infrared absorbing black ink on white paper, it is straightforward to provide 7 discrete response levels from the infrared sensor. The 7 levels include white (where there is no absorbing ink), black (where there is absorbing black ink arbitrarily defined as 100%), and a number of intermediate levels designated 1 to 5, where the amount of infrared absorbing black ink is 8% for level 1, 17% for level 2, 27% for level 3, 38% for level 4, and 54% for level 5, relative to the amount printed for the 100% black area. If the printing is on a colored area printed with typical cyan, magenta, and yellow printing inks (all non-infrared absorbing), printing at these levels with infrared absorbing black inks can become substantially undetectable.

It can be very simply seen that measuring each successive level as a maximum and minimum, as shown in fig. 3, can be faced with a variety of different sequences. According to the simplest approach one wishes to generate a sinusoidal response as shown in fig. 3, each response sequence must be "high-low-high-low". If the motion sensor traverses the sequence of regions shown in FIG. 2 and is specified on FIG. 3, then the following code sequence that satisfies the sinusoidal requirement is:

ABCDE

0513240

0514230

0523140

0524130

0534120

0413250

0415230

0423150

0425130

0435120

0315240

0324150

0325140

0314250

0214350

0215340

the software within the sensor device may be programmed for reaction, once the various levels of reflectivity have been read, i.e. once a sufficient number of maxima and minima have been detected, these levels are stored for later classification of the unknown response when the sensor device is placed and separated against the paper, since the sequence can be identified, i.e. the "code" can be identified.

This technique of identifying maxima and minima is used in order to detect the first level in any particular sequence, all of which are required because the previous level is lower than the first level. Likewise, the last level detected only requires that the subsequent level be below that level in order to identify it as the maximum. Accordingly, it is not necessary to start with zero absorption (i.e. white paper) on the printed material for the grades preceding and following the sequence of different printed areas. Of course, the code levels can be read in reverse, and if desired, can be operated in either order, in reverse or not, to detect successive maxima and minima, but this does not reduce the possible number of uniquely identifiable codes that can be detected in this manner.

If desired, the code sequences can be controlled to represent them with a single low resolution value in order to save processing power in the sensor device itself. This is simply done by taking each maximum or minimum value that is read out and summing the number of levels that were previously less than that level. It should be noted that in connection with this, the first two measurements, i.e. the first maximum and minimum of the measurements, are often one of them, so the first two terms of a sequence are not important in this sense. If the measured sequence is 0514230, starting at the third max/min value, in this example the value 4, the number of previous elements numerically below this element is calculated, giving compressed data: 223, with weighting factors 1, 2 and 8, respectively, the sum of the specific values multiplied by the weighting factors can be calculated as follows:

(2×1)+(2×2)+(3×8)＝30

the other scheme is that the formula is used:

where n = the number of levels of non-zero elements in the sequence, x is the index of the elements in the code sequence (from zero to n), and the formula

Where f is ₂ (x)＝[1，2，8] ^x-1

A unique code can be generated for each sequence as follows:

sequence code No

0534120 21

0435120 21

0524130 28

0425130 29

0514230 30

0415230 31

0523140 36

0325140 37

0513240 38

0315240 39

0215340 41

0423150 44

0324150 45

0413250 46

0314250 47

0214350 49

It can thus be seen that this produces 16 different codes, each represented by a two-digit number, thus enabling the device to determine in which of the 16 possible program modes it should operate.

If it is desired to select from an even wider variety of modes of operation, the code sequence can simply be extended as long as the last level measurement (which tells the device that the read mode of the code is to be terminated) is not repeated. Thus, the sequence 0514230 can be extended to 051514230 or 05142424230. In doing so, the available code sequences, and obviously the number of levels added for each pair, are doubled, and of course each pair of added levels may be printed the same as the other levels, doubling the number of possible sequences, making the uniquely identifiable code quickly becoming very large. When longer sequences are used in this manner, the compression algorithm discussed above must have more weighting factors, such as 1, 2, 8, 16, and 32. In each case, completing the process will result in only recognizable short digital code artifacts.

Fig. 2 shows a sequence of printed areas, which are adjacent to each other. If desired, the regions may be separated from one another so that the sensor detects the reflectance of the background paper between each region as it passes over them. This makes it easier to interpret the recorded signal, which in practice leads to greater operational stability, especially in the case of an increased number of different levels of reflectivity. This solution is more resistant to the variations in the printing tolerances of the printed material and indeed in the sensor head itself.

By using such a sequence of areas printed on, for example, each page of a repetitive exercise book, automatic calibration and recalibration can be carried out simply and reliably. Considerable savings can be achieved in terms of device internal programming by avoiding having to have an internally stored "look-up table". When the sensor device is started to be used, the levels in question are essentially stored and lost at the end of the user session, although the software may retain them if it enters a power saving "sleep" mode, e.g. 60 seconds after no input change, before entering a power off mode if there is no change, e.g. within the first 10 minutes of the sleep mode.

Referring now to fig. 4, there is shown in block diagram form how the differential comparator detection technique can be used to overcome the variations and tolerances associated with the detection of absolute levels of IR ink absorbance. This technique only measures the difference in absorbance as a relative value. To achieve this with minimal expense, the circuit configuration shown in fig. 4 may be used. Microcontroller 1 handles constant current analog output (A) _out ) Usually either directly or via a buffer circuit (buffer 3) for driving an external audio device (loudspeaker 2). The technique utilizes this characteristic to output a current A _out Compared with the current drawn by the infrared sensor circuit atIn this sensor circuit, the sensor 4 mainly functions as a variable current source. In a passive signal conditioning circuit, both the analog output of the microcontroller 1 and the output of the sensor 4 are conditioned and scaled so that the two current levels can be directly compared by the comparator 6. The digital output of the comparator 6 is then fed to the digital input of the microcontroller (D) _in ) Indicating whether the analog output is greater or less than the output of the sensor 4.

By varying the analogue output A of the microcontroller 1 _out The microcontroller 1 is able to determine in software the relative analogue output level of the sensor 4. The microcontroller then provides a direct digital representation of the sensor analog output in much the same way as a conventional successive approximation analog-to-digital converter (ADC).

Under normal operation, the mechanism will require the audio output of the microcontroller 1 to be temporarily suspended while the analog output is used for comparison with the sensor output. However, a novel feature of this circuit is that the analog comparison is done continuously, even when there is audio output. This works because the audio output level range is specifically designed to cover the full range of equivalent sensor outputs through the signal conditioning circuit. Because the audio signal consists essentially of Alternating Current (AC) waveforms on any audio output sequence (i.e., sound or speech), the level nature of the analog output covers the full range of possible outputs of the sensor. Thus, by continuously monitoring the digital output of the comparator 6, the absolute level of the sensor can be continuously monitored. This does require that the microcontroller 1 be able to determine the digital equivalent level of the analog output in order for the microcontroller's software to determine the relative levels of the sensors. The comparator means 6 may be located inside the microcontroller, if desired.

Analog output A _out And may be generated externally (by using a digital-to-analog converter or DAC) or internally by the microcontroller. In both cases, the microcontroller must be able to read or determine the equivalent digital level of the instantaneous analog output level.

The audio buffer 3 mayIn order to be output by the microcontroller in digital form (D) _out ) And (5) controlling. This allows the microcontroller to drive its analog output for comparison purposes even when no audio output is required, nor is any output required from the speaker.

Referring now to fig. 5, this figure shows a preferred circuit for a sensor device according to the present invention, although the power supply (conventional power supply) has been omitted for clarity. Shown is a micro-processor unit 10 which has been programmed for the required operation and which can become active when a switch 11 is closed. The switch 11 may be, for example, a "tip switch" mounted in the elongate end of a "sensor pen" so that when the tip is pressed against the printed material, the switch 11 is closed. Also connected to the microprocessor unit 10 is an audio transducer 12. The microprocessor unit 10 also controls whether current flows through the transmitter diode of the optical infrared transmitter/receiver assembly 13. The part receiving the sensor is connected to one input of the comparator 14. Connected to the other input of the comparator 14 is a resistive ladder network, generally indicated at 15, the ladder network 15 also being connected to the output of the microprocessor unit 10.

In operation, when switch 11 is closed, circuitry in microprocessor 10 applies a signal to the "rung" of resistive ladder 15 that changes the voltage applied to the positive input of comparator 14. The voltage applied to this input may be incrementally dropped over a very rapid time interval (e.g., every 100 microseconds), while at the same time the voltage applied to the negative terminal of comparator 14 will depend on the amount of infrared radiation reflected from the surface on which optical transmitter/receiver assembly 13 is placed. The voltage on the negative input of comparator 14 is often less than the supply voltage so that the output of comparator 14 therefore remains logic 1 until the voltage applied to the positive input of comparator 14 drops below the level applied to the negative input, at which point the output drops to logic 0, and accordingly, because the output of comparator 14 is connected to the input of microprocessor 10, the microprocessor knows exactly what voltage level is applied to the negative input of the comparator, i.e., it has a measure of the infrared reflectivity of the surface against component 13. As the assembly 13 moves over the printed sequence of different reflective regions, the various levels of absorption can be detected and decoded for use as described above.

The circuit shown in fig. 5, which is very simple and inexpensive to manufacture, is ideal for use with hand-held sensor devices used in conjunction with printed materials, both for early learning activity books and for mental test books. With appropriate programming, the output from the transducer 12 can provide attractive and exciting interactive activity between the sensor unit, the printed material, and the user.

Claims (8)

1. An interactive information device comprising printed material and sensor means adapted to respond to printed features on the printed material, wherein the printing on the printed material has a mental component and a non-mental component, the latter being sensed by the sensor means, wherein the sensor means senses different values of a property on the printed non-mental image, and wherein the printed material and/or internal programming in the sensor means is arranged to ensure that the sensor means is automatically aligned with the printed material as the sensor means is placed on the printed material as part of a structurally, mentally mediated interaction between the sensor means and the printed material.

2. The apparatus of claim 1, wherein the sensor device and the printed material are configured as an interactive game that requires a user to place the sensor on a given number of different interactive areas of the printed material.

3. Apparatus according to claim 2, wherein the printed material comprises a printed track or maze which must be traversed by the user successively over different printed areas having different values of the property to be sensed.

4. Apparatus according to claim 1, wherein the printed material is in the form of a quiz book or repeated testing exercise and the sensor means has been programmed to record different values of the sensed property of the printed material, and the recording of the different values is then processed to extract information therefrom about the different values themselves.

5. The interactive information device of claim 1, wherein the sensor means comprises a processing unit adapted to identify the maximum and minimum values of the sensed property value following sequential successive applications of the sensor means over a sufficiently large number of different (non-intellectual) printed areas of the printed material.

6. An apparatus according to claim 5, wherein the sensor means is programmed to set the property-specific values of the maximum and minimum values to specific values, and thereafter the sensor means is programmed for identification, wherein if the property being sensed is within a preset tolerance band of the maximum and minimum values, the sensor will identify the printing as having the property at one of the given values.

7. Apparatus according to claim 6, wherein the printed material exhibits different maximum and minimum values in a finite fixed sequence, whereby the sensor means may calibrate itself to the particular printed material and change its mode of operation from a default mode or some earlier mode of operation to a new mode of operation.

8. The apparatus of claim 7, wherein the interactive printed material and sensor device form a puzzle or early learning book, and wherein the rules of interaction between the sensor device and the printed material vary from page to page, or from one printed book or board-bound exercise to the next.

Images