GB2413406A

GB2413406A - Text viewing apparatus and related method

Info

Publication number: GB2413406A
Application number: GB0408882A
Authority: GB
Inventors: Donald Power
Original assignee: NEC Technologies UK Ltd
Current assignee: NEC Technologies UK Ltd
Priority date: 2004-04-21
Filing date: 2004-04-21
Publication date: 2005-10-26
Anticipated expiration: 2024-04-21
Also published as: GB0408882D0; GB2413406B

Abstract

The invention provides for a text viewing apparatus (10) comprising an image capture device (14) arranged to capture an image of at least part of a document (24), a screen (16) arranged for the display of the said captured image, optical character recognition means arranged for the recognition of characters within the said captured image, means (28) for selecting at least part of the image as displayed on the screen (16) for application of the optical character recognition function thereon, and means for accessing a data source to obtain information (30) relating to the said selected part of the image, the apparatus further being arranged for the display of the said information (30) on the said screen (16). The data source may be a dictionary, thesaurus or language translation tool.

Description

241 3406

TEXT VIEWING APPARATUS AND RELATED METHOD

The present invention relates to text viewing apparatus and a related method and, in particular, to such apparatus for use in relation to a printed document containing text that may not be immediately understood, or might require further interpretation When viewing text, for example such as in printed documents including text passages, or printed matter such as that attached to products and packaging, it is often found that a particular word, term or phrase is not immediately understood by the individual reading the document.

A variety of source material is available, such as various dictionaries, glossaries, bilingual dictionaries etc that can assist in the user's understanding of such terminology. However, the manner of accessing such source materials can sometimes prove impossible, and generally at least inconvenient, and this can lead to the individual not obtaining a full appreciation or understanding of the information being presented by way of the document.

Such limitations and inconvenience can therefore prove problematic and the present invention seeks to provide for an apparatus and related method that can serve to limit, and in some cases overcome, such disadvantages.

According to a first aspect of the present invention, there is provided text viewing apparatus comprising an image capture device arranged to capture an image of at least part of a document, a screen arranged for the display of the said captured image, optical character recognition means arranged for the recognition of characters within the said captured image, means for selecting at least part of the image as displayed on the screen for application of the optical character recognition function thereto, and means for accessing a data source to obtain information relating to the said selected part of the image, the apparatus further being arranged for display of the said information on the said screen.

The invention can advantageously provide an easily used apparatus by means of which a particular word, passage, or particular characters within a portion of the text can be readily selected and highlighted for further definition in a particular simple manner, and with little extra inconvenience, as compared with that required when simply reading the document.

Advantageously, the image capture arrangement can comprise a lens and an electro optical sensor arrangement such as a Charged Coupled Device (CCD) sensor comprising an array of pixels.

In particular, the image capture device can comprise a digital camera element and, further, the said screen can be arranged to be located on the opposite side of the apparatus to the lens. In the preferred embodiment, the screen is continuously updated with the latest image captured by the camera.

In this manner, movement of the apparatus over the document can give the impression to the user that the document is actually being viewed through the apparatus in a manner simulating use of a reading glass.

In one advantageous arrangement, the optical character recognition means can comprise a software processing arrangement provided within the apparatus. Alternatively, such processing arrangement can be provided remote from the apparatus, which apparatus is then arranged to include means for signalling the captured image to the remote optical character recognition function.

The selection of the word or acronym of interest is simply by pointing the camera at the word; to make this more precise, a cross-hair can be provided on the screen or superimposed by software. The OCR software needs to locate the boundaries of the word pointed at, highlight it in the actual image, and recognise it (i.e. generate the characters corresponding to the image). In the preferred embodiment this process would be so fast that the user could pan the camera and any words pointed to would be highlighted almost instantly. Alternatively the highlighting could occur after the user has dwelled on a particular word for a short period. In this embodiment, to highlight a single word no keypresses are necessary but to highlight a group of words the user would press a key when pointing to the first word and hold it down while panning along the text to the last word. A second possible embodiment is that the user presses a key when the word of interest is beneath the cross-hair and this triggers the OCR software and highlighting of the word. The means for selecting the word, acronym or phrase of interest is not restricted to the aforementioned methods and can comprise any appropriate form of user-activated cursor or highlighting means.

As with the optical character recognition arrangement the said data source to be accessed so as to obtain information relating to the selected part of the image can be located remote from the apparatus or provided within the apparatus.

In an embodiment in which the data source is to be located remotely from the apparatus, the apparatus can advantageously be arranged to read characters on the printed document indicating, and possibly connecting to, the location of the remote data source.

While the data source for comparison with the selected part of the image can be located on the apparatus, the apparatus can in any case be arranged to allow for selective downloading of such data from a remote location and for subsequent use locally within the apparatus.

The data source can of course comprise any appropriate form of data such as that representing a dictionary and/or thesaurus and/or translation tools.

In one particularly advantageous embodiment, the apparatus comprises a handheld apparatus which can be in the form of a mobile radio communications device such as a cellular phone. In this advantageous arrangement, the optical character recognition means and the data source can be loaded within the apparatus for ease of mobility and use.

According to another aspect of the present invention where there is provided a method of obtaining information relating to characters on a document, the method comprising capturing an image of at least part of the said document, displaying the said captured image by way of a display screen, selecting at least part of the image as displayed on the screen for application of an optical character recognition function thereon, accessing a data source so as to obtain information relating to the selected part of the image as recognised in accordance with optical recognition function, and subsequently displaying of the said information relating to the said selected part of the image on the said screen.

The method can be arranged to provide for further steps for achieving the additional advantages discussed above in relation to the document viewing apparatus as discussed above.

The invention is described further hereinafter, by way of example only, with reference to the accompanying drawings in which: Fig. 1 is a schematic block diagram of a mobile radio communications device and including one embodiment of the present invention; and Fig. 2 is a schematic representation of the use of device of Fig. 1 in relation to a document containing printed text including a term not initially understood by the individual reading the text.

Turning first to Fig. 1, there is illustrated in block diagrammatic form, a cellular phone 10 arranged for communication with a mobile network by way of appropriate radio circuitry (not shown) and an antenna 12.

As is common with a wide variety of cellular phones, an image capture device including a lens 14 is included along with an LCD screen 16 for displaying images captured by way of the lens 14.

The further features of the mobile radio communications device illustrated in Fig. 1 are only those relevant to the present invention and, as noted above, the standard radio access technology has not been included for ease of reference.

For use in relation to the present invention therefore, the cellular phone further includes an optical character recognition function 18 associated with the image capture device for applying optical character recognition to characters appearing within the image captured by way of the lens 14 and as displayed on the screen 16.

A central data source 20 associated with central processing means is also provided and forms a database of source material against which any particular words and/or phrases identified by way of the optical character recognition function 18 can be further defined and/or translated and/or explained for the individual using the device.

Within the illustrated embodiment, this data source 20 can be connected to the antenna 12 so as to allow for the selected downloading of a data source that might be of potential interest to the individual using the device when reading a particular document.

The device can then effectively be primed with, for example, an appropriate glossary of terms that might arise in relation to the document being read.

The control of such functionality within the cellular phone 10 is achieved by the user pointing the camera at the word or acronym of interest, assisted by a cross-hair or similar aiming aid. Either instantly, or after dwelling briefly on the word, or after pressing a key on the keypad 22, optical character recognition locates the boundaries of the word, highlights it within the image displayed on the screen 16, and converts it to character format. The character format is then used for subsequent comparison with the data stored in the data source 20. Additional information is retrieved from the data source and displayed on the screen in a pop-up window on top of part of the image captured by the camera.

In this manner, an individual can readily identify only the particular word, acronym or phrase within the document being read and in respect of which further explanation and/or translation etc is required.

Such functionality of the illustrated embodiment of the present invention is shown further and with reference to Fig. 2. Here, the cellular phone 10 with the antenna 12 and display screen 16 is shown at the time of use in relation to a printed document 24 having text 26 thereon.

As indicated by arrow A the lens (not shown in Fig. 2) of the cellular phone 10 is pointed so as to capture a portion of the text surrounding the abbreviation "CB" since it is this portion of the text in particular that the individual reading the document does not immediately understand.

By way of an aiming aid of superimposed cross-hairs, the user can point at the "CB". Then either automatically or when initiated by the user, the cellular phone activates character recognition of the abbreviation and then subsequent comparison of the recognised abbreviation with a previously downloaded glossary.

In the illustrated example, the glossary identifies "CB" as representing the phrase "cell broadcast" and so this result is then illustrated within the screen 16 by way of a pop-up display 30.

Thus, it will be appreciated that the present invention provides for a device for reviewing, in particular, paper documentation by way of an onboard camera and screen so as to offer a pop-up definition of a pointedto word which definition can comprise a dictionary definition, a translation into another language, an expansion of an acronym or abbreviation or any appropriate further clarification or explanation that might be required.

The use of the apparatus is advantageously intuitive since the apparatus is handheld and can therefore be held over the document being read in a manner simulating use of a reading glass might, but of course with the addition of the required pop-up definition etc. When provided within a radio communications device, the user can advantageously download any preselected glossary so as to focus the available data source on the particular subject area relevant to the document being read.

Further, the printed document can be arranged to contain a reference, such as a URL, to a network location containing a relevant glossary, whilst also containing additional semantically encoded information. Such additional encoded information can comprise, for example, for a technical document, the status of the document and whether it comprises the most upto-date version.

Also, for a consumer product, for example a food product, information concerning content per serving can be captured by way of the device and then subsequently compared with an appropriate data source for such products which can assist in indicating a "buVdon't buy" decision for the user.

A typical user sequence for the apparatus such as that discussed above in relation to this specific embodiment might therefore be as follows.

First, the user is presented with a new printed document that has to be read and which the user might suspect contains terminology he/she might not understand. The printed document however has been primed with server reference detail which identifies a remote location from which a glossary can readily be retrieved.

The user then captures the image of this reference data and applies appropriate optical character recognition to it so as to determine the true identity of the reference, such as a URL. Assuming that the device being used includes a network function, the URL can then be selected within the device so as to allow access to the remote data source from which the glossary can then be extracted and downloaded to the handheld device.

Subsequently, the user can then read the printed document as normal and upon encountering for example a word, phrase, abbreviation or acronym not readily understood, the user can then point at that particular element of the text as appearing on the screen of the device. Subsequently to optical character recognition of that element, it is highlighted and then compared against the glossary recently downloaded so as to obtain a definition, translation etc as required.

Advantageously, any glossary format, semantic encoding of the additional data and the server reference can advantageously be built on existing standards such as XML.

It should be appreciated that the invention is not restricted to the details of the foregoing embodiment and need not necessarily be provided as part of a networked device nor as part of a cellular phone.

Claims

Claims 1. Text viewing apparatus comprising an image capture device

arranged to capture an image of at least part of a document, a screen arranged for the display of the said captured image, optical character recognition means arranged for the recognition of characters within the said captured image, means for selecting at least part of the image as displayed on the screen for application of the optical character recognition function thereto, and means for accessing a data source to obtain information relating to the said selected part of the image, the apparatus further being arranged for display of the said information on the said screen.
2. Apparatus as claimed in Claim 1 wherein the image capture device includes lens and electro-optical sensor arrangement.
3. Apparatus as claimed in Claim 2, wherein the said screen is located on the opposite side of the apparatus to the said lens.
4. Apparatus as claimed in Claim 1, 2 or 3, wherein the optical character recognition means is provided within the apparatus.
5. Apparatus as claimed in Claim 1, 2, 3 or 4, wherein the means for selecting the said at least part of the image comprises a movable cursor also arranged to be displayed on the said screen.
6. Apparatus as claimed in Claim 5, wherein the means for selecting the said at least part of the image comprises a cross-hair also arranged to be displayed on the said screen and optical character recognition software which identifies the boundaries of the word under the cross-hair and modifies the image to highlight it.
7. Apparatus as claimed in Claim 1, 2, 3, 4, 5 or 6, wherein the said data source is located within the apparatus.
8. Apparatus as claimed in Claim 1, 2, 3, 4, 5 or 6, wherein the said data source is located remotely from the said apparatus.
9. Apparatus as claimed in any one or more of the preceding claims, wherein the said data source comprises any one or more of a dictionary, thesaurus and translation tools.
10. Apparatus as claimed in any one or more of the preceding claims and comprising a handheld device.
11. Apparatus as claimed in Claim 10, and comprising a mobile radio communications device.
12. A method of obtaining information relating to characters on a document, the method comprising capturing an image of at least part of the said document, displaying the said captured image by way of a display screen, selecting at least part of the image as displayed on the screen for application of an optical character recognition function thereon, accessing a data source so as to obtain information relating to the selected part of the image as recognized in accordance with optical recognition function and subsequently displaying the said information relating to the said selected part of the image on the said screen.
13. A method as claimed in Claim 12, wherein the step of selecting part of the said image includes the step of moving a cursor within the image represented on the said screen.
14. A method of obtaining the information relevant to characters on a document and substantially as hereinbefore described with reference to, and as illustrated in, Fig. 2 of the accompanying drawings.

14. A method as claimed in Claim 13, wherein the step of selecting part of the said image includes the step of pointing the camera such that a cross-hair is over the word of interest and applying an optical character recognition function capable of identifying the boundaries of the word pointed at.

15. A method as claimed in Claim 12, 13 and 14, wherein the data source is accessed remotely from the said apparatus.

16. A method as claimed in Claim 15, wherein the data source is accessed within the apparatus.

17. A method as claimed in Claim 14, and including the step of downloading the said data source from a remote location to the said apparatus for use therein.

18. Document viewing apparatus substantially as hereinbefore described, with reference to, and as illustrated in the accompanying drawings.

19. A method of obtaining the information relevant to characters on a document and substantially as hereinbefore described with reference to, and as illustrated in, Fig. 2 of the accompanying drawings. 1"

Amendments to the claims have been filed as follows 1. A hand-held text viewing apparatus comprising a lens and electro- optical sensor arrangement arranged to capture an image formed by text on at least part of a document, a screen arranged for the display of the said captured image and located on an opposite side of the apparatus to that of the lens, the apparatus being arranged to be pointed at the document for the selective display of the text on the screen in real time, and including aiming means visible in the display screen for identifying at least one word within the text, and the apparatus further including optical character recognition means arranged first to identify boundaries of the image of the at least one word identified by the aiming means, and subsequently to apply character recognition to the bound image for the recognition of the at least one word, and means for accessing a data source to obtain information relating to the at least one word, the apparatus further being arranged for display of the said information on the said screen.

2. Apparatus as claimed in Claim 1 wherein the aiming means comprises a movable cursor.

3. Apparatus as claimed in Claim 1, wherein the aiming means comprises a cross-hair and further including means arranged to highlight the bound image.

4. 4. Apparatus as claimed in Claim 1, 2 or 3 wherein the said data source is located within the apparatus.

5. Apparatus as claimed in Claim 1, 2 or 3 wherein the said data source is located remotely from the said apparatus.

6. Apparatus as claimed in any one or more of the preceding claims, wherein the said data source comprises any one or more of a dictionary, thesaurus and translation tools.

7. A method of obtaining information relating to characters on a document by means of a hand-held apparatus comprising a lens and an electrooptical sensor arrangement and a screen located on an opposite side of the apparatus to that of the lens, the method comprising pointing the apparatus at the document for selectively capturing by way of the lens and sensor arrangement an image formed by text on at least part of the said document, displaying the said captured image by way of the screen in real time and identifying at least one word within the text by means of an aiming device visible in the display screen, applying an optical character recognition function hrst to identify boundaries of the at least one word identified and subsequently to apply character recognition to the bound image for the recognition of the at least one word and accessing a data source so as to obtain information relating to the said at least one word and subsequently displaying the said information on the said screen.

8. A method as claimed in Claim 7 wherein the step of identifying the at least one word includes the step of moving a cursor within the image represented on the said screen.

9. A method as claimed in Claim 7, wherein the step of identifying the at least one word includes the step of pointing the camera such that an aiming means in the form of a cross-hair is over the word of interest.

10. A method as claimed in Claim 7, 8 or 9, wherein the data source is accessed remotely from the said apparatus.

11. A method as claimed in Claim 10, wherein the data source is accessed within the apparatus.

12. A method as claimed in Claim 9, and including the step of downloading the said data source from remote location to the said apparatus for use therein.

13. Document viewing apparatus substantially as hereinbefore described, with reference to, and as illustrated in the accompanying drawings.