KR19990036555A - Intelligent video capture and message display device - Google Patents

Abstract

An image capturing and message display apparatus for capturing an image and processing and displaying information related to an object having an image captured by a camera includes a main body having a front end and a rear end and a lens for capturing an optical image of an object from the front of the lens. A lens coupled to the front end via a mounting rim, a display mounted at a predetermined position relative to the lens on the body, a switch mounted at a predetermined position relative to the display on the body, and one end of the body A distance induction device including a stand-off arm extending forward from the front end having two ends coupled to the body and a lateral image width induction device coupled to the other end of the stand-off arm at a predetermined angle; have.

The image capturing apparatus continuously generates a signal representing an image of an object selected by the user, displays an image displayed as a signal received in real time, and simultaneously captures a still image of the selected object by activating the switch, thereby making it a specific object. Identifies a still image and retrieves information about the object.

Description

Intelligent video capture and message display device

The present invention relates generally to an image processing system, and more particularly, to capture an image of an object, recognize the captured image as a recognized object, retrieve information related to the recognized object, and display the retrieved information on a display. It relates to a system with a camera.

Modern society is in an environment with a lot of information and knowledge for humans to absorb, and that information is available in many formats. For example, it can come from television programs, radio broadcasts, the Internet, and the like. Nevertheless, the most widely used sources of information still come from printed media, ie newspapers, magazines or books. Newspapers or books are unlike any other medium in which they provide information in a printed format, making it a reliable future reference. Even information obtained from the Internet is printed in hardcopy for future reference.

Students acquire knowledge by reading textbooks and other printed materials provided by the teacher. Experts read professional magazines or weekly magazines to know the trends of the times. Technicians and scientists read textbooks, technical journals or trade magazines to keep up with recent developments in their respective areas of expertise. Salesmen and merchants read trade magazines to see trends in their owned products. The large amounts of information available in the printed format make it an essential part of human daily life in reading these materials. When reading a document, a person may sometimes need to look up words or idioms in a dictionary. If you need to find hundreds of unknown words in a dictionary, people can be disappointed. This makes it impossible for a person to read, which can interfere with a person's ability to learn.

Electronic dictionaries are devices that provide the features of a dictionary, but allow a person to find the meaning of a word without turning the pages of an old dictionary. Moreover, recent technological developments have made it possible to manufacture electronic dictionaries with phonetic attributes that can represent the pronunciation of words. However, one major drawback of conventional electronic dictionaries is that the user needs to press a key of the desired word. Recently, handwriting recognition techniques have entered words through handwriting on specific writing pads. This key-in or handwriting behavior tends to be slow and error prone, which slows the understanding of the document. If a typographical error causes the electronic dictionary to find an incorrect word, it may cause the user to misinterpret the meaning of the sentence. This applies especially if the user does not understand that a person has made a typographical error. Thus, it is desirable for the electronic dictionary to accept input that is different from the usual type-in or recording method.

Electronic dictionaries can be carried in the same size as electronic filings, but they are still too bulky to carry. Thus, it is desirable to be able to put the electronic dictionary in a pocket so that the user can easily use it so that it is not too burdensome to carry.

Most of the conventional scanners scan an entire page or a line of documents. This is not suitable for use in electronic dictionaries. Since electronic dictionaries typically only provide the definition of a single word, scanning an entire page or even a line of a document can be too burdensome and uneconomical. Thus, it is desirable to have a lightweight portable device that can capture images of more than one word at a time.

It is therefore an object of the present invention to provide a device capable of capturing an image of an object with a camera and displaying information related to the object on the device.

Another object of the present invention is to provide an electronic dictionary capable of capturing an image of a word so as to eliminate the need for a user to manually press a key of the word, and displaying information related to the word on a display attached to the device. .

Another object of the present invention is to provide a portable electronic dictionary which can be carried comfortably by a user.

1 is a perspective view of an image capture and message display apparatus of the preferred embodiment;

2 is a perspective view of an image capture and message display device of another preferred embodiment;

3 is a block diagram showing the main elements of a preferred embodiment;

4 is a schematic diagram of a CCD elementary image sensor of the preferred embodiment;

Fig. 5 is a flowchart for explaining an image capturing operation and a message displaying operation in a preferred embodiment.

6 is a flow chart illustrating an image recognition process of a preferred embodiment.

FIG. 7 is an exemplary diagram illustrating characteristic pixel selection of an object in a matching operation of the preferred embodiment. FIG.

8 is a flowchart illustrating a matching operation of the loop aware subprocess of the preferred embodiment.

9 is a flowchart illustrating an information look-up operation.

In certain embodiments of the present invention, one or more images of one or more selected objects are captured and processed, and the selected images are processed to identify the selected images as specific objects having a code recognizable by a computer system, An apparatus for retrieving and displaying information related is described. In one embodiment, the device comprises a body having a front end and a rear end, a lens coupled to the front end via a lens-mounted rim that captures an optical image of an object at the front of the lens, a display mounted on the body. And a switch located on the body and a distance guide. Such distance inducing devices further include stand-off arms and lateral image width inducing devices. The stand-off arm extends forward from the front of the body and has two ends, one end coupled to the body and the other end coupled to the transverse width guide device. The image capturing apparatus continuously generates a signal representing an image of an object selected by the user, displays an image represented by a signal received in real time, and simultaneously captures a still image of the selected object by activating the switch, A still image is identified as a specific object, and information about the object is retrieved.

One advantage of the present invention is that it provides a stationary camera capable of capturing an image of an object and displaying information related to the object on a device.

Another advantage of the present invention is to provide an electronic dictionary capable of capturing words to eliminate the need for a user to enter words from a keyboard or recording pad, and displaying information related to the words on a display mounted to the device. will be.

Another advantage of the present invention is to provide a portable electronic dictionary that can be carried comfortably by the user.

The above and other objects, features and advantages of the present invention will become apparent to those skilled in the art after reading the following detailed description of the preferred embodiments with reference to the drawings.

While the invention may be embodied in many forms, preferred details are shown schematically in Figs. 1-9, and such techniques are not to be construed as limiting the embodiments to the invention.

The present invention features an image capture and message display apparatus. These devices capture optical images of objects. Such objects can be graphs in two-dimensional formats of words, numbers, images, and printed materials, or three-dimensional objects such as watches, dogs, and vehicles. In an electronic dictionary, an object may be a word composed of many letters. The device divides a word into character subparts and converts the optical image signal of each character into a digital signal. The device then performs a recognition process by that word to search for information related to the word in the database, retrieve the found information and display the information on the video display of the device.

1, a perspective view of a preferred embodiment of an image capture and message display apparatus is described. The image capture and message display apparatus 20 has a main body 12. In this embodiment, the body 12 resembles a pen that a user can comfortably hold in his hand. The device 10 has a front end 14 and a back end 16. The lens 18 is located at the front end 14 of the body 12. Behind the lens 18 inside the main body 12, a CCD camera (not shown) for capturing an optical image of an object is mounted. The lens 18 has a lens mounting rim 19 around it. Rim 19 may be snapped to, adhered to, threaded or secured to body 12 in any other suitable manner. The stand-off arm 20 extends forward at the front end 14 of the body 12. The stand-off arm 20 has two ends, one end 22 of the arm 20 being connected to the body 12. The transverse width guide device 24 is connected to the distal end 26 of the arm 20 which is fixed to the arm 20 by any suitable means. The lateral image width induction device 24 may be fixed to the arm 20 at a predetermined angle. The angle at which the lateral width guide device 24 is fixed on the stand-off arm 20 does not affect the operation of the device 10. Guide device 24 may also be fixed perpendicularly to arm 20. Optionally, the transverse width guide device 24 can be coupled to the stand-off arm 20 by a ball joint, so that the guide device 24 can be at an angle to the body 12. It can have flexibility. One advantage of using the ball joint to connect the induction device 24 to the arm 20 is that the user can comfortably use it even in a tilted position. Another advantage is that the distance guidance mechanism is more robust and does not destroy the arm 20. The stand-off arm 20 and the lateral image width induction device 24 form a distance induction device 25 with each other. The length of the arm 20 is determined by the focal length of the lens 18 so that the lens 18 is always connected to the object at the same distance as the length of the arm 20 in front of the lens. At the rear end 16 of the body 12, the cap 42 is fixed on the body 12. Like the lens mounted rim 19, the cap 42 can be snapped onto the body 12, or screwed onto the body 12 (not shown). It is mounted inside the body 12 at the rear end 16 to supply power for it.

The display 28, which is a flat-panel display, is located on the body 12 of the device 10. In a preferred embodiment, the display is a liquid crystal display. A number of buttons 30, 32 are located near the display 28 to adjust the intensity of the display 28. By pressing the first button 30 the intensity of the display is increased. The intensity of the display is reduced by pressing the second button 32. There are also a number of switches 34, 36, 38, 40 on the body. The first switch 34 is for taking a snap-shot image of the object through the lens and other elements in the device 10. Other elements and their operation are described below. The other three switches 36, 38, 40 adjust the focus of the lens, control the zoom operation for observing the object, generate sample pronunciation of the object, activate the voice recognition operation, and communicate It can be used for other applications such as activating operation or simply acting as an on / off switch. Another alternative to these three switches is that such switches can be used as a shift key, alt-key or control key to combine the application of the switches. This additional feature can also be applied to buttons 30 and 32 that control the intensity of the display.

The device 10 captures an optical image of the object in front of the lens 18 directly above the lateral image induction device 24, generates an optical image signal of the object, and provides instructions from the first switch 34 ( At the same time as receiving the indications, the object can be identified and recognized, search for information related to the object, and an image of the object and the related information can be displayed on the display. Such objects can be words, numbers or other forms of objects.

The device 10 can be carried like a pen. In an example electronic dictionary, image capture and message display apparatus 10 captures an optical image of a word, finds definitions and other information related to the word, and displays the word and its definition and information on a display. . In an example personal information management (PIM) system, the image capture and message display device 10 captures a person's name or organization name and includes the person's telephone and / or facsimile number, address, meeting schedule and / or agenda. Find your back. Calendar events or appointments of a particular date can also be retrieved by capturing the calendar date, recognizing the date, retrieving the schedule at that time, and displaying the schedule on a display. In addition, the information on the office card can be captured by the image capturing apparatus and stored in the database. The information stored in the database can be retrieved by identifying the key element on the business card and capturing a still image of the element via the image capturing device.

Lens 18 is a circular lens. However, in an electronic dictionary, the object whose image is captured by the device is usually rectangular. This is why in English or other Western languages, each word is produced by many alphabets. When putting these alphabets together, the alphabets are usually arranged from left to right, resulting in rectangular words. Thus, in order to capture an optical image of a rectangular object, the lens 18 need not be circular. This may be part of a circular lens because the image capture process of the present invention relates to an object that is nearly rectangular. Optionally, the lens may also be part of a Fresnel lens or Fresnel lens to reduce the thickness of the lens. It may also be a cylindrical lens or part of a cylindrical lens. However, for foreign languages such as Chinese or Korean, words are usually square. Thus, lenses used in Chinese or Korean electronic dictionaries may be circular instead of rectangular.

2 is a perspective view of an alternative embodiment of the preferred embodiment. This figure shows a second device 44 that resembles a computer mouse to capture an image of an object and display information related to the object. In order to distinguish the device 10 mentioned in FIG. 1 from the second device 44, the second device 44 is identified as a mouse 44 in the present invention.

The mouse 44 has a body 46. The mouse 44 has a front end 48 and a back end 50. Mouse 44 also has an upper portion 52, a lower portion (not shown), and two side portions 54, 56. The stand-off carrier 58 positions the lens (not shown) at a predetermined position relative to the front end 48. The stand-off carrier 58 includes an upward rise 62 section and a platform 64 section. The upward rise 62 is rotatably fixed to the front end 48 of the body 46, whereby the platform 64 is raised to a predetermined height above the body 46. Platform 64 includes an upper surface 66 and a lower surface 68. The lens is mounted on the lower surface 68 in such a way that the optical axis is in parallel with the upward rise 62 of the stand-off carrier 58. Upper surface 66 and lower surface 68 surrounded by a number of raised sides 70 define a housing (not shown). The distance between the upper surface 66 and the lower surface 68 provides sufficient space for placing the CCD camera behind the lens.

Guide member 72 extends to front end 48 near the bottom of front end 48 of mouse 44. Guide member 72 is a substantially rectangular member defining opening 74. The opening 74 is also nearly rectangular. Guide member 72 is positioned in such a way that the optical axis of the lens passes through opening 74.

In the preferred embodiment, the platform 64 is rotatably fixed to the upward rise 62 through the axis 69. The platform 64 may be rotated to have the platform 64 folded downward to be in parallel with the upward rise 62. The upward rise 62 is rotatably mounted on an axis (not shown) with respect to the body 46. The stand-off carrier 58 including the upward rise 62 and the platform 64 can be folded downward in front of and in parallel with the upward rise 62. Stand-off carrier 58 may then slide into body 46. Guide member 72 is mounted on a slidable carrier (not shown) and can dent into body 46 of mouse 44. The detailed mechanism of this feature is described in pending applications.

Display 76 is located on top 52 of body 46. In a preferred embodiment, the display is an LCD display. Multiple switches 78, 80 are located on top 52 of body 46. Optionally, the switch 82 may also be located on either or both sides 54, 56 of the body 46. The first switch 78 is for taking a fast-shooting image of the object through the CCD camera and the lens in the mouse 44. Other switches 72 and 74 can be used for the other applications described above. The compartment is located in the main body 46 which is accessible at the bottom of the mouse 44 where the battery can be stored. Battery loss has a snap-on type cover that houses the battery in its location.

Like the lenses described in the image capture and message display device 10 of FIG. 1, the lenses used in the mouse-like device 44 can also be manufactured in many forms. It may be a circular lens, part of a circular lens, Fresnel lens, part of Fresnel lens, cylindrical lens or part of cylindrical lens.

3 is a block diagram of a preferred embodiment of a device 10 or mouse 44 that captures an image of an object 84 and displays information related to the object 84 on a display 28. Although the description below is described with respect to the device 10, the internal functional structures and operations of the device 10 and the mouse 44 are the same, and therefore, the same can be applied to the mouse 44 evenly. Device 10 is an analog-digital camera 86 with lens 18 (or 62 in mouse 44) and a two-dimensional image sensor 88 that senses an optical image of an object 84. (A / D) converter 90, digital signal processor (DSP) 92, read only memory (ROM) 94, random access memory (RAM) 96, display 28, switch do.

The two-dimensional image sensor 88 is located behind the lens 18. The two-dimensional image sensor 88 receives an optical image signal of the object 90 passing through the lens 16, converts the optical image signal into an analog image signal, and converts the analog image signal into an analog-digital (A / D) signal. Send to converter 90. The A / D converter 90 generates a digital video signal from the analog video signal, and sends the digital video signal to a digital signal processor (DSP) 92. The switch 34 is coupled to the DSP 92 and controlled by the user, such that the opening 74 of the lateral image width induction device 24 of the device 10 (see FIG. 1) or of the guide member 72. A quick still image of the object 84 within (see FIG. 2) is taken. DSP 92 is coupled to ROM 94 and RAM 96. The ROM 94 includes instructions for controlling the operation of the DSP 80. Also included in ROM 94 is a database containing all information related to object 84, or a template library for character recognition operations to be performed by device 10. The ROM 94 is also flash memory type so that recent information can be stored before turning off the power on the device 10. The RAM 96 is a general storage device for the device operation controlled by the DSP 92. The DSP 92 is electrically coupled to the two-dimensional image sensor 88 to control the operation of the two-dimensional image sensor 88.

Loudspeakers, speakers and communication means are also included within the apparatus and are all coupled to the DSP 92 to provide speech recognition, speech synthesis or communication characteristics. Additional switches may also be coupled to the DSP 92 to control on / off of loudspeakers, speakers, communications, or devices.

In the preferred embodiment, the device 10 captures an image of the object 84. In order to capture an image of the object 84, the lens 18 must be positioned at a distance 100 from the object 84. For camera 86 to capture a clear image of object 84, distance 100 should be within the focal length of lens 18. In the case of the pen-shaped device 10, the distance is calculated and predetermined by the length of the stand-off arm 20. In the case of the mouse-like device 44, the distance 100 is the distance between the lens and the object located in the opening 74 of the guide member 72, ie, approximately the length of the upward rise 62.

The object 84 is in document format like a word in a document. The image capturing device 10 captures an image of the object 84 in the opening 74 or on the lateral image width inducing device 24. For example, in an electronic dictionary, the image capturing apparatus 10 of the preferred embodiment captures an image of a word in a single operation as if taking a picture with a camera. In the prior art, an image of a word is typically scanned into a computer as part of a page or line of a document. Conventional scanners cannot selectively scan one word from another word in the same line of the document. Thus, the user of the electronic dictionary cannot use the scanner as an input device to retrieve the definition of a word. In addition, PIM users cannot query the assignment of a specific date by scanning the calendar date through a conventional scanner. The image capturing apparatus 10 of the present invention can capture an image of a single word, many words, or a part of a word depending on the length of the word. Thus, such devices provide more flexibility for capturing images of small objects or parts of items.

An optical image of the object passes through the lens and is projected onto the two-dimensional image sensor 88. In a preferred embodiment, the two-dimensional image sensor 20 comprises a charge coupled device (CCD) optical image sensor module for converting, for example, an optical image signal of an object into a two-dimensional directed analog image signal. Such conversion processes are well known in the art. The analog video signal is then converted into a digital video signal by the analog-to-digital converter 90. The digital video signal is then processed by the DSP 92 and transmitted to the display 28 displaying the image of the object. All these processes are performed in real time, such that the image of the object shown on the display reflects the image of the object within the scope of the lens. The user can see an image of the object captured by the device on the display in real time. Once the user sees an object on the display that a person wishes to acquire more information, the user can simply press the switch 34 to take a quick still image of the object. The DSP 90 detects that the switch 34 is pressed, and recognizes that an object on which an image is displayed is selected, and that information about the object is retrieved and displayed.

In other embodiments, the user may place the switch 34 in the "on" position to activate the image capture operation. When activated, the image capture device captures an image of an object located within the scope of the lens. The user of the image capturing apparatus can see an image of the object captured on the display. Rapid still images of the object are taken at the same time the switch is released.

The two-dimensional image sensor 88 uses an image sensor driver embedded in the ROM 94. In a preferred embodiment, the image sensor driver includes a sequence of instructions that control the operation of the DSP 92 in the form of a computer program. The 2D image sensor 88 converts an optical image signal into an analog image signal of an object according to a command of an image sensor driver.

In a preferred embodiment, the DSP 92 may also be a microcomputer, microcontroller, CISC processor or RISC processor. The processor type is not affected by the result of the operation of the image capturing apparatus 10. Digital signal processors are known to handle large numbers. Because computational polynomials are heavily used in signal processing techniques, digital signal processors appear to be the best processor type for these applications.

4 is a schematic diagram of a two-dimensional image sensor 88 including an image sensing circuit 102 connected to a CCD analog shift register 104 via a shift gate 106. Image sensing circuitry 102 includes a number of light sensing elements (also known as photosites) arranged in a two dimensional matrix format, each device having photo-diode 108 and capacitor 110 coupled in parallel. ). One end of the image sensing circuit 102 is connected to the chassis ground 112 of the image capture device 10, while the other end of the image sensing circuit 102 is coupled to one side of the shift gate 106. The other side of the shift gate 106 is coupled to the CCD analog shift register 104. By closing the shift gate 106, the reflected off of the target object changes the state of the photo-diode 108, generating a sequence of optical signals. The sequence of optical signals is clocked into the CCD analog shift register 104. Through the CCD analog shift register 104, an analog image signal is generated.

Optionally, the CCD analog shift register can be replaced with a CMOS-based analog shift register and the photodiode 108 can be replaced with a photo resistor. As such, the CCD-based two-dimensional image sensor 88 becomes a CMOS-based image sensor.

In FIG. 5, an image capturing operation performed by the apparatus 10 of the preferred embodiment is described. The user needs to activate the image capture operation, causing the user to press a camera switch to activate the image capture operation (step 114). An optical image of the object passes through the lens 18 and is detected by the camera 86. The optical image signal is converted into a two-dimensional analog image signal (step 116). The analog video signal is then converted into a digital video signal by an analog-to-digital converter (step 118). In the pre-process operation, the DSP removes noise in the digital video signal received through the filtering subprocess (step 120) to generate a filtered video signal.

The DSP then performs an optical image recognition subprocess. In the electronic dictionary, the recognition operation of the present invention is a novel optical character recognition subprocess (step 124), the detailed operation of which is described in FIG. The DSP then performs a character or word matching subprocess by matching the recognized word with a word in the database (step 126). This matching subprocess of the preferred embodiment is described in FIG. 9 below. After the DSP 92 finds a matching word in the database, it retrieves information related to the word from the database (step 128). The retrieved information and the words found in the database are then displayed on display 28 of device 10 (step 130).

As will be appreciated by those skilled in the art, the preferred embodiment is made to have a high tolerance for angle between the device and the object so that a person can comfortably hold the device as if holding the pen at an angle. Moreover, new camera calibration features are implemented in the preferred embodiment such that the image coordinates can be rotated, which allows the user to tilt the camera freely with respect to the object.

6 is a flow chart illustrating an optical character recognition subprocess in a preferred embodiment. This subprocess begins with the reception of a matrix of filtered digital video signals, which matrix contains data points representing the optical signals, which are arranged in a matrix by X and Y coordinates (step 132). Each element of the matrix represents the gray-scale value of a pixel of P _xy (x, y), where (x, y) is the position of pixel P _{xy in} the matrix. Image data in binary gray format is then generated from the gray-scale filtered digital image signal (step 134). The basic operation of this process is to convert an image signal in gray-scale format or color scale to binary gray format. This operation converts the pixels of the various shades of gray to white or black (or 0 or 1) pixels, meaning that the pixels are either black dots or pixels without dots. Various algorithms can be used to determine how gray-scale data is converted to binary data. After generating the binary data, the filtering process is performed to remove dot noise in the image (step 136). Clear image data matrix may be obtained as a result of the filtering subprocess.

The next step may be to create an image block of the object. Before generating an image block, the image of the object must first be separated into subparts. In an electronic dictionary, word framing subprocesses are performed on an image of each captured word. The word framing subprocess is first performed by detecting the space before and after the word. Each letter of a word is also framed by the word framing process, where each letter is framed by recognizing the discontinuity of the letter. This process generates a word image block or a character image block from the filtered image signal through a word framing or character framing process (step 138). This step generates the multiple image blocks from the image data matrix according to the point densities of the multiple image blocks. In an electronic dictionary, this needs to separate each letter in a word, which is accomplished by detecting the space between letters. The purpose of this process is to obtain characters that are easier to recognize in subsequent processes. From the character image block, the character may be recognized through a loop recognition subprocess (step 140). Essentially, the loop recognition subprocess is performed by recognizing each character reproduced through the preceding subprocess and placing each character in the correct order to reconstruct the word captured by the image capturing apparatus. Certain aspects of this step are further described in FIGS. 7 and 8. The previous subprocess (steps 132 through 140) continues until all the letters of the word have been placed in the character image block (step 142).

The above operation of matching characters through a loop aware subprocess is further described in FIG. 7 using the character "z" as an example. The first step is to identify a number of specific pixels. In this example, four specific pixels are identified as cp-1 144, cp-2 146, cp-3 148 and cp-4 150. The relative position of each feature pixel relative to the other feature pixel is then determined.

For example, cp-2 146 is to the right of cp-1 144, cp-3 148 is below cp-1 144, and cp-4 150 is to the lower right of cp-1 144. The next step is to identify the angle formed within the character at each feature pixel. For example, the angle of cp-3 148 is an angle between a line connecting cp-2 146 and cp-3 148 and a line connecting cp-3 148 and cp-4 150. In this example, the angle is approximately 60 degrees. The next step is to determine the distance between the two characteristic pixels. This distance can be expressed as a ratio with respect to one determined distance. This ratio preferably supports different font sizes of characters. After finding the relative tricks of the feature pixels, the engraver processor is performed by comparing the properties of these characters with the properties of the characters previously stored in the template library of the host system. Template libraries are another form of database.

FIG. 8 is a flow chart illustrating the character loop recognition subprocess identified by step 140 of FIG. In the loop recognition subprocess of the electronic dictionary, first the strokes of the captured character image are identified (step 152). Using the letter “z” in FIG. 7 as an example, three strokes can be identified, where the first stroke is from cp-1 144 to cp-2 146 and the second stroke is from cp-2 146 to cp-3 Up to 148 and the third stroke is from cp-3 148 to cp-4 150. The next step is to determine the relative length of each stroke (step 154). In the letter "z", this step is the relative length of each stroke, i.e. the second stroke, the relative length of the first stroke, the distance between cp-1 144 and cp-2 146, and the distance between cp-2 146 and cp-3 148. Determine the relative length of and the relative length of the third stroke, which is the distance between cp-3 148 and cp-4 150. The next step is to determine the stroke count of the character image captured by the image capturing apparatus (step 156). In the letter "z", the stroke count is three, which is identified as above as the first, second and third strokes. The next step is to determine the angle between the two connecting strokes (step 158). As described above, the angle at characteristic pixel cp-3 is the angle between the second stroke and the third stroke. In this example, the second stroke and the third stroke are two connecting strokes. Further, the first stroke and the second stroke identified as above are two connecting strokes in the letter "z". The two connecting strokes, namely the first stroke and the second stroke, also form an angle in the characteristic pixel cp-2 146 determined at this stage. The next step is to determine the direction of each stroke (step 160). Again using the letter "z" as an example, the first stroke is typically drawn from left to right, the second stroke is typically drawn from upper right to lower right, and the third stroke is typically drawn from left to right. . The characteristic information of the captured character image is then generated from the stroke characteristic including the information determined in the above step (step 162). This feature information includes the relative length of each stroke, the number of stroke counts of the character image, the angle formed between the two connecting strokes, and the direction of each stroke. The characteristic information of the captured character image is then compared with the characteristic of the character in the template library (step 164). The template library is a database containing prototype images in the computer for each unique graphic symbol to be recognized. After comparing the characteristics of the characters in the template library with the characteristic information of the captured character image, it is necessary to determine whether a complete match has been found (step 166). If a perfect match is found, the character found as a result of the matching subprocess is used as a reference character (step 168). If no complete match is found, the character closest to the characteristic information of the character image in which the characteristic is captured is selected as the reference character (step 170).

After all the character images of the words captured by the image capturing device are identified, the next subprocess can retrieve information related to the words from the database. The subprocess for retrieving the information of the identified word is described in the flowchart shown in FIG. This word matching subprocess begins with the retrieval of the recognized word from storage (step 172). After retrieving the recognized word, the next procedure is to find a word in the database that matches the recognized word (step 174). A decision is then made as to whether or not a matched word is found in the database (step 176). It is essential to provide some information to the captured word whether the word can be found in the database. If a matched word is found in the database, definitions and information related to the matched word are retrieved from the database (step 178). In addition, the words in the database closest to the captured words are selected, and their definitions and information are retrieved from the database (step 180). After retrieving the information from the database, the words, their definitions and related information are displayed on the display of the device (step 182).

Although the present invention has been described in particular with reference to certain embodiments, it is understood that various changes and modifications are possible to those skilled in the art. Accordingly, the following claims are intended to cover all such variations and modifications within the true spirit and scope of this invention.

Claims (22)

An apparatus for capturing and displaying one or more images of at least one selected object, the apparatus processing the selected image to identify the selected image as a specific object having a code recognizable by the apparatus, and to retrieve and display information about the selected object. Apparatus for image capture and display of Processor, Storage devices and displays containing information about a plurality of identifiable objects, A camera having a switch for taking a rapid still image of an object and generating a signal representing a still image of the object, The camera continuously generates a signal representing an image of an object selected by the user, and passes this signal to the processor, which continuously displays the image represented by the signal in real time, and simultaneously with the activation of the switch, Taking a rapid still image of the object selected by the processor, and wherein the processor identifies the still image as a specific object and retrieves information about the object. The method of claim 1, The camera A lens for sensing an optical image of at least one selected object, An image sensor behind the lens operative to receive an optical image signal of the object through the lens and generate an analog image signal of the object, An analog-to-digital converter for generating a digital video signal of the object in response to the analog video signal of the object from the image sensor; And a processor for displaying a digital image of the object on a display in response to the digital image signal of the object generated by the analog-to-digital converter. The method of claim 2, And the image sensor is a CCD-based image sensor. The method of claim 2, And the image sensor is a CMOS-based image sensor. The method of claim 1, And the device is a personal information management system. The method of claim 1, And said display is a flat pattern display. The method of claim 1, And said display is a liquid crystal display. A method for capturing an image of an object, the method comprising: capturing an image of an object by displaying the image as a specific object by a device and identifying the information, and searching for and displaying information on the device; Continuously capturing an optical image signal of an object to generate a signal representing an image of the object, Processing the signal to generate an image indicated by the signal, and displaying the image; Generating a still image simultaneously with activation of a switch controllable by a user, Identifying the still image as a particular object having a code recognizable by a processor in the device, Recognizing a specific object through an image recognition subprocess, Matching the recognized object with an object previously stored in a storage device, Retrieving information about the matched object from the storage device; And displaying the information on a display. The method of claim 8, The recognition process is Generating a gray-scale digital image data matrix from a digital image signal of a specific object, Generating a digital video signal of a specific object in binary-gray format from a gray-scale digital video signal, Framing the particular object from a binary-gray format digital video signal of the particular object; And generating an image block of the object from the frame of the object. The method of claim 8, The subprocess that matches the recognized object with the object in the database, and identifies the matched object, Separating the object into a plurality of subparts, Identifying a plurality of feature pixels in each subpart, wherein the properties of the subparts depend on the properties of each feature pixel, wherein the identifying step determines the relative position of each feature pixel relative to another feature pixel of the subpart. Identifying a plurality of feature pixels, including determining an angle formed at each feature pixel of the subpart, and determining a distance between two feature pixels of the subpart, Comparing the characteristics of the subpart with other subparts in the database, Selecting a subpart from a database whose characteristic matches the subpart of the object, or selecting a subpart from a database whose characteristic is closest to the characteristic of the subpart of the object. The method of claim 8, The recognition subprocess is an optical character recognition process. An electronic dictionary device for capturing and displaying one or more images of one or more selected objects, the electronic dictionary device processing the selected images to identify the selected images as specific words with codes recognizable by the device, and searching and displaying information about them. An electronic dictionary device for capturing and displaying an image of an object Processor, A storage device and display comprising information relating to a plurality of identifiable words, A camera having a switch for taking a rapid still image of a word and generating a signal representing the still image of the word, The camera continuously generates a signal representing an image of a word selected by the user and passes this signal to the processor, the processor continuously displaying the image represented by the signal in real time, and simultaneously with the activation of the switch, And taking a quick still image of the word selected by the processor, and wherein the processor identifies the still image as a specific word and retrieves information about the same. The method of claim 12, The camera A lens for detecting an optical image of one or more selected words, An image sensor behind the lens operative to receive an optical image signal of a word through the lens and to generate an analog image signal of the word; An analog-to-digital converter for generating a digital video signal of a word in response to the analog video signal of the word from the image sensor; And a processor for displaying a digital image of the word on a display in response to the digital image signal of the word generated by the analog-to-digital converter. The method of claim 13, And said image sensor is a CCD-based image sensor. The method of claim 13, And said image sensor is a CMOS-based image sensor. The method of claim 12, And the device is a personal information management system. The method of claim 12, And the display is a flat pattern display. The method of claim 12, And said display is a liquid crystal display. A method of capturing an image of a word, comprising: displaying and identifying the image as a specific word by an electronic dictionary device, searching for information on the same, and displaying the image on the electronic dictionary device, Continuously capturing an optical image signal of a word and generating a signal representing the image of the word, Processing the signal to generate an image indicated by the signal, and displaying the image; Generating a still image of a word simultaneously with the activation of a switch controllable by a user, Identifying the still image as a specific word having a code recognizable by a processor in the device, Recognizing a specific word through an image recognition subprocess, Matching the recognized word with an object previously stored in a storage device, Retrieving information about the matched word from the storage device; And displaying the information on a display of an electronic dictionary. The method of claim 19, The recognition process is Generating a gray-scale digital image data matrix from a digital image signal of a specific word, Generating a digital video signal of a specific word in binary-gray format from a gray-scale digital video signal, Framing the specific word from a binary-gray format digital image signal of a specific object; And generating an image block of the word from the frame of the word. The method of claim 19, The subprocess that matches the recognized word with a word in the database, and identifies the matched word, Separating the word into a plurality of subparts, Identifying a plurality of characteristic pixels within each subpart, wherein the characteristics of the subparts depend on the characteristics of each characteristic pixel, wherein the identifying step determines the relative position of each characteristic pixel relative to the other characteristic pixels of the subpart. Identifying a plurality of characteristic pixels, including determining an angle formed at each characteristic pixel of the subpart, and determining a distance between two characteristic pixels of the subpart, Comparing the characteristics of the subpart with other subparts in the database, Selecting a subpart from a database whose characteristic matches the subpart of the word, or selecting a subpart from a database whose characteristic is closest to the characteristic of the subpart of the word. The method of claim 19, A recognition subprocess is an optical character recognition process.