JP2000020677A - Image capture/massage display device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To obtain a device capable of capturing images of an object with a camera and displaying information related to the object on the device by identifying a selected image as a specific object having a code to be recognized by a computer system by processing the selected image, and extracting and displaying the information related to the object. SOLUTION: This device 10 captures optical images of the object existing in front of a lens 18, arranged directly above a lateral image width guide 24 and generates an optical image signal of the object. After receiving an instruction from a 1st switch 34 for taking a snapshot image of the object by the lens 18 and other constitutional parts in the device 10, the object is identified and recognized, information related to the object is checked, and the image of the object and information related to the object are displayed on a display. The object can be expressed as words, numbers, an image or a graph which are two-dimensional type printed data, or as a three-dimensional object such as a clock, a dog and an automobile.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

FIELD OF THE INVENTION The present invention relates generally to image processing systems, and more particularly, to capturing an image of a subject,
The present invention relates to a system having a camera that recognizes a captured image as a recognized target, extracts information related to the recognized target, and displays the extracted information on a display.

[0002]

2. Description of the Related Art Modern society is an environment in which individuals have a lot of information and knowledge to be absorbed, and such information is available in many forms. For example, it can come from television programs, radio broadcasts, the Internet, etc. Nevertheless, the most widely used information sources are still print media, ie newspapers, magazines, or books. Newspapers or books differ from other media in that they provide information in printed form and can be reliably referenced for future use. Even information obtained from the Internet is often printed on hardcopy for future reference.

[0003] Students usually gain knowledge by reading textbooks and other printed materials provided by the teacher. Experts read magazines or newsletters in their areas of expertise to keep them out of date. Engineers and scientists read data books, technical journals, or marketing and trading magazines to keep up to date with their specialties. Salesmen and market people read market magazines to learn about the trends in the products they sell. The large amount of information available in printed form basically requires an individual to spend a significant portion of his or her daily life reading these materials. When reading a document, people sometimes need to look up words in a dictionary. If you need to look up many unknown words in a dictionary, you can be frustrated.
This can prevent a person from reading, thereby hindering his or her learning ability.

[0004] An electronic dictionary is a device that provides the function of a dictionary and allows a person to look up the meaning of a word without turning over the pages of a conventional dictionary. Furthermore, recent technological developments have made it possible to produce electronic dictionaries with voice functions that can verbalize the pronunciation of words. However, one major disadvantage of prior art electronic dictionaries is that they require the user to key in the desired word. More recently, handwriting recognition technology has made it possible to input words by handwriting on special writing pads. This key entry or handwriting operation tends to be slow and error prone, thereby further slowing down the understanding of the document. If the electronic dictionary finds the wrong word due to a typographical error, the user may misunderstand the meaning of the sentence as a result. The user
This is especially true if he or she is unaware that he has made a typographical error. Therefore, it is desirable for the electronic dictionary to accept inputs other than conventional typed or written input methods.

Although electronic dictionaries have become portable in the size of an electronic organizer, they are still too bulky to carry around. Therefore, it is desirable that the electronic dictionary can be put in a pocket so that the user can easily use the electronic dictionary without burdening the user.

[0006] Most prior art scanners are for scanning entire pages or lines of a document. This is not suitable for use in electronic dictionaries. Because electronic dictionaries typically only provide the definition of a single word, scanning an entire page of a document, or even a single line, can be burdensome and uneconomical. Therefore, it is desirable to have a portable, lightweight device that can capture images of one word or several words at a time.

[0007]

SUMMARY OF THE INVENTION It is, therefore, an object of the present invention to provide an apparatus capable of capturing an image of an object with a camera and displaying information relating to the object on the apparatus.

Another object of the present invention is to capture an image of a word, eliminating the need for the user to manually key in the word,
An electronic dictionary capable of displaying information related to the word on a display attached to the device.

Another object of the present invention is to provide a portable electronic dictionary that can be carried comfortably by a user.

[0010]

SUMMARY OF THE INVENTION In certain embodiments of the present invention, a computer system captures and displays one or more images of one or more selected objects, processes the selected images, and provides a computer system. A device is disclosed that identifies the selected image as a particular object having a recognizable code, retrieves and displays information associated with the selected image. 1
In an embodiment, the device is coupled to a body having a front end and a rear end, and a front end by a lens mounting rim.
A lens for capturing an optical image of the object in front of the lens,
Includes a display mounted on the body, a switch positioned on the body, and a distance guide. The spacing guide further includes a standoff arm and a sideways image width guide. The standoff arm extends forward from the front of the body and has two ends, one end of which is connected to the body and the other end of which is connected to the sideways image width guide. The image capture device constantly generates a signal representing the image of the object selected by the user, displays the image represented by the received signal in real time, takes a still image of the selected object when the switch is activated, and then takes this The still image is identified as a specific object, and information related thereto is extracted.

One advantage of the present invention is that an apparatus is provided that can capture an image of an object from a camera and display information related to the object on the apparatus.

Another advantage of the present invention is that an electronic dictionary capable of capturing words without entering the words from a keyboard or writing pad and displaying information related to the words on a display attached to the device. Is to be provided.

[0013] Another advantage of the present invention is that a portable electronic dictionary is provided that can be carried comfortably by a user.

[0014] Other objects, features and advantages of the present invention will become apparent to those skilled in the art from a reading of the following detailed description of a preferred embodiment with reference to the several drawings.

[0015]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS While the invention may be embodied in many forms, details of a preferred embodiment are schematically illustrated in FIGS. It is to be understood that this disclosure is not intended to limit the invention to the illustrated embodiments.

The present invention deals with an image capture / message display device. This device captures an optical image of an object. The target is words, numbers,
It can be a diagram, a graph, or a three-dimensional object such as a watch, a dog, a car, or the like. In an electronic dictionary, the target is a word consisting of several characters. The device divides words into character subparts and converts the optical image signal of each character into a digital signal. This device then
A recognition process is performed on the word, information related to the word is searched in a database, the found information is retrieved, and the information is displayed on a video display screen of the device.

Referring to FIG. 1, there is shown a perspective view of a presently preferred embodiment of the image capture / message display. The image capture / message display device 10 is the body 1
2 In this embodiment, the body 12 resembles a pen that the user can comfortably hold in his or her hand just like a pen. The device 10 has a front end 1
4 and a rear end 16. The lens 18 is located at the front end 14 of the body 12. A CCD-based camera (not shown) for capturing an optical image of an object is arranged inside the body 12 behind the lens 18. The lens 18 has a lens mounting rim 19 around it. Rim 19 may be secured to body 12 by snapping, welding, screwing, or any other suitable method. Standoff arm 20 extends forward from front end 14 of body 12. The standoff arm 20 has two ends,
One end 22 of the arm 20 is connected to the body 12. Sideways image width guide 24 connected to distal end 26 of arm 20
Is fixed to the arm 20 via suitable means. The horizontal image width guide 24 is fixed to the arm 20 at a predetermined angle. The horizontal image width guide 24 is the stand-off arm 2
The angle fixed at 0 has no effect on the operation of this device. The guide 24 may be fixed to the arm 20 at right angles. Alternatively, the horizontal image width guide 24
The ball joint may be coupled to the stand-off arm 20 so that it has any angle of flexibility with respect to the body 12. One advantage of using a ball joint to connect the guide 24 to the arm 20 is that the user can easily use it in any inclined position. Another advantage is that it increases the durability of the spacing guide mechanism and makes the arm 20 less susceptible to breakage. Standoff arm 20 and landscape image width guide 24 together form spacing guide 25. Arm 2
The length of zero is determined by the focal length of the lens 18 so that the lens 18 will always focus on an object at a distance equal to the length of the arm 20 in front of the lens. At the rear end 16 of the body 12, a cap 42 is fixed to the body 12. Like the lens mounting rim 19, the cap 42 can be snapped or screwed to the body 12. The battery (not shown) is connected to the rear end 1
6 is installed inside the body 12 to supply power for operating the device 10.

A display 28, which is a flat panel display, is positioned on the body 12 of the device 10. In a currently preferred embodiment, the display is a liquid crystal display. A plurality of buttons 30, 32 are provided on the display 28 to adjust the brightness of the display 28.
It is placed near. Pressing the first button 30 increases the brightness of the display. Pressing the second button 32 reduces the brightness of the display. Multiple switches 3 on the body
There are also 4, 36, 38 and 40. The first switch 34 is for taking a snapshot image of an object by a lens or other components in the device 10. Other components and their operation are discussed in the following description.
The other three switches 36, 38, 40 control the focus of the lens, control the zooming of the subject, generate a sample speech pronunciation of the subject, activate a speech recognition operation, activate a communication operation, or simply on / off. Can be used for other applications, such as functioning as a switch. As another alternative to these three switches, they can be used as a shift key, an alt key, and a control key to provide more combinations of switch applications. These additional functions can also be applied to buttons 30, 32 that control the brightness of the display.

The apparatus 10 captures an optical image of the object in front of the lens 18 just above the horizontal image width guide 24,
Generating an optical image signal of the object, identifying and recognizing the object after receiving an instruction from the first switch 34, examining information related to the object, displaying the image of the object and the information related thereto on the display can do. The target is
Can be subject to words, numbers, or other forms.

The device 10 is as portable as a pen.
In the example of the electronic dictionary, the image capture / message display device 1
0 captures the optical image of the word, finds definitions and other information related to the word, and displays the word and its definitions and information on the display. Personal information management (P
In an IM) system, the image capture / message display 10 captures an individual or organization name and finds the telephone and / or facsimile number, address, meeting schedule and / or schedule of the contacting individual. A calendar event or appointment for a particular date can also be retrieved by capturing the calendar date, recognizing that date, retrieving the appointment for the day, and displaying the appointment on the display. Similarly, information about a business card can be captured by an image capturing device and stored in a database. By identifying the basic elements of a business card and capturing a still image of the element via an image capture device, the information stored in the database can be retrieved.

The lens 18 can be a circular lens. However, in an electronic dictionary, an object whose image is to be captured by the device is usually in a rectangular shape. This is because in English and other Western languages, each word is generated from several alphabets. To put the alphabet together, they are usually arranged from left to right, resulting in words in the shape of a rectangle. Therefore, it is not necessary to make the shape of the lens 18 circular in order to capture an optical image of a rectangular target. Since the image capture process of the present invention involves a substantially rectangular shaped object, it can be part of a circular lens. Alternatively, the lens may be a Fresnel lens or a part of a Fresnel lens, and the thickness of the lens may be reduced. It can also be a cylindrical lens or a part of a cylindrical lens. However, in the case of a foreign language such as Chinese or Korean, the word shape is usually a square.
Therefore, the shape of the lens used in the Chinese or Korean electronic dictionary may be circular instead of rectangular.

FIG. 2 is a perspective view showing an alternative embodiment of the presently preferred embodiment. This figure shows a second device 44, similar to a computer mouse, that captures an object and displays information related to the object. In order to distinguish the device 10 referred to in FIG. 1 from the second device 44, the second device 44 is referred to as a mouse 44 in the present invention.

The mouse 44 has a body 46. Mouse 44 has a front end 48 and a rear end 50. Mouse 44
Has a top 52, a bottom (not shown), and two sides 5
4, 56. A standoff carrier 58 is mounted upwardly at the front end 48 and positions a lens (not shown) in position with respect to the front end 48 of the body 46 of the mouse. Standoff carrier 58 includes an upward riser 62 portion and a platform 64 portion. The upward riser 62 is rotatably attached to the front end 48 of the body 46 so as to raise the platform 64 to a predetermined height above the body 46. Platform 64 includes an upper surface 66 and a lower surface 68.
The lens is mounted on the lower surface 68 such that its optical axis is parallel to the upward riser 62 of the standoff carrier 58. An upper surface 66 and a lower surface 68 surrounded by a plurality of tall sides 70 define a housing (not shown) therein. Upper surface 66 and lower surface 68
The distance between provides sufficient space to place the CCD-based camera behind the lens.

Near the bottom of the front end 48 of the mouse 44, a guide member 72 extends from the front end 48. Guide member 72
Has a rectangular shape with an opening 74 formed therein. The shape of the opening 74 is also substantially rectangular. The guide member 72 is positioned so that the optical axis of the lens passes through the opening 74.

In the preferred embodiment, platform 64 is rotatably secured to upward riser 62 via a shaft 69. The platform 64 is rotated, and the platform 64 is folded downward so that the upward riser 62
Can also be parallel. The upward riser 62 is rotatably attached to the body 46 by a shaft (not shown). The standoff carrier 58, which includes the upward riser 62 and the platform 64,
You can fold it down side by side with this before. The standoff carrier 58 can then be slid into the body 46. The guide member 72 is mounted on a slidable carrier (not shown),
4 can be retracted into the body 46. The detailed mechanism of this feature is described in the co-pending patent application.

The display 76 is located on the top 52 of the body 46.
Is positioned on top. In a currently preferred embodiment, the display is an LCD display. A plurality of switches 78, 80 are located on top 52 of body 46.
Alternatively, switch 82 may be connected to sides 54, 5
6 or one or both. The first switch 78 is for taking a snapshot image of the object via the lens in the mouse 44 and the CCD-based camera. Other switches 80, 82 can be used for the other applications described above. Mouse 4
There is a room in body 46 that can store a battery accessible from the bottom of 4. The battery compartment has a snap-on type cover that holds the battery in place.

Image capture / message display device 1 of FIG.
Similar to the lens described above, the lens used in the mouse-type device 44 can be made in several shapes.
This is a circular lens, part of a circular lens, Fresnel
It can be a lens, a part of a Fresnel lens, a cylindrical lens, or a part of a cylindrical lens.

FIG. 3 is a block diagram of a preferred embodiment of device 10 or mouse 44 for capturing an image of object 84 and displaying information related to object 84 on display 28. Since the functional architecture and operation inside the device 10 and the mouse 44 are the same, the following description is a description related to the device
Can be applied equally. The device 10 includes a camera 86 having a lens 18 (or 62 for the mouse 44) and a two-dimensional image sensor 88 for sensing an optical image of the object 84, an analog / digital (A / D) converter 90,
Digital signal processor (DSP) 92, read only memory (ROM) 94, random access memory (RAM) 96, display 28, switch 34
-40.

The two-dimensional image sensor 88 is arranged behind the lens 18. The two-dimensional image sensor 88 receives the optical image signal of the object 84 passing through the lens 16, transforms the optical image signal into an analog image signal, and converts the analog image signal into an analog / digital (A / D) converter 90. Forward. The A / D converter 90 generates a digital image signal from the analog image signal, and transmits the digital image signal to a digital signal processor (DSP) 92. Switch 34
Is coupled to the DSP 92 and allows the user to take a snapshot still image of the object 84 on the landscape image width guide 24 of the device 10 (see FIG. 1) or in the opening 74 of the guide member 72 (see FIG. 2). Is controlled by D
SP 92 is coupled to ROM 94 and RAM 96.
ROM 94 contains instructions for controlling DSP 92. ROM
94 also includes a database containing all information related to the object 84, or a template library for character recognition operations performed by the device 10. ROM94
Can be in the form of flash memory so that the latest information can be stored before the device 10 is powered off. The RAM 96 is a general-purpose memory mainly for the device operation controlled by the DSP 92. DS
P92 is electrically coupled to the two-dimensional image sensor 88,
The operation of the three-dimensional image sensor 88 is controlled.

Microphones, speakers and communication means can also be included in this device, all of which are DSP
92, and provides speech recognition, speech synthesis, or communication functions. Additional switches may be coupled to the DSP 92 to control the on / off of the microphone, speaker, communication, or device.

In a preferred embodiment, the device 10
4 is captured. In order to capture an image of the object 84, the lens 18 must be placed at a distance 100 from the object 84. Distance 100 must be approximately within the focal length of lens 18 for camera 86 to capture a sharp image of object 84. Pen type device 1
If zero, this distance is calculated and predetermined by the length of the standoff arm 20. In the case of the mouse-type device 44, the distance 100 is the distance between the lens and the guide member 7.
It should be the distance to the object located in the two openings 74, ie approximately the length of the upward riser 62.

The object 84 may be in the form of a text such as a word in a document. The image capturing device 10 has an opening 7
4 or an image of the object located on the horizontal image width guide 24. For example, in an electronic dictionary, the image capture device 10 of the presently preferred embodiment captures an image of a word in a single operation, just as if taking a picture with a camera. In the prior art, an image of a word is typically scanned as part of a page or line of a document and placed into a computer. Prior art scanners cannot selectively manipulate one word among other words on the same line of the document. Therefore, the user of the electronic dictionary cannot retrieve the definition of a word using the scanner as an input device. Similarly, a PIM user scans a calendar date via a conventional scanner,
You cannot query his or her reservation for a particular day. The image capture device 10 of the present invention can capture an image of a single word, several words, or a portion of the word by word length. This makes the apparatus more flexible in capturing images of small objects or small portions of an item.

The light image of the object passes through the lens and is projected onto the two-dimensional image sensor 88. In a preferred embodiment, the two-dimensional image sensor 88 includes, for example, a charge-coupled device (CCD) optical image sensor module that converts an optical image signal of interest into a two-dimensional directional analog image signal. This conversion process is well known in the art. Next, the analog image signal is supplied to the analog / digital converter 90.
Is converted into a digital image signal. The digital image signal is then processed by the DSP 92 and sent to a display 28 that displays the image of interest. All of these processes are performed in real time, so the image of the object shown on the display reflects the image of the object within range of the lens. The user can view the image of the object captured by the device on the display in real time. After the user sees the object on which he or she wants more information on the display, he simply presses switch 34 to take a snapshot still image of the object. DSP92 is switch 3
It is detected that 4 has been pressed, and it is recognized that the target on which the image is displayed is selected, and that information on the target should be retrieved and displayed.

In another embodiment, the user has the switch 3
The image capture operation can be activated by placing 4 in the "on" position. When activated, the image capture device captures an image of an object located within the lens. The user of the image capturing device can view the captured image of the target on the display. The snapshot still image of the subject is taken when the switch is released.

The two-dimensional image sensor 88 uses an image sensor driver incorporated in the ROM 94. In a preferred embodiment, the image sensor driver is a DSP 9
2 in the form of a computer program including a sequence of instructions for controlling the operation of the computer. The two-dimensional image sensor 88
The optical image signal is converted into a target analog image signal according to an instruction of the image sensor driver.

The DSP 92 of the preferred embodiment may be a microcomputer, microcontroller, CISC processor, or RISC processor. The type of processor does not affect the results of operation of the image capture device 10. However, its performance may be affected. Digital signal processors are known for being able to process many. Due to the large amount of computationally intensive polynomials used in signal processing techniques, digital signal processors are considered to be the best type of processor to use in this application.

FIG. 4 shows that C
FIG. 2 is a schematic diagram of a two-dimensional image sensor 88 including an image sensing circuit 102 connected to a CD analog shift register 104. The image sensing circuit 102 includes a plurality of photosensitive elements (referred to as photosites) arranged in a two-dimensional matrix including a photodiode 108 and a capacitor 110, each element coupled in parallel.
including. One end of the image sensing circuit 102 is the image capturing device 1
0, and the other end of the image sensing circuit 102 is coupled to one side of the shift gate 106. The other side of shift gate 106 is coupled to CCD analog shift register 104. When the shift gate 106 is closed, light reflected from the targeted object changes the state of the photodiode 108, producing a continuous optical signal. Continuous optical signal is CC
Clocked into D analog shift register 104. An analog image signal is generated via the CCD analog shift register 104.

Alternatively, a CMOS based analog shift register may be used instead of a CCD analog shift register.
A resistor may be used, and a photo resistor may be used instead of the photodiode 108. When such a substitution is made, the CCD-based two-dimensional image sensor 88 becomes CMOS-based.
It becomes a base image sensor.

Referring now to FIG. 5, the image capture operation performed by the preferred embodiment apparatus 10 is shown.
It is essential that the user activates the image capture operation, which requires the user to activate the image capture operation by pressing a switch on the camera (step 11).
4). The optical image of the object passes through the lens 18 and the camera 8
6 is detected. The optical image signal is transformed into a two-dimensional analog image signal (step 116). Next, the analog image signal is converted into a digital image signal by an analog / digital converter (step 118). In the pre-processing operation, DSP
Removes noise in the digital image signal received via the filtering sub-process (step 120) and generates a filtered image signal.

Next, the DSP performs an optical image recognition sub-process. In the electronic dictionary, the recognition operation of the present invention is a new optical character recognition sub-process (step 124), the detailed operation of which is shown in FIG. The DSP then performs a character or word matching sub-process that matches the recognized words with words in the database (step 126). This matching sub-process of the preferred embodiment is illustrated in FIG. 9 below. After finding a matching word in the database, DSP 92 retrieves information related to the word from the database (step 128). The retrieved information and the words found in the database are then displayed on display 28 of device 10 (step 130).

As is well known to those skilled in the art, the preferred embodiment increases the angular tolerance between the device and the object so that the individual can comfortably hold the device as if holding the pen at an angle. can do. Further, the preferred embodiment can also implement a new camera calibration function that allows the image coordinates to be rotated,
With this function, the user can further freely determine the angle of the camera with respect to the object.

FIG. 6 is a flowchart illustrating the optical character recognition sub-process of the preferred embodiment. The sub-process begins with receiving a matrix of filtered digital image signals, including data points representing the optical signal, arranged in a matrix represented by X and Y coordinates (step 132). Each element of the matrix is P _xy (x,
y) represents the gray scale value of the pixel. (X, y) is the position of the pixel _{Pxy in} the matrix.

Next, an image in binary gray format is generated from the grayscale filtered digital image signal (step 134). The basic operation of this process is to convert a grayscale or color scale image signal into a binary gray format. This operation will cause the various gray shaded pixels to be white or black (or 0 or 1)
Pixel. That is, the pixels are either black dots or pixels without dots. Various algorithms can be used to determine how to transform the grayscale data into binary data.
After generating the binary data, a filtering process is performed to remove dot noise in the image (step 136). The result of this filtering sub-process is a better and sharper image data matrix.

The next step is to generate a target image block. Before generating an image block, the target image must first be divided into subparts. In an electronic dictionary, a word framing subprocess is performed on each captured word image. The word framing sub-process is implemented by first detecting spaces before and after the word. Each character in the word is also framed in the word framing process, and each character is framed by recognizing the breaks between each character. These processes generate word or character image blocks from the filtered image signal via a word or character framing process (step 138). At this stage, a plurality of image blocks are generated from the image data matrix according to the density of those points. In electronic dictionaries, this requires breaking each character in a word, but this can be achieved by detecting the space between each character. The purpose of this process is to obtain characters that can be more easily recognized in the following process. Characters can be recognized from the character image block by a loop recognition sub-process (step 140). fundamentally,
This loop recognition sub-process is performed by recognizing each character reproduced in the previous sub-process, placing each character in the correct order, and reconstructing the words captured by the image capture device. Some aspects of this stage are further illustrated in FIGS. The previous sub-process (steps 132 to 140) continues until all the characters in the word have been placed in the character image block (step 142).

The above operation of matching characters through the loop recognition subprocess is further illustrated in FIG. Using the letter "Z" as an example, the first step is to identify some feature pixels. In this example, CP-1 14
4, CP-2 146, CP-3 148, and CP
-4 150 four feature pixels were identified. Next, the relative position of each characteristic pixel with respect to the other characteristic pixels is determined. For example, CP-2 146 is to the right of CP-1 144, CP-3 148 is below CP-1 144, and CP-4 150 is to the lower right of CP-1 144. The next step is to identify the angle formed by each feature pixel in the character. For example, the angle of CP-3 148 is the angle between the line connecting CP-2 146 and CP-3 148 and the line connecting CP-3 148 and CP-4 150. In this example, this angle is about 6
0 degrees. The next step is to determine the distance between the two feature pixels. This distance can be expressed as a ratio to one predetermined distance. This ratio is different for various fonts
Preferred to support size characters. After the relative relationship between the characteristic pixels is known, the character
The matching process is performed by comparing the character features previously stored in the system's template library. Template libraries are another form of database.

Referring to FIG. 8, a flow chart of the character loop recognition sub-process shown at step 140 is shown in FIG. In the electronic dictionary loop recognition sub-process, the stroke of the first captured character image is identified (step 15).
2). Using the character "Z" in FIG. 7 as an example, the first stroke is CP1 144 to CP2 146, and the second stroke is C.
From P2 146 to CP3 148, the third stroke is CP3
Three strokes from 148 to CP4 150 can be identified. The next step is to determine the relative length of each stroke (step 154). In the character “Z”, at this stage, the relative length of each stroke, ie, CP1 144 and C
The relative length of the first stroke, which is the distance between P2 146, C
The relative length of the second stroke, which is the distance between P2 146 and CP3 148, and the relative length of the first stroke, which is the distance between CP3 148 and CP4 150, are determined. The next step is to determine the number of strokes of the character image captured by the image capture device (step 156). For the letter "Z", the number of strokes is three, identified above as first, second, and third, respectively. The next step is to determine the angle between the two connected strokes (step 158). As described above, the characteristic pixel CP3
The angle at 148 is the angle between the second stroke and the third stroke. In this example, the second stroke and the third stroke are two connected strokes. Similarly, the first stroke and the second stroke identified above are also two connected strokes in the character “Z”. The two connected strokes, the first stroke and the second stroke, form at the feature pixel CP2 146 the angle also determined at this stage. The next step is to determine the orientation of each stroke (step 160). Again, using the letter "Z" as an example, the first stroke is usually written from left to right, the second stroke is usually written from upper right to lower left, and the third stroke is usually written left to right. From right to right. Next, feature information of the captured character image including the information determined in the above step is generated from the stroke feature (step 162). The feature information is the relative length of each stroke,
It includes the number of strokes of a character image, the angle formed between two connected strokes, and the orientation of each stroke. The captured character image feature information is then compared to the character features in the template library (step 164). The template library is a database that contains in the computer a prototype image for each distinct graphic symbol to be recognized. After comparing the feature information of the captured character image with the features of the characters in the template library, it is essential to determine whether a perfect match has been found (step 166). If an exact match is found, the character found as a result of the matching sub-process will be used as the reference character (step 1).
68). If a perfect match is not found, a character having a characteristic closest to the characteristic information of the captured character image is selected as a reference character (step 170).

After all the characters in the word captured by the image capture device have been identified, the next sub-process is to retrieve information related to the word from the database. A sub-process to retrieve the information of the identified word,
This is shown in the flowchart of FIG. This word matching subprocess:
Beginning with retrieving the recognized word from memory (step 172). After retrieving the recognized word, the next step is to find a word in the database that matches the recognized word (step 174). It is then determined whether a matching word has been found in the database (step 1).
76). It is essential to provide some information about whether the captured words are found in the database. If a matching word is found in the database, definitions and information relating to the matching word are retrieved from the database (step 178). Otherwise, the word in the database that is closest to the captured word is selected and its definition and information are retrieved from the database (step 180). After retrieving the information from the database, the word, its definition, and the information to which it pertains are displayed on the display of the device (step 182).

While the invention has been particularly shown and described with respect to particular embodiments, those skilled in the art will recognize that various changes and modifications in form and detail may be made. It is, therefore, intended that the appended claims cover all such changes and modifications as fall within the spirit and scope of the invention.

[Brief description of the drawings]

FIG. 1 is a perspective view of an image capture / message display device of the presently preferred embodiment.

FIG. 2 is a perspective view of another preferred embodiment image capture / message display device.

FIG. 3 is a block diagram showing the main components of the preferred embodiment.

FIG. 4 is a schematic diagram of a CCD-based image sensor of a preferred embodiment.

FIG. 5 is a flowchart of an image capturing operation and a message displaying operation of the preferred embodiment.

FIG. 6 is a flow chart of the image recognition process of the preferred embodiment.

FIG. 7 is a diagram illustrating an example of selection of a target feature pixel in a matching operation according to a preferred embodiment;

FIG. 8 is a flowchart of the matching operation of the loop recognition sub-process of the preferred embodiment.

FIG. 9 is a flowchart of an information search operation.

[Explanation of symbols]

 Reference Signs List 10 device 18 lens 28 display 34-40 switch 84 object 86 camera 88 two-dimensional image sensor 90 analog / digital (A / D) converter 92 digital signal processor (DSP) 94 read-only memory (ROM) 96 random access memory (RAM)

────────────────────────────────────────────────── ───

[Procedure amendment]

[Submission date] August 18, 1998 (1998.8.1)
8)

[Procedure amendment 1]

[Document name to be amended] Drawing

[Correction target item name] All figures

[Correction method] Change

[Correction contents]

FIG.

FIG. 2

FIG. 3

FIG. 4

FIG. 7

FIG. 5

FIG. 6

FIG. 8

FIG. 9

 ──────────────────────────────────────────────────の Continuing on the front page (72) Inventor David D.S.H., Taiwan, Taipei She-Lin District Hou-Khan Street 66 (72) Inventor, Fred H.W.Cheng, Taiwan, Taipei County Chang Ho City Bao-Xien Road Lane 106.38.4 T.F.F. (72) Inventor Samson XS One Shanghai Guidong Road Run 557.11 Room 202 (72) Inventor Lin Lee China, Sansi Province, Cusian, Jenter District, Jinson Road, 7F Term (Reference) 5B047 AA01 AA30 AB01 BA03 BB04 BC05 BC15 CB09 CB16 CB22 EA01 EB20 5B050 AA06 BA1 6 BA20 CA04 DA06 EA07 EA15 EA18 FA02 FA09 FA19 GA04 GA08 5B075 ND07 PP04 PP10 PQ02 PQ32 PR06 QM08 UU01

Claims (22)

[Claims] 1. Capture and display one or more images of one or more selected objects and process the selected images to identify the selected images as specific objects having codes recognizable by the device. An apparatus for identifying, retrieving and displaying information related thereto, comprising a processor, a memory storage device containing a display and information relating to a plurality of identifiable objects, and a switch for taking a snapshot still image of the objects. A camera that generates a signal representing a still image of the object, wherein the camera constantly generates a signal representing the image of the object selected by the user and passes the signal to the processor, and the processor generates the signal. The processor constantly displays the image that it represents in real time, and when the switch is activated to take a still image of the selected object, the processor Identifies an image as a specific object, retrieve information related thereto device. 2. A lens for sensing an optical image of one or more selected objects, receiving an optical image signal of the object via the lens positioned at the back of the lens, and generating an analog image signal of the object. Responsive to an analog image signal of interest from the image sensor.
An analog-to-digital converter for generating a digital image signal of the object, and a processor responsive to the digital image signal of the object generated by the analog-to-digital converter for displaying a digital image of the object on a display. Item 10. The apparatus according to Item 1. 3. The apparatus according to claim 2, wherein the image sensor is a CCD-based image sensor. 4. The apparatus according to claim 2, wherein the image sensor is a CMOS based image sensor. 5. The apparatus according to claim 1, which is a personal information management system. 6. The apparatus according to claim 1, wherein the display is a flat panel display. 7. The device according to claim 1, wherein the display is a liquid crystal display. 8. A method of capturing an image of a subject, displaying said image by a device, identifying it as a particular subject, extracting relevant information and displaying the same on a device, comprising: Capturing a signal and generating a signal representative of the image of the subject; processing the signal to generate an image represented thereby and displaying the image; and activating a user-controllable switch. Generating a still image; identifying the still image as a particular object having a code recognizable by a processor in the device; recognizing the particular object via an image recognition sub-process; Matching the recognized object with an object previously stored in a memory storage device in the device; and Method comprising the steps of retrieving information from the memory storage device of the device, and displaying the information on a device display. 9. A method for generating a grayscale digital image data matrix from a digital image signal of a specific object, and generating a digital image signal of a specific object in a binary gray format from the grayscale digital image signal. 9. The method of claim 8, further comprising: framing a particular object from a digital image signal in the form of a binary gray signal of the particular object; and generating an image block of interest from the frame of interest. 10. A sub-process for matching a recognized object to an object in a database, identifying the matched object, dividing the object into a plurality of sub-parts, wherein each feature pixel is associated with another feature pixel of the sub-part. Determining the position; determining the angle formed by each feature pixel of the subpart; and determining the distance between the two feature pixels of the subpart, wherein the feature depends on the feature of each feature pixel. Identifying a plurality of feature pixels in each subpart; comparing features of the subpart with other subparts in the database; subparts having features matching the subpart of interest;
Or selecting from the database a subpart having features closest to the features of the subpart of interest. 11. The method according to claim 8, wherein the recognition sub-process is an optical character recognition process. 12. Capture and display one or more images of one or more selected objects and process the selected images to convert the selected images as specific words having codes that the device can recognize. An electronic dictionary device for identifying, retrieving and displaying information related thereto, comprising: a processor; a memory storage device including a display and information relating to a plurality of identifiable words; and a switch for taking a snapshot image of the words. A camera that generates a signal representing a still image of the word, the camera constantly generating a signal representing an image of the word selected by the user and passing this signal to the processor, the processor comprising: When the image represented by the signal is constantly displayed in real time and the switch is activated to take a still image of the selected word, the processor There identifying the still picture as a particular word, the electronic dictionary device to retrieve the information related thereto. 13. A lens for sensing an optical image of one or more selected words, and receiving an optical image signal of the word via the lens positioned at the back of the lens to generate an analog image signal of the word. An image sensor operable to respond to an analog image signal of a word from the image sensor.
An analog-to-digital converter for generating a digital image signal of the word, and a processor for displaying a digital image of the word on a display in response to the digital image signal of the word generated by the analog-to-digital converter. Item 13. The electronic dictionary according to item 12. 14. The electronic dictionary according to claim 13, wherein the image sensor is a CCD-based image sensor. 15. The electronic dictionary according to claim 13, wherein the image sensor is a CMOS-based image sensor. 16. A personal information management system according to claim 1.
3. The device according to 2. 17. The apparatus according to claim 12, wherein the display is a flat panel display. 18. The device according to claim 12, wherein the display is a liquid crystal display. 19. A method of capturing an image of a word, displaying the image by an electronic dictionary, identifying the image as a specific word, extracting information related to the word, and displaying the information on the electronic dictionary, wherein the word is constantly displayed. Capturing an optical image signal and generating a signal representing the image of the word; processing the signal to generate an image represented thereby and displaying the image; Generating a still image of the word after being coded, identifying the still image as a particular word having a code recognizable by a processor in the device, and recognizing the particular word via an image recognition sub-process. Matching a recognized word with a word previously stored in a memory storage device in the device; Method comprising the steps of retrieving information related to words from the memory storage device of the device, and displaying the information on the electronic dictionary display. 20. Generating a grayscale digital image data matrix from a digital image signal of a specific word, and generating a digital image signal of a specific word in a binary gray format from the grayscale digital image signal. 20. The method of claim 19, further comprising: framing a particular word from a digital image signal in the form of a binary gray signal of the particular word; and generating an image block of the word from the frame of the word. 21. A sub-process of matching a recognized word to a word in a database and identifying the matched word comprises: dividing the word into a plurality of subparts; Determining the position, determining the angle formed by each feature pixel of the subpart, and determining the distance between the two feature pixels of the subpart, the features depending on the features of each feature pixel. Identifying a plurality of feature pixels in each subpart; comparing features of the subpart with other subparts in the database; subparts having features matching the word subparts;
Or selecting from the database a subpart having features closest to the features of the word subpart. 22. The method according to claim 19, wherein the recognition sub-process is an optical character recognition process.