Classifications

• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
  • G06V30/00—Character recognition; Recognising digital ink; Document-oriented image-based pattern recognition
  • G06V30/10—Character recognition
  • G06V30/14—Image acquisition
  • G06V30/142—Image acquisition using hand-held instruments; Constructional details of the instruments
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof

Definitions

• This invention is generally directed to electronic cameras and more particularly to capturing and building a text file using an electronic camera.
• Electronic cameras such as digital cameras and video cameras are popular consumer products.
• the electronic camera has special optics and an electronic imager circuit that work together to capture 3-dimensional scenes in electronic form and are used in the same way as conventional chemical film cameras.
• Another area of conventional electronic imaging is dominated by the scanner which is specifically tailored for scanning a document into a graphics file using image stitching techniques.
• the graphics file can be fed to an optical character recognition engine (OCR) which recognizes text in the file and then creates a file that contains text strings.
• OCR optical character recognition engine
• Scanners cannot take pictures of 3-dimensional objects as can be done using the conventional electronic camera.
• the conventional electronic camera while constantly being improved to provide higher quality images that rival chemical film cameras, cannot "scan" text into a file.
• OCR optical character recognition engine
• An embodiment of the invention is directed to a method of building an electronic file, featuring the steps of forming a number of light images of portions of a scene on a camera imager, the imager being a part of an electronic camera that captures 3-dimensional objects, and generating a number of image data tiles that represent the images using the camera.
• a number of text data tiles each containing text recognized in a corresponding one of the image data tiles is generated.
• the method includes searching for overlapping text in the text tiles, and aligning the text tiles.
• Fig. 1 shows a block diagram of an electronic camera configured according to an embodiment of the invention.
• Fig. 2 illustrates a flow chart of operations performed according to an embodiment of the invention.
• Fig. 3 shows how two text tiles are aligned.
• Fig. 4 depicts a flow chart of correction operations.
• An embodiment of the invention is directed to an apparatus and method for building an electronic file using an electronic camera.
• Such an electronic camera thus plays a dual role, as both a conventional camera as well as a scanner, thereby reducing the cost of taking conventional pictures of distant objects and scanning text.
• Configuring an electronic camera to scan text according to the techniques described below permits the creation of an electronic file such as a plain text file or a composite text/graphics word processor file that depicts text on a 3-dimensional object, such as a packing container, or on distant objects with unusually large text, such as black boards and white boards. Scanning text from such objects would present a problem for the conventional handheld scanner.
• a further advantage provided by an embodiment of the invention is that no image stitching techniques are needed to form the text file.
• Image stitching techniques that operate upon rastergraphics data require a significant amount of data processing and storage resources.
• the text linking techniques used in the invention search for and align text strings (letters and words) which have much fewer degrees of freedom than rastergraphics data and are therefore much less compute- intensive to manipulate. This makes text linking more suitable for use in a limited resource environment such as the electronic camera.
• Fig. 1 illustrates an electronic camera 100 configured according to an embodiment of the invention.
• the camera 100 includes conventional optics 104 having an optical path that leads to an imager 108 positioned at the focal plane of the optics.
• the optics 104 may feature a zoom lens and an adjustable aperture.
• Such a combination allows the camera 100 to capture scenes under a wide range of field of view/depth of field, covering both close-ups of documents as well as distance shots of 3- dimensional scenes that cannot be captured with conventional scanners.
• the optics 104 and the imager 108 together are capable of a field of view /depth of field that cannot be achieved with conventional scanners.
• the optics 104 may also support a variable field of view/ depth of field by, for instance, incorporating a zoom lens.
• the imager 108 may be implemented according to a wide range of conventional techniques, using technologies such as charge coupled device(CCD) or complimentary metal oxide semiconductor (CMOS).
• CCD charge coupled device
• CMOS complimentary metal oxide semiconductor
• the digitized raw image data generated by the imager 104 is processed by conventional digital image and signal processing circuitry 112 to yield digital image tiles (e.g. pixel arrays) of the captured scene. These tiles may have the full spatial resolution of the imager sensor array, or they may be scaled or cropped portions thereof.
• the tiles may represent portions of a document 102, and are normally obtained by moving the camera 100 around the document 102 while the camera captures a series of overlapping tiles to cover the entire document.
• the tiles may be transferred to an external data processing device 116 that is not exclusively a stand alone camera, such as a personal computer (PC), in digital image file format.
• An interface 120 in the camera 100 may be to a conventional computer peripheral bus that connects the camera to the PC or to a PC peripheral.
• Software in the PC can then render or further process the digital files to display them as still images or as motion video.
• the PC may also be configured with additional software to perform the text recognition and text linking steps according to certain embodiments of the invention.
• the interface 120 may also be used to transfer an electronic file containing the linked text to the PC.
• the camera 104 has a character recognition engine 124 that works with a text linker 128.
• the character recognition engine 124 may be based on a conventional optical recognition engine (OCR) that recognizes printed text in a graphics image file and in response outputs the recognized text in a format readily useable by a computer.
• OCR optical recognition engine
• the character recognition engine 124 provides a text data tile that contains strings of recognized text appearing in a corresponding one of the image tiles. These text tiles are collected and linked together by the text linker 128 thus building a "linked text file.”
• the text linker 128 looks for a matching text string in two different text tiles, and pastes the text tiles in proper alignment into the text file. To maximize the likelihood of proper alignment, the matching string should be relatively long and should also occur infrequently in a text tile. This linking process repeats with successive text tiles to build up the stitched text file that eventually represents the document 102.
• Both the text linker and the character recognition engine may be implemented as a processor executing instructions from a memory, either in the camera 100 or in the digital processing device 116.
• Fig. 2 shows a text recognition and linking procedure according to an embodiment of the invention
• Fig. 3 shows two exemplary text tiles being linked.
• Operation begins with a first digital image tile, first_scan_img, being fed to an OCR in step 204 to yield a first text tile, first_scan_tex.
• Each of these tiles may be represented by a two- dimensional array of strings, where each element of the array may be a phrase, letters, or even a single character.
• the OCR will be configured to focus only on the text in "high confidence regions" of an image tile to reduce the possibility of recognition errors.
• the OCR may provide a confidence index value that gives an indication of the OCR's level of confidence in recognizing a particular text string in the image tile.
• a full resolution image obtained by a digital camera exhibits geometric distortion in the corners and edges.
• a tile having the full resolution of a camera may be cropped prior to being fed to the OCR to maximize the high confidence region and the confidence index. Operation then continues with step 208.
• the electronic file doc_tex is initialized with the first text tile.
• the text file doc_tex may initially be represented by a sufficiently large two-dimensional array of strings.
• the array may then be converted into any one of a number of known word processor formats.
• An array corresponding to prev_scan_tex is initialized in step 208 to the first_scan_tex. Operation then continues with a loop beginning with step 212. While there are still image tiles to process, a current text tile, curr_scan_text, is obtained from the OCR in step 216, and a feature to be matched, str_ftr, is extracted therefrom in step 220.
• the str_ftr should include a relatively long word or sequence of characters. If a long word is not available in the prev_scan_tex, then a series of sequentially occurring words should be selected as the str_ftr. Alternatively or in addition, the linker may be configured to ignore certain "stop-words" such as "the", “of", “an”, and "and", such that the selected str_ftr should not contain any such stop- words.
• step 224 the linker searches the curr_scan_tex and prev_scan_tex for the str_ftr.
• the curr_scan_tex is appended to doc_tex in step 228 if properly aligned with prev_scan_tex.
• the alignment and appending steps are depicted by an example in Fig. 3.
• the word "cameras” (starting at 2 nd row and 3 rd column of curr_scan_tex) is selected as the str_ftr and its corresponding location in prev_scan_tex is at 3 rd row and 17 th column.
• the validity of this corresponding location i.e. the alignment, can be further confirmed by its neighboring text strings, such as "sue", "ts. The”, “elec”, etc.
• Operation begins with step 404 in which the correction routine determines whether there are no more errors to be corrected. If there are any, then operation continues in step 408 in which the routine waits for the camera to be positioned over a region of the scene that contains text corresponding to the error. Operation then continues with step 412 in which a new image tile, new_scan_img, of the region is obtained by the camera and fed to the OCR. The OCR proceeds with text recognition and generates a new text tile, new_scan_tex, based on the new_scan_img. Assuming that the new_scan_tex contains no recognition errors, a string feature str_ftr is then extracted from the new_scan_tex in step 416.
• the str_ftr should be as unique as possible and should have a high recognition confidence index as given by the OCR, such that the string can be quickly found while searching the previously created doc_tex around the region where the error occurred, as in step 420.
• the str_ftr is found in both the doc_tex and the new_scan_tex, as in step 424, the alignment of the tiles is verified and, if aligned, the new_scan_tex is pasted over the previously recovered text which contained the error in the doc_tex. This correction can be further verified by displaying the new_scan_tex correction on a monitor being viewed by the user. Operation then loops back to step 404 to correct any additional errors.
• the system may be configured to warn the user that the text tiles being received from the camera imager are not sufficiently clear and are likely to result in text recognition errors, or that they do not contain sufficient overlap to yield a reliable alignment. This may be done by, for instance, sending a warning to a camera display window or to a window of the external data processing device 116 (see Fig. 1).
• a conventional image stitching routine may be loaded into the separate data processing device 116 to allow the scanning of graphical figures in addition to the text linking described above. If the OCR does not recognize any text in some of the image tiles, then these tiles may be fed to the image stitching routine (not shown) rather than the text linker 128 to generate a graphics figure. The user may then be signaled an option as to whether this graphics figure should also be appended to the electronic file. Of course, the user may later replace the stitched graphics figure with a higher quality original if desired.

Landscapes

• Engineering & Computer Science (AREA)
• Multimedia (AREA)
• Theoretical Computer Science (AREA)
• Computer Vision & Pattern Recognition (AREA)
• Physics & Mathematics (AREA)
• General Physics & Mathematics (AREA)
• Signal Processing (AREA)
• Studio Devices (AREA)
• Facsimile Scanning Arrangements (AREA)
• Image Input (AREA)
• Character Discrimination (AREA)
• Processing Or Creating Images (AREA)
• Television Signal Processing For Recording (AREA)
• Character Input (AREA)

Priority Applications (5)

Applications Claiming Priority (2)

Publications (1)

Family

ID=23164714

Family Applications (1)

Country Status (9)

Cited By (5)

Families Citing this family (22)

Citations (4)

Family Cites Families (16)

• 1999
  • 1999-04-29 US US09/301,753 patent/US6563948B2/en not_active Expired - Fee Related
• 2000
  • 2000-04-21 JP JP2000615961A patent/JP2002543536A/ja active Pending
  • 2000-04-21 WO PCT/US2000/010732 patent/WO2000067195A1/en not_active Ceased
  • 2000-04-21 DE DE10084525T patent/DE10084525B4/de not_active Expired - Fee Related
  • 2000-04-21 AU AU43665/00A patent/AU4366500A/en not_active Abandoned
  • 2000-04-21 GB GB0125643A patent/GB2363935B/en not_active Expired - Fee Related
  • 2000-04-21 KR KR1020017013752A patent/KR20020038574A/ko not_active Ceased
  • 2000-04-27 MY MYPI20001807A patent/MY125948A/en unknown
  • 2000-04-28 TW TW089108145A patent/TW460743B/zh not_active IP Right Cessation

Patent Citations (4)

Non-Patent Citations (2)

Also Published As

Similar Documents

Legal Events