KR101126466B1 - Photographic document imaging system - Google Patents

Abstract

Apparatus and methods are provided for processing a captured image, and more specifically, apparatus and methods for processing a captured image including a document. In one embodiment, an apparatus comprising a camera for capturing a document is described. In another embodiment, the method comprises distinguishing the imaged document from its background, adjusting the captured image to reduce distortion created from the use of the camera, and properly orienting the document. A method of processing a captured image is described.

Graphical Information, Pixel Intensity, Perspective Distortion, Edge Removal, Distortion Removal

Description

Photo document imaging system {PHOTOGRAPHIC DOCUMENT IMAGING SYSTEM}

The present invention relates to an apparatus and method for processing a captured image, and more particularly, to an apparatus and method for processing a captured image including a document.

1-A is a block diagram illustrating typical components of a scanner. The scanner is typically used to capture an image of the document 110. Document 110 is disposed on scanner plate 112. The scan head 120, which is generally comprised of an optical subsystem 122 and a charge-coupled device (CCD), is moved across the document 110. 1A only shows a two-dimensional view, the scan head 120 may move across the document in the direction indicated by the arrow 114 and in a direction perpendicular to the document 110. Optical subsystem 122 focuses the light reflected from document 110 onto CCD 124. CCD 124 is often implemented as a two-dimensional array of photosensitive capacitive elements. When light is incident on the photosensitive element of CCD 124, the charge is trapped in the depletion region of the semiconductor element. The amount of charge associated with the photosensitive capacitive element is related to the intensity of light incident on each element received during the sampling period. Thus, the image is captured by determining the intensity of incident light at each photosensitive capacitive element through sampling of the element. The analog information generated by the photosensitive capacitive element is converted into digital information by an analog-to-digital (A / D) converter 130. The A / D converter 130 may convert analog information received from the CCD 124 in a serial or parallel manner. The converted digital information may be stored in the memory 140. The digital information is then processed by processor 150 in accordance with control software stored in ROM 180. The user may control the scanning parameters through the user interface 170, and the scanned images are output through the output port 160.

A block diagram of the digital camera is shown in FIG. 1B. The light subsystem 122 of the digital camera can be used to focus the light reflected from the document 110 onto the CCD 124, as in a scanner. In other digital cameras, devices other than CCDs, such as CMOS sensors, are used to capture light reflected from the image. In the context of a digital camera, in contrast to a scanner, the optical subsystem 122 does not move along the surface of the document as in a scanner. Rather, in a digital camera, the optical system 122 generally does not move relative to an object, such as a document to be imaged. In addition to digital cameras, photos captured from film based cameras can also be digitized.

The camera offers advantages over scanners for capturing document images and other images. For example, cameras are generally more portable than scanners. In addition, because the scanner requires a captured image to be placed on the scanner plate, the camera can capture a wider array of images than the scanner. However, the use of a camera causes difficulties in image capturing that do not exist when using the scanner. For example, light conditions change when using a camera, while light conditions are generally controlled at the scanner. In addition, the use of the camera depends on several variables, such as the camera's angle to the image, the lens used by the camera and its distance from the image, whether the image containing the document lies on a plane or on a surface, and other factors. Induce image distortion that can vary. Since the scanner uses a moving scanner head, at a fixed distance from the document to be imaged, these distortions generally do not occur in the scanner.

Therefore, there is a need for an apparatus and method for capturing an image of a document that takes advantage of the camera over a scanner, but which reduces the difficulty presented by capturing the document image through the camera in contrast to the scanner.

An apparatus and method for processing a captured image including an imaged document is described. In one embodiment, the apparatus includes a still camera used to capture the imaged document. In another embodiment, a still camera is used to capture the imaged document. In another embodiment, a method of processing a captured image comprising a document comprises distinguishing the imaged document from its background, adjusting the captured image to reduce distortion created from the use of the camera, and Adjusting the orientation as appropriate.

1A shows a conventional document scanner.

1B shows a conventional digital camera.

2 illustrates a general flowchart of a method of processing a captured image.

3 illustrates a flowchart of another embodiment of a method of processing a captured image.

4 illustrates a flowchart of a method for performing segmentation in accordance with one of the implementations of a method of imaging a document disclosed herein.

FIG. 5 shows a flowchart of one method of implementing the random sample consensus shown in FIG. 4. FIG.

FIG. 6 shows a flowchart of one method of performing the outlier removal step shown in FIG. 4. FIG.

FIG. 7 illustrates a flowchart of another method of performing segmentation in accordance with a method of imaging a document disclosed herein.

8 shows a flowchart of one method of performing the distortion removal step shown in FIGS. 2 and 3.

9 shows a flowchart of one method of executing the line step of the text shown in FIG.

FIG. 10 illustrates a flowchart of one method of determining whether a document is properly properly oriented according to one implementation of a method of imaging a document disclosed herein.

FIG. 11 illustrates one embodiment of an apparatus for capturing and processing an image comprising an imaged document.

FIG. 12 illustrates a flowchart of one method of determining whether a document is straightened according to one implementation of a method of imaging a document disclosed herein.

13 illustrates one embodiment of a system for processing a captured image.

Embodiments disclosed herein may be operable to process images captured from a camera including a document. The embodiment disclosed herein can be operative to identify a captured document image from its background. After the captured document image is separated from its background, the embodiments disclosed herein can operate to reduce or eliminate distortion of the captured document image. After the distortion of the captured document image is corrected, the embodiments disclosed herein can be operated to rotate the captured document image in its proper direction. Incidentally, the embodiments disclosed herein provide the user with a successful evaluation of implementing each step in the various embodiments of the present invention.

2 shows a general flowchart of a method of processing a captured image. After start 210, an image is received 220. The image can be received from various sources. For example, in one embodiment, the image may be received from a digital camera. In another embodiment, the image may be received from a still unit that includes a digital camera. In yet another embodiment, the image may be received from a digitized film picture. The received image 220 includes a document image. Step 230 operates to identify the captured document image from the rest of the image or the background. Step 230 is called division. This step 230 may act to detect the edge of the captured image document. This step 230 may also operate to crop the background of the image from the captured document image to separate the document from its background. Step 240, referred to as distortion removal, operates to reduce or eliminate distortion of the captured document image. Some distortions that can be corrected by this step 240 are perspective distortion, lens distortion, and light distortion. Other distortions may also be corrected at this step 240. Step 250 operates to correct the orientation of the document. This step 250 may determine whether the captured document image should be in portrait or landscape orientation and thereby rotate the captured document image. This step 250 may also operate to determine if the captured document image is upside down and to rotate the captured document image accordingly. In step 260, the processed document image is output. The processed document image may be output through various means, such as displaying an image on the monitor of the processed document image, storing the processed document image in a computer file, electronically transmitting the document image, or printing the processed document image. (260).

3 is a flowchart 300 of another embodiment of a method of processing a captured image. After start 305, an image is received 310. In step 315, the received image is converted into a device independent bit map. In step 320, partitioning is performed using edge-based partitioning processing. Edge-based segmentation 320 processing identifies the edges of the captured image document to distinguish the captured document image from its background.

4 is a flowchart of an embodiment of an edge based segmentation 320. In this embodiment, the location of the horizontal and vertical edge points is found. This is done by searching for edge points. The edge point is determined by identifying the portion of the received image that includes the transition from the background portion of the received image to the document portion of the received image. In one embodiment, the received image is scanned starting at the center of the received image (410) and also scanning starting at the boundary of the received image (420). In one embodiment, the document image is assumed to occupy the center of the received image. In another embodiment, the non-text portion of the captured document image is assumed to have a higher pixel intensity than its background. In scanning 410 starting from the center of the received image, after finding an area that can be identified as a document pixel, the transition to the background pixel is searched along the scan. In scanning 420 starting from the boundary of the received image, an area is identified as a background pixel and a transition to the document image pixel is identified. Processing may be performed using one or both of these scans 410, 420. In one embodiment, the received image is scanned 410, 420 in both horizontal and vertical directions.

Then, a random sample consensus step 430 is executed. 5 illustrates one embodiment of a random sample summation step. In this embodiment, the random sample agreement 430 is executed by selecting 510 randomly two points from the edge points selected in steps 410 and 420. Then, a line connecting these two randomly selected points is calculated (520). In one embodiment, angle-distance coordinates are used, where the angle value corresponds to the angle of the line segment around the center of the received image, and the distance value corresponds to the distance from the center of the received image to the closest point in the line segment. do. In other embodiments, other coordinate systems may be used, including, for example, Cartesian or polar coordinates. These values are then stored. The process of selecting two random points from the edge points obtained in steps 410 and 420 is repeated to obtain a sufficient sample group (530). In one embodiment, this process is repeated 5,000 times, although different sample sizes may be used. After sampling, pairs of points all lying on the same line are grouped into bins. If the initial edge points selected in steps 410 and 420 accurately represent the edges of the document in the received image, then approximately one quarter points are distributed in four small ranges corresponding to four document edges, while the remaining points are The remaining possible coordinates will generally spread evenly. Four sets of grouping segments that have the largest grouping segment (540) and that meet the minimum threshold of the grouping segments are identified as representing 550 four edges of the document in the received image. In one embodiment, these sets of segments are then determined to be left, right, top and bottom edges according to their relative position in the received image.

After the random sample agreement 430 is executed, in one embodiment, the edge removal step 440 is performed among a set of edge points to further refine the identification of the document edge. In one embodiment, shown in FIG. 6, this is done by performing a linear regression between the set of edge points corresponding to one of the edges of the received document image. In the linear regression technique, a line is drawn (610) to which the set of edge points is most accurately connected. If the point furthest from this linear regression line is determined to be sufficiently far from the linear regression line (620), the point is removed (630) and a new linear regression is performed. This process is repeated until the point farthest from the linear regression line is within the threshold and the final linear regression line is determined to be an edge line. This is done at each of four sets of edge points representing the four edges of the received image document.

Referring again to FIG. 3, at step 325, an accuracy calculation of edge line identification from edge based segmentation 320 is determined. This step 325 may be referred to as reliability calculation. In one embodiment, the confidence is calculated for each edge of the received document image and the lowest value is determined as the overall confidence. In another embodiment, the highest confidence value of the edge lines is determined as the overall confidence. In yet another embodiment, a reliability combination of edge lines is used, such as, for example, the reliability average of the line edges to determine the overall reliability. One embodiment of calculating the determination reliability of a particular line edge calculates the ratio between the number of pixel points remaining in the set of edges after edge removal 440 and the total number of pixel points that could be found at that edge. will be. Reliability determination can be used to improve distortion reduction 240 and 350 of the received document image, and can also be used to inform the user of the accuracy of system performance for a particular received image. In step 330, if the reliability in edge-based segmentation step 320 is not high enough, the content-based segmentation of step 335 is performed.

A content based segmentation step 335 of which one embodiment is shown in FIG. 7 identifies the text of the captured image document and calculates the edge of the captured image document in relation to the text. This is accomplished by identifying the connected elements in the received document image (710) and finding the nearest neighbor to those elements (720). Connected elements generally represent those black or dark pixels adjacent to each other. Then, their adjacent pixels are connected by a line (730), which lines are then used to determine the boundaries of the text (740). From these boundaries, a margin is added 750 to identify the edge location of the received document image. The size of the margin can vary, in one embodiment, a standard margin is added at step 750.

In step 340, the corners of the captured document image are calculated. In one embodiment, the corner may be calculated from the intersection of the edge lines.

Distortion removal 240 and 350 may involve multiple adjustments to the received image. In one embodiment, distortion removal 240, 350 will adjust the received document image to correct for perspective distortion in the received image. For example, in situations where the picture is not taken from the angle just above and not in the center of the document, there will be perspective distortion of the received document image.

One embodiment of adjusting the image to correct perspective distortion is shown in FIG. 8. This embodiment involves mapping 810 a set of image coordinates, for example (x, y), to a new set of image coordinates, for example (u, v). After the dividing step 230, 320, 335, four corners of the document are determined 340. Typically, in documents containing perspective distortion, these four corners correspond to a trapezoid while the document will generally have a rectangular shape. Therefore, in one embodiment, mapping 810 is performed between the received trapezoids and the desired rectangles. One embodiment of achieving this mapping 810 is a homogeneous between the non-distorted pixel coordinates and the distorted pixel coordinates through a homogeneous matrix representing the transformation from the distorted pixel coordinates to the non-distorted pixel coordinates known in the art. Is to use a transform. The transform can be calculated by comparing the four corners determined during segmentation 230, 320, 335 with the corrected size of the non-distorted received document image. In one embodiment, the need to calculate the transform at each pixel point can be eliminated by simply calculating the transform for each line and using linear interpolation to calculate the new pixel coordinates. After mapping the new coordinates corresponding to the document with the reduced perspective distortion, resampling of the pixels is performed (815).

Another aspect of the received image that may be adjusted in the distortion removal step 240, 350 is the adjustment of the distortion caused by the camera lens (820). The distortion caused by the camera lens can be made differently to bend straight lines. This distortion depends on the particular lens used and the distance of the camera from the captured image. Since the curvature produced by lens distortion will generally be radial, uniform radius adjustment of lens distortion can be performed using parameters that approximate the degree of lens distortion. This parameter can be calculated by the system or entered by the user.

Another aspect of the received image that can be adjusted in the distortion removal steps 240 and 350 is the adjustment of the distortion caused by the document that is not entirely flat. For example, if the imaged document is a page in a book, the page may have a curvature that creates distortion when captured in the photograph. This distortion can also be corrected in the distortion removal steps 240 and 350. Other distortions may also be corrected, and the description of certain types of distortions herein is not intended to limit the type of distortions that may be reduced or eliminated.

In step 365, the thresholding process is performed on the image created in step 360. Thresholding process 365 has the potential advantage of reducing the color depth of the image and reducing the distortion generated by the flash that can be used when taking the image as a picture. In one embodiment, the thresholding process 365 reduces the 24-bit color image to a 1-bit black and white image. Potential advantages of reducing the image to black and white are the reduction of the effects introduced by the flash of the camera, and the reduction in the amount of information required for processing by the system 300. The thresholding process 365 can be executed in a number of ways. One embodiment may use dithering techniques known in the art. For the dithering technique can be found in the existing image software, such as snow-bound image library by Microfiche laser ^{(SNOWBOUND? IMAGE LIBRARY by Snowbound Software} Corp.). However, one disadvantage of using dithering techniques is the influx of noise into the image. Another embodiment of thresholding 365 involves the selection of a global threshold for an image. In such a technique, a threshold is selected. Pixels with intensity greater than the threshold are considered white and the remaining pixels are considered black. The threshold can be selected in several ways. In one embodiment, a threshold is selected and applied to all received images. This technique has the disadvantage that it does not account for the changed light conditions in the received image. In another embodiment, the threshold is calculated from the analysis of the received image, such as a histogram. In one such embodiment involving analysis of the received image, an assumption is made that the received image has two peaks in the intensity histogram corresponding to the foreground and background of the received document image. This embodiment cannot work well for those images for which the assumption does not apply. Another embodiment for thresholding 365 is to select a separate threshold for each pixel in the received image. This embodiment has the advantage of responding to changes in conditions in the document such as changes in lighting or background contrast. One embodiment of this technique is called adaptive thresholding. In this embodiment, the previous pixel value is taken into account when each new pixel is analyzed for the determination of the threshold. One way to achieve this is to calculate a weighted average of each pixel as each sequential pixel of the received image is analyzed. One possible drawback of this embodiment is the insertion of noise when the received image contains a colorized document.

In step 370, a line step of text is executed. In this step 370, the system determines the line of text in the received document image. 9 illustrates one embodiment of a line 370 of text. In one embodiment, the system assumes that the pixel corresponding to the text in the received document image has a lower intensity than the background pixel of the received document image. In this embodiment, the sum of the intensities of all the pixels in each row of the received document image is calculated 910. These sums are then used to identify the local highest and lowest of the pixel intensities. These highs and lows are then analyzed to determine text lines in the document. For example, if the received document image had a white background and black text lines, the pixel lines that were white entirely would have the highest total intensity, and the lines containing black text would have significantly lower pixel intensities. The difference in these intensities can then be calculated, whereby a line of text can be determined. In a preferred embodiment, the line of text 370 runs in both horizontal and vertical directions across the received document image.

Another embodiment of executing a line of text 370 is to perform a text line search similar to that performed in step 335. In one such embodiment, the text of the captured document image is identified and formed into lines. This can be accomplished by identifying linked elements in the captured document image and finding the neighbors closest to those elements. Connected elements generally represent those black or darker pixels that are adjacent to each other. Then, their adjacent pixels are connected by lines. This process is similar to that described in steps 710, 720, and 730 of FIG. 7.

Step 375 determines whether the captured document image should be in landscape or portrait format. In one embodiment, this is accomplished by comparing the text line 370 results in the vertical direction with the text line 370 results in the horizontal direction. In one embodiment, as a result, a direction with a larger number of lines is determined to orient the received document image. For example, in a received document image having a height greater than the width, if the text line 370 in the vertical direction gets a larger number of lines than the text line 370 in the horizontal direction, the received document image is horizontal. It is determined to have a direction. As another example, in the same received document image, if the text line 370 in the horizontal direction gets a larger number of lines than the text line 370 in the vertical direction, then the received document image is said to have a vertical direction. It is determined.

Step 380 determines the straight direction of the document. FIG. 10 illustrates an embodiment of determining 380 whether a received document image is properly pointed straight. In one embodiment, each line of text is analyzed. Although fewer lines of text can be analyzed, this can result in less reliable results. In one embodiment, each line of text is divided into three sections 1010: a rising section, a middle section and a falling section. English characters include certain unique statistical properties that can be used in certain embodiments to determine the straight orientation of the received document image. For example, the English alphabet has only five characters (ie g, j, p, q, and y) that go down the lower boundary of the sentence, and more characters (eg, b, d) that rise above the upper boundary of the sentence. , f, h, i, k, and q). In one embodiment, this property of the English character may be taken into account when calculating the number of pixels included in the rising and falling sections (1020) and calculating their pixel concentrations (1030, 1040). For example, a received document image with English characters with more rising character pixels than falling character pixels would probably be in a straight position and would not need to be rotated, while the same document would have more falling character pixels than rising document pixels. The document will probably need to be rotated 180 degrees (1050).

In other embodiments, other characteristics of English characters may also be considered. For example, the characteristics of the pixel position in the horizontal direction can be considered. In addition, non-statistical methods such as optical character recognition (“OCR”) can also be used to determine the straight direction of the document. Other embodiments may use a neural network approach. Similar inherent properties can also be used for non-English documents. For example, Spanish characters are similar to English, so they will have similar unique characteristics. As another example, Arabic characters include a greater number of falling characters, and embodiments may adjust for their characteristics accordingly.

FIG. 12 illustrates another embodiment of performing step 380 to determine whether a received document image is properly pointed straight. In one embodiment, connected elements are used to determine each character line of text. Each element is classified 1210 by height into two categories, namely cattle and cages. The center of the text line is determined 1220. In one embodiment, lowercase height is used to determine the center of the text line (1220). This can improve the evaluation of text line-centric evaluation in the case of distortion, such as when bending across the page. The capital letters are then aligned to the center of the text line and grouped up or down based on their relative position to the center (1230). The total number of rising and falling letters is calculated. In a typical English document, many characters will go to the top of the page. Therefore, in one embodiment, if the number of rising capital letters is greater than the number of falling capital letters, the document does not need to be rotated in step 385 before being output in step 390. However, if the number of falling capital letters is greater than the number of rising capital letters, the document is rotated in step 385 before being output in step 390.

The image is then rotated in step 385 according to the determination of steps 380 and 375. Then, a new document image is output 390.

As described above, system imaged documents can be captured with a film camera or digital camera. As an alternative to these freestyle devices, a still camera system can be used to capture the imaged document. 11 illustrates an embodiment of a still camera system for capturing document images. In this embodiment, document 1110 is placed on base 1120 of the system. In a preferred embodiment, the base 1120 of the system is of a predetermined color, which may have the advantage of facilitating the segmentation process described above. Extending from the base 1120 is a stand 1130 that can receive the camera 1140 and the illumination 1150. The camera and lights may be permanently housed in the stand 1130 or may be detachable or adjustable. The illumination may be placed anywhere on the base 1120 or stand 1130. In other embodiments, there is no additional illumination included on the base 1120 or stand 1130. In yet another embodiment, the illumination is separated from the base 1120 or stand 1130. The still system is then connected to the computer 1160 to perform the above described processing of the received image document. In other embodiments, the computer may also be embedded in the device. In yet another embodiment, the captured image document may simply be stored in the digital camera 1140 or in another memory source and later connected to a computer for processing. Such a still camera system can be deployed, for example, as part of a user workstation in an office.

In contrast to free-form cameras, there are several advantages to using still camera systems. For example, when using a still camera system, the amount of perspective distortion can be reduced because the document is more likely to be perpendicular to the camera lens and centered relative to the camera lens. Another advantage is that the system can better adjust lens distortion because the distance between the camera and the lens used is known, thereby reducing the need to calculate or approximate these parameters. Another possible advantage may be to reduce the distortion produced by the camera flash. In a preferred embodiment, the illumination 1150 of the stationary system may be positioned to reduce glare or other distortion produced by the camera flash.

The approach described herein for processing a captured image is applicable to any type of processing application, and is particularly suitable for computer-based applications that process (without limitation) a captured image. The scheme described herein may be implemented in hardware circuitry, computer software, or a combination of hardware circuitry and computer software, and is not limited to any specific hardware or software implementation.

13 is a block diagram illustrating a computer system 1300 in which an embodiment of the present invention may be implemented. Computer system 1300 includes a bus 1345 or other communication mechanism for communicating information, and a processor 1335 connected to the bus 1345 to process information. Computer system 1300 is also coupled to bus 1345 and includes main memory 1320, such as random access memory (RAM) or other dynamic storage device, that stores information and instructions to be executed by processor 1335. Main memory 1320 may also be used to store temporary variables or other intermediate information during execution of instructions to be executed by processor 1335. Computer system 1300 further includes read-only memory (ROM) 1325 or other static storage device coupled to bus 1345 for storing static information and instructions for processor 1335. A storage device 1330, such as a magnetic disk or an optical disk, is provided and connected to the bus 1345 to store information and instructions.

Computer system 1300 may be connected via bus 1345 to a display 1305, such as a cathode ray tube (CRT) for displaying information to a computer user. Input device 1310 comprising alphanumeric and other keys is coupled to bus 1345 to communicate information and command selections to processor 1335. Another type of user input device is a cursor controller 1315, such as a mouse, trackball or cursor direction key, for communicating direction information and command selection to the processor 1335 and controlling cursor movement on the display 1305. This input device typically has two degrees of freedom in two axes, i.e., the first axis (e.g. x) and the second axis (e.g. y), which allow the device to specify locations within one plane. .

The method described herein relates to the use of computer system 1300 to process a captured image. According to one embodiment, the processing of the captured image is provided by computer system 1300 in response to processor 1335 executing one or more sequences of one or more instructions contained in main memory 1320. Such instructions can be read into main memory 1320 from other computer-readable media, such as storage device 1330. Execution of the sequence of instructions contained in main memory 1320 causes processor 1335 to execute the processing steps described herein. One or more processors in a multi-processing configuration may also be used to execute the sequence of instructions contained in main memory 1320. In alternative embodiments, hard-wired circuitry may be used in place of or in combination with software instructions to implement the embodiments described herein. Therefore, the embodiments described herein are not limited to any particular combination of hardware circuitry and software.

The term "computer-readable medium" as used herein refers to any medium that participates in providing instructions to the processor 1335 for execution. Such media can take various forms, including but not limited to non-volatile media, volatile media, and transmission media. Non-volatile media includes, for example, optical or magnetic disks, such as storage device 1330. Volatile media includes dynamic memory, such as main memory 1320. The transmission medium includes coaxial cables, copper wires, and optical fibers, including the wires that make up the bus 1345. The transmission medium may also take the form of acoustic or light waves, such as those generated during radio wave and infrared data communications.

Common forms of computer readable media include, for example, floppy disks, flexible disks, hard disks, magnetic tape, or any other magnetic media, CD-ROM, any other optical media, punch cards, paper tapes, hole patterns. Any other physical medium having, RAM, ROM and EPROM, flash-EPROM, any other memory chip or cartridge, carrier described below, or any other medium readable by a computer.

Various forms of computer readable media may be involved in carrying one or more sequences of one or more instructions to processor 1335 for execution. For example, the instructions may be initially included on the magnetic disk of the remote computer. The remote computer can load the instructions into dynamic memory and send the instructions over a telephone line using a modem. A modem limited to computer system 1300 may receive data over a telephone line and use an infrared transmitter to convert the data into an infrared signal. An infrared detector connected to the bus 1345 may receive data placed in an infrared signal and place the data on the bus 1345. The bus 1345 delivers data to the main memory 1320, and the processor 1335 retrieves and executes instructions from the main memory 1320. Instructions received by main memory 1320 may optionally be stored on storage device 1330 before or after execution by processor 1335.

Computer system 1300 also includes a communication interface 1340 coupled to bus 1345. The communication interface 1340 connects to a network link 1375 that is connected to the local network 1355 to provide bidirectional data communication. For example, communication interface 1340 may be an Integrated Services Digital Network (ISDN) card or modem for providing data communication to a corresponding type of telephone line. As another example, communication interface 1340 may be a local area network (LAN) card for providing a data communication connection to a compatible LAN. Wireless links may also be implemented. In any such implementation, communication interface 1340 transmits and receives electrical, electronic or optical signals that carry digital data streams representing various types of information.

Network link 1375 typically provides data communication to other data services over one or more networks. For example, the network link 1375 may provide a connection over the local network 1355 to the host computer 1350 or to data equipment operated by an Internet service provider (ISP) 1365. ISP 1365 provides data communication services over a world wide packet data communication network, which is now generally referred to as " Internet " Both local network 1355 and the Internet 1360 use electrical, electronic or optical signals that carry digital data streams. Signals over various networks, and signals over communication interface 1340 on network link 1375 and transferring digital data to / from computer system 1300 are exemplary forms of carriers for conveying information.

Computer system 1300 may include program code via network (s), network link 1375 and communication interface 1340 to transmit messages and receive data. In the Internet example, the server 1370 may send the requested code for the application via the Internet 1360, ISP 1365, local network 1355, and communication interface 1340. According to the present invention, such one downloaded application provides for the processing of the captured image as described herein.

The received code may be executed by the processor 1335 when received and / or stored for later execution in storage 1330 or other non-volatile storage. In this manner, computer system 1300 may obtain application code in the form of a carrier wave.

Claims (10)

A method of processing a captured image that includes an imaged document, the method comprising: Detecting graphical information of the captured image, related to edges of the imaged document; Separating the imaged document from the background of the captured image based on the detected graphic information related to edges of the imaged document; Calculating deviations of the imaged document from a non-distorted perspective of the imaged document; Based on the calculated deviations, mapping pixels of the imaged document to a new image of the imaged document with reduced perspective distortion; Resampling pixels of the new image; Detecting graphic information of the new image of the imaged document, related to the orientation of the imaged document in the new image; And Rotating the new image based on the graphical information related to the orientation of the imaged document in the new image And a captured image comprising the imaged document. The method of claim 1, Detecting graphical information in the captured image may include detecting graphical information in the captured image in response to a transition between the imaged document and the rest of the captured image, and detecting the graphical information in the captured image. By selecting one or more lines from the graphical information corresponding to edges, The separating step is performed by separating the imaged document from the background of the captured image based on the one or more lines corresponding to edges of the imaged document, And said calculating step is performed by calculating deviations of corners of said imaged document from a non-distorted perspective of said imaged document. 3. The method of claim 2, Calculating corners of the imaged document based on intersections of the one or more lines corresponding to edges of the imaged document; And Based on the calculated deviation, mapping coordinates of pixels of the imaged document to coordinates corresponding to a non-distorted perspective of the imaged document. And further comprising a captured image comprising the imaged document. A method of processing a captured image that includes an imaged document, the method comprising: Detecting graphical information in the captured image regarding a transition between the imaged document and the remaining portion of the captured image; Selecting one or more lines from the graphical information corresponding to edges of the imaged document; Calculating corners of the imaged document based on intersections of the one or more lines corresponding to edges of the imaged document; Separating the imaged document from the background of the captured image based on the one or more lines corresponding to edges of the imaged document; Calculating deviations of corners of the imaged document from a non-distorted perspective of the imaged document; Generating a new image of the imaged document with reduced perspective distortion based on the calculated deviations; Mapping coordinates of pixels of the imaged document to coordinates corresponding to a non-distorted perspective of the imaged document based on the calculated deviation; Converting a non-distorted imaged document into a two-color representation of the imaged document; Calculating pixel intensities of the two-color representations along the vertical axis of the non-distorted imaged document; Calculating pixel intensities of the two-color representations along a horizontal axis of the non-distorted imaged document; Identifying contrasts of pixel intensities along a vertical and horizontal axis of the non-distorted imaged document; Identifying text lines of the imaged document based on the contrasts of the pixel intensities; Determining a format of the non-distorted imaged document based on a direction of the text lines of the non-distorted imaged document relative to sizes of the edges of the imaged document; And Rotating the non-distorted imaged document in accordance with the format determination of the non-distorted imaged document. And a captured image comprising the imaged document. 5. The method of claim 4, Dividing the text lines into three portions along the vertical axis of the text lines; Determining the direction of the text lines based on a comparison of pixel intensities of the portions of the text lines; And Rotating the non-distorted imaged document based on the direction determination And further comprising a captured image comprising the imaged document. delete delete delete A computer readable storage medium for processing a captured image comprising an imaged document, the computer readable storage medium comprising: The computer readable storage medium has one or more sequences of one or more instructions that, when executed by one or more processors, cause the one or more processors to execute computer-implemented steps, The computer-implemented steps, Detecting graphical information of the captured image, related to edges of the imaged document; Separating the imaged document from the background of the captured image based on the detected graphic information related to edges of the imaged document; Calculating deviations of the imaged document from the non-distorted perspective of the imaged document; Mapping pixels of the imaged document to a new image of the imaged document with reduced perspective distortion based on the calculated deviations; Resampling pixels of the new image; Detecting graphic information of the new image of the imaged document, related to the orientation of the imaged document in the new image; And Rotating the new image based on the graphical information related to the orientation of the imaged document in the new image And a captured image comprising the imaged document. An apparatus for processing a captured image comprising an imaged document, the apparatus comprising: One or more processors; Memory communicatively coupled to the one or more processors; And One or more modules included in the memory and configured to be executed by the one or more processors Including; The one or more modules, A module for detecting graphical information of the captured image, related to edges of the imaged document; A module for separating the imaged document from the background of the captured image based on the detected graphic information related to edges of the imaged document; A module for calculating deviations of the imaged document from the non-distorted perspective of the imaged document; A module for mapping pixels of the imaged document to a new image of the imaged document with reduced perspective distortion based on the calculated deviations; A module for resampling pixels of the new image; A module for detecting graphical information of the new image of the imaged document, related to the orientation of the imaged document in the new image; And And a module for rotating the new image based on the graphical information related to the orientation of the imaged document in the new image.

Images