US9076058B2 - Methods, systems and apparatus for determining orientation in a document image - Google Patents

Methods, systems and apparatus for determining orientation in a document image Download PDF

Info

Publication number
US9076058B2
US9076058B2 US13753423 US201313753423A US9076058B2 US 9076058 B2 US9076058 B2 US 9076058B2 US 13753423 US13753423 US 13753423 US 201313753423 A US201313753423 A US 201313753423A US 9076058 B2 US9076058 B2 US 9076058B2
Authority
US
Grant status
Grant
Patent type
Prior art keywords
text
image
pixel
wise
differences
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active, expires
Application number
US13753423
Other versions
US20140212047A1 (en )
Inventor
Ahmet Mufit Ferman
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Sharp Corp
Original Assignee
Sharp Laboratories of America Inc
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Grant date

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING; COUNTING
    • G06KRECOGNITION OF DATA; PRESENTATION OF DATA; RECORD CARRIERS; HANDLING RECORD CARRIERS
    • G06K9/00Methods or arrangements for reading or recognising printed or written characters or for recognising patterns, e.g. fingerprints
    • G06K9/20Image acquisition
    • G06K9/32Aligning or centering of the image pick-up or image-field
    • G06K9/3208Orientation detection or correction, e.g. rotation of multiples of 90 degrees
    • GPHYSICS
    • G06COMPUTING; CALCULATING; COUNTING
    • G06KRECOGNITION OF DATA; PRESENTATION OF DATA; RECORD CARRIERS; HANDLING RECORD CARRIERS
    • G06K9/00Methods or arrangements for reading or recognising printed or written characters or for recognising patterns, e.g. fingerprints
    • G06K9/20Image acquisition
    • G06K9/32Aligning or centering of the image pick-up or image-field
    • G06K9/3275Inclination (skew) detection or correction of characters or of image to be recognised
    • G06K9/3283Inclination (skew) detection or correction of characters or of image to be recognised of characters or characters lines
    • GPHYSICS
    • G06COMPUTING; CALCULATING; COUNTING
    • G06KRECOGNITION OF DATA; PRESENTATION OF DATA; RECORD CARRIERS; HANDLING RECORD CARRIERS
    • G06K9/00Methods or arrangements for reading or recognising printed or written characters or for recognising patterns, e.g. fingerprints
    • G06K9/62Methods or arrangements for recognition using electronic means
    • G06K9/68Methods or arrangements for recognition using electronic means using sequential comparisons of the image signals with a plurality of references in which the sequence of the image signals or the references is relevant, e.g. addressable memory
    • G06K9/6807Dividing the references in groups prior to recognition, the recognition taking place in steps; Selecting relevant dictionaries
    • G06K9/6814Dividing the references in groups prior to recognition, the recognition taking place in steps; Selecting relevant dictionaries according to the graphical properties
    • G06K9/6821Alphabet recognition, e.g. Latin, Kanji, Katakana

Abstract

Aspects of the present invention are related to systems, methods and apparatus for determining the orientation of a text line or a page in a document image. According to a first aspect of the present invention, an image of text blobs may be generated from a text mask associated with a document image. From the text-blob image, pixel-wise horizontal differences may be accumulated and compared to accumulated pixel-wise vertical differences. Text line orientation may be determined as horizontal when the accumulated pixel-wise horizontal differences are less than the accumulated pixel-wise vertical differences. According to a second aspect of the present invention, page orientation may be determined by reconciling an estimated text-line orientation with document language information.

Description

FIELD OF THE INVENTION

Embodiments of the present invention relate generally to methods, systems and apparatus for document processing and, in particular, to methods, systems and apparatus for determining the orientation of a text line or a page in a document image.

BACKGROUND

Orientation detection may be an important pre-processing step in a document workflow. An image of a document may be captured at an orientation other than the nominal reading position, and the captured document image may require rotation prior to subsequent processing and/or prior to presentation for content consumption. Computationally fast and efficient methods and systems for orientation detection that do not require complicated and memory-intensive processing methods may be desirable.

SUMMARY

Embodiments of the present invention comprise methods, systems and apparatus for determining the orientation of a text line or a page in a document image.

According to a first aspect of the present invention, an image of text blobs may be generated from a text mask associated with a document image. From the text-blob image, pixel-wise horizontal differences may be accumulated and compared to accumulated pixel-wise vertical differences. Text line orientation may be determined as horizontal when the accumulated pixel-wise horizontal differences are less than the accumulated pixel-wise vertical differences.

According to a second aspect of the present invention, a text mask may be generated from an input document image.

According to a third aspect of the present invention, an image of text blobs may be generated from a down-sampled version of a text mask associated with a document image. From the text-blob image, pixel-wise horizontal differences may be accumulated and compared to accumulated pixel-wise vertical differences. Text line orientation may be determined as horizontal when the accumulated pixel-wise horizontal differences are less than the accumulated pixel-wise vertical differences.

According to a fourth aspect of the present invention, page orientation may be determined by reconciling an estimated text-line orientation with document language information.

The foregoing and other objectives, features, and advantages of the invention will be more readily understood upon consideration of the following detailed description of the invention taken in conjunction with the accompanying drawings.

BRIEF DESCRIPTION OF THE SEVERAL DRAWINGS

FIG. 1 is a chart showing exemplary embodiments of the present invention comprising comparing accumulated pixel-wise vertical differences and accumulated pixel-wise horizontal differences to determine text-line orientation, in a document image, from a received text mask associated with the document image;

FIG. 2 is a chart showing exemplary embodiments of the present invention comprising comparing accumulated pixel-wise vertical differences and accumulated pixel-wise horizontal differences to determine text-line orientation, in a document image, from a received text mask associated with the document image, and wherein, a horizontal text line may be determined when the accumulated pixel-wise horizontal differences is less than the accumulated pixel-wise vertical differences;

FIG. 3 is a chart showing exemplary embodiments of the present invention comprising comparing accumulated pixel-wise vertical differences and accumulated pixel-wise horizontal differences to determine text-line orientation, in a document image, from a down-sampled version of a received text mask associated with the document image;

FIG. 4 is a chart showing exemplary embodiments of the present invention comprising comparing accumulated pixel-wise vertical differences and accumulated pixel-wise horizontal differences to determine text-line orientation, in a document image, from a down-sampled version of a received text mask associated with the document image, and wherein, a horizontal text line may be determined when the accumulated pixel-wise horizontal differences is less than the accumulated pixel-wise vertical differences;

FIG. 5 is a chart showing exemplary embodiments of the present invention comprising comparing accumulated pixel-wise vertical differences and accumulated pixel-wise horizontal differences to determine text-line orientation, in a document image, from a received image and a text mask generated from the received image;

FIG. 6 is a chart showing exemplary embodiments of the present invention comprising comparing accumulated pixel-wise vertical differences and accumulated pixel-wise horizontal differences to determine text-line orientation, in a document image, from a received image and a text mask generated from the received image, and wherein, a horizontal text line may be determined when the accumulated pixel-wise horizontal differences is less than the accumulated pixel-wise vertical differences;

FIG. 7 is a picture illustrating exemplary embodiments of the present invention comprising a comparator for comparing accumulated pixel-wise vertical differences and accumulated pixel-wise horizontal differences to determine text-line orientation, in a document image, from a received image and a text mask generated from the received image or from a received text mask associated with a document image;

FIG. 8A is a picture depicting an exemplary input image;

FIG. 8B is a picture depicting a luminance image generated from the exemplary input image shown in FIG. 8A;

FIG. 8C is a picture depicting a text mask generated from the luminance image depicted in FIG. 8B;

FIG. 8D is a picture showing a down-sampled version of the text mask shown in FIG. 8C;

FIG. 8E is a picture depicting a text-blob image generated by applying a dilation operation to the down-sampled text mask, shown in FIG. 8D, and subsequently applying a hole filling operation to the results of the dilation operation;

FIG. 8F is a picture depicting a text-blob image generated by applying morphological closing to the down-sampled text mask shown in FIG. 8D;

FIG. 8G is a picture depicting a text-blob image generated by applying a smearing operation to the down-sampled text mask shown in FIG. 8D; and

FIG. 9 is a chart showing exemplary embodiments of the present invention comprising reconciling a text-line orientation estimate with received document language information to determine page orientation.

DETAILED DESCRIPTION OF EXEMPLARY EMBODIMENTS

Embodiments of the present invention will be best understood by reference to the drawings, wherein like parts are designated by like numerals throughout. The figures listed above are expressly incorporated as part of this detailed description.

It will be readily understood that the components of the present invention, as generally described and illustrated in the figures herein, could be arranged and designed in a wide variety of different configurations. Thus, the following more detailed description of the embodiments of the methods, systems and apparatus of the present invention is not intended to limit the scope of the invention, but it is merely representative of the presently preferred embodiments of the invention.

Elements of embodiments of the present invention may be embodied in hardware, firmware and/or a non-transitory computer program product comprising a computer-readable storage medium having instructions stored thereon/in which may be used to program a computing system. While exemplary embodiments revealed herein may only describe one of these forms, it is to be understood that one skilled in the art would be able to effectuate these elements in any of these forms while resting within the scope of the present invention.

Although the charts and diagrams in the figures may show a specific order of execution, it is understood that the order of execution may differ from that which is depicted. For example, the order of execution of the blocks may be changed relative to the shown order. Also, as a further example, two or more blocks shown in succession in a figure may be executed concurrently, or with partial concurrence. It is understood by those with ordinary skill in the art that a non-transitory computer program product comprising a computer-readable storage medium having instructions stored thereon/in which may be used to program a computing system, hardware and/or firmware may be created by one of ordinary skill in the art to carry out the various logical functions described herein.

Orientation detection may be an important pre-processing step in a document workflow. An image of a document may be captured at an orientation other than the nominal reading position, and the captured document image may require rotation prior to subsequent processing and/or prior to presentation for content consumption. Computationally fast and efficient methods and systems for orientation detection that do not require complicated and memory-intensive processing methods may be desirable.

Some embodiments of the present invention may be described in relation to FIG. 1. FIG. 1 illustrates exemplary method(s) 100 for determining the line orientation in a document image. The document image may be an image generated by scanning a document page, a camera-captured image of a document page or another digital image of a document page.

A text mask associated with a document image may be received 102 in a document processing system comprising a processor. The text mask comprises a binary image, wherein one of the binary values (“one” or “zero”) corresponds to pixels that may be considered a portion of a text component in the document image and the other of the binary values corresponds to pixels that may not be considered a portion of a text component in the document image. In some embodiments of the present invention, the resolution of the text mask may be the same as the resolution of a captured document image. In alternative embodiments of the present invention, the resolution of the text mask may be less than the resolution of the captured document image.

From the text mask, text blobs may be generated 104, wherein neighboring text characters in a text line may be merged into one, or more, text blobs, thereby generating a text-blob image.

In some embodiments of the present invention, text-blob generation, also considered text-blob image generation, may comprise morphological processing methods. In some of these embodiments, a dilation operation may be applied to the text mask, and a hole filling operation may be applied to the results of the dilation operation. In alternative embodiments, morphological closing, comprising a dilation operation followed by an erosion operation using the same structuring element, may be used to connect neighboring text characters in a text line. The closing operation may create text blobs that cover words and/or entire text lines, but may produce a relatively small number of cross-line merges.

In still alternative embodiments, a smearing operation may be used to merge text line components. In the smearing operation, the distance between successive text pixels along a row, and a column, may be computed. When the distance between successive text pixels is less than a predetermined threshold, all non-text pixels between the successive text pixels may be set to the binary value associated with text pixels. In some embodiments of the present invention, the predetermined threshold for row processing may be the same as the predetermined threshold for column processing. In some embodiments of the present invention, the predetermined threshold for row processing may be different than the predetermined threshold for column processing. In some embodiments of the present invention, a predetermined threshold value of 10 may be used for the row processing, and a predetermined threshold value of 3 may be used for column processing. In some embodiments of the present invention, the predetermined threshold values may be related to the expected language family of the document.

Pixel-wise horizontal differences may be accumulated 106 and pixel-wise vertical differences may be accumulated 108 from the text-blob image.

In some embodiments of the present invention the accumulated pixel-wise horizontal differences may be determined according to:

Δ H = x , y M TextBlob ( x , y ) - M TextBlob ( x - 1 , y ) ,
where ΔH may denote the accumulated pixel-wise horizontal differences, x may denote the horizontal pixel-location index, y may denote the vertical pixel-location index,

x , y
may denote the summation over valid pixel locations, for example, pixel locations on the left vertical edge of the image may be excluded from the accumulation due to the non-existence of a horizontal left-neighboring pixel, and |•| may denote absolute value.

In some embodiments of the present invention the accumulated pixel-wise vertical differences may be determined according to:

Δ V = x , y M TextBlob ( x , y ) - M TextBlob ( x , y - 1 ) ,
where ΔV may denote the accumulated pixel-wise vertical differences, x may denote the horizontal pixel-location index, y may denote the vertical pixel-location index,

x , y
may denote the summation over valid pixel locations, for example, pixel locations on the top horizontal edge of the image may be excluded from the accumulation due to the non-existence of a vertical top-neighboring pixel, and |•| may denote absolute value.

In some embodiments of the present invention the accumulated pixel-wise horizontal differences may be determined according to:

Δ H = x , y M TextBlob ( x , y ) M TextBlob ( x - 1 , y ) ,
where ⊕ may denote a logical exclusive-OR operation.

In some embodiments of the present invention the accumulated pixel-wise vertical differences may be determined according to:

Δ V = x , y M TextBlob ( x , y ) M TextBlob ( x , y - 1 ) ,
where ⊕ may denote a logical exclusive-OR operation.

The accumulated pixel-wise horizontal differences and the accumulated pixel-wise vertical differences may be compared 110, and the line orientation may be selected 112 based on the comparison of the accumulated pixel-wise horizontal differences and the accumulated pixel-wise vertical differences.

Some embodiments of the present invention may be described in relation to FIG. 2. FIG. 2 illustrates exemplary method(s) 200 for determining the line orientation in a document image. The document image may be an image generated by scanning a document page, a camera-captured image of a document page or another digital image of a document page.

A text mask associated with a document image may be received 202 in a document processing system comprising a processor. The text mask comprises a binary image, wherein one of the binary values (“one” or “zero”) corresponds to pixels that may be considered a portion of a text component in the document image and the other of the binary values corresponds to pixels that may not be considered a portion of a text component in the document image. In some embodiments of the present invention, the resolution of the text mask may be the same as the resolution of a captured document image. In alternative embodiments of the present invention, the resolution of the text mask may be less than the resolution of the captured document image.

From the text mask, text blobs may be generated 204, wherein neighboring text characters in a text line may be merged into one, or more, text blobs, thereby generating a text-blob image.

In some embodiments of the present invention, text-blob generation, also considered text-blob image generation, may comprise morphological processing methods. In some of these embodiments, a dilation operation may be applied to the text mask, and a hole filling operation may be applied to the results of the dilation operation. In alternative embodiments, morphological closing, comprising a dilation operation followed by an erosion operation using the same structuring element, may be used to connect neighboring text characters in a text line. The closing operation may create text blobs that cover words and/or entire text lines, but may produce a relatively small number of cross-line merges.

In still alternative embodiments, a smearing operation may be used to merge text line components. In the smearing operation the distance between successive text pixels along a row, and a column, may be computed. When the distance between successive text pixels is less than a predetermined threshold, all non-text pixels between the successive text pixels are set to the binary value associated with text pixels. In some embodiments of the present invention, the predetermined threshold for row processing may be the same as the predetermined threshold for column processing. In some embodiments of the present invention, the predetermined threshold for row processing may be different than the predetermined threshold for column processing. In some embodiments of the present invention, a predetermined threshold value of 10 may be used for the row processing, and a predetermined threshold value of 3 may be used for column processing. In some embodiments of the present invention, the predetermined threshold values may be related to the expected language family of the document.

Pixel-wise horizontal differences may be accumulated 206 and pixel-wise vertical differences may be accumulated 208 from the text-blob image.

In some embodiments of the present invention the accumulated pixel-wise horizontal differences may be determined according to:

Δ H = x , y M TextBlob ( x , y ) - M TextBlob ( x - 1 , y ) ,
where ΔH may denote the accumulated pixel-wise horizontal differences, x may denote the horizontal pixel-location index, y may denote the vertical pixel-location index,

x , y
may denote the summation over valid pixel locations, for example, pixel locations on the left vertical edge of the image may be excluded from the accumulation due to the non-existence of a horizontal left-neighboring pixel, and |•| may denote absolute value.

In some embodiments of the present invention the accumulated pixel-wise vertical differences may be determined according to:

Δ V = x , y M TextBlob ( x , y ) - M TextBlob ( x , y - 1 ) ,
where ΔV may denote the accumulated pixel-wise vertical differences, x may denote the horizontal pixel-location index, y may denote the vertical pixel-location index,

x , y
may denote the summation over valid pixel locations, for example, pixel locations on the top horizontal edge of the image may be excluded from the accumulation due to the non-existence of a vertical top-neighboring pixel, and |•| may denote absolute value.

In some embodiments of the present invention the accumulated pixel-wise horizontal differences may be determined according to:

Δ H = x , y M TextBlob ( x , y ) M TextBlob ( x - 1 , y ) ,
where ⊕ may denote a logical exclusive-OR operation.

In some embodiments of the present invention the accumulated pixel-wise vertical differences may be determined according to:

Δ V = x , y M TextBlob ( x , y ) M TextBlob ( x , y - 1 ) ,
where ⊕ may denote a logical exclusive-OR operation.

The accumulated pixel-wise horizontal differences and the accumulated pixel-wise vertical differences may be compared 210. When the accumulated pixel-wise horizontal differences is less than the accumulated pixel-wise vertical differences 212, then the text-line orientation of the document image may be classified 214 as horizontal. When the accumulated pixel-wise horizontal differences is not less than the accumulated pixel-wise vertical differences 216, then the text-line orientation of the document image may be classified 218 as vertical.

Some embodiments of the present invention may be described in relation to FIG. 3. FIG. 3 illustrates exemplary method(s) 300 for determining the line orientation in a document image. The document image may be an image generated by scanning a document page, a camera-captured image of a document page or another digital image of a document page.

A text mask associated with a document image may be received 302 in a document processing system comprising a processor. The text mask comprises a binary image, wherein one of the binary values (“one” or “zero”) corresponds to pixels that may be considered a portion of a text component in the document image and the other of the binary values corresponds to pixels that may not be considered a portion of a text component in the document image.

The text mask may be down sampled 304. In some embodiments of the present invention, the down sampling may comprise nearest-neighbor interpolation to generate a lower-resolution version of the text mask. In some embodiments of the present invention, the scaling factor used in the down sampling operation may be determined according to:

S = 640 max ( r i , w i ) ,
where S may denote the scaling factor, ri and wi may denote the height and width of the input text mask, respectively, and max(•,•) may denote a maximum operation.

From the down-sampled text mask, text blobs may be generated 306, wherein neighboring text characters in a text line may be merged into one, or more, text blobs, thereby generating a text-blob image.

In some embodiments of the present invention, text-blob generation, also considered text-blob image generation, may comprise morphological processing methods. In some of these embodiments, a dilation operation may be applied to the text mask, and a hole filling operation may be applied to the results of the dilation operation. In alternative embodiments, morphological closing, comprising a dilation operation followed by an erosion operation using the same structuring element, may be used to connect neighboring text characters in a text line. The closing operation may create text blobs that cover words and/or entire text lines, but may produce a relatively small number of cross-line merges.

In still alternative embodiments, a smearing operation may be used to merge text line components. In the smearing operation the distance between successive text pixels along a row, and a column, may be computed. When the distance between successive text pixels is less than a predetermined threshold, all non-text pixels between the successive text pixels are set to the binary value associated with text pixels. In some embodiments of the present invention, the predetermined threshold for row processing may be the same as the predetermined threshold for column processing. In some embodiments of the present invention, the predetermined threshold for row processing may be different than the predetermined threshold for column processing. In some embodiments of the present invention, a predetermined threshold value of 10 may be used for the row processing, and a predetermined threshold value of 3 may be used for column processing. In some embodiments of the present invention, the predetermined threshold values may be related to the expected language family of the document.

Pixel-wise horizontal differences may be accumulated 308 and pixel-wise vertical differences may be accumulated 310 from the text-blob image.

In some embodiments of the present invention the accumulated pixel-wise horizontal differences may be determined according to:

Δ H = x , y M TextBlob ( x , y ) - M TextBlob ( x - 1 , y ) ,
where ΔH may denote the accumulated pixel-wise horizontal differences, x may denote the horizontal pixel-location index, y may denote the vertical pixel-location index,

x , y
may denote the summation over valid pixel locations, for example, pixel locations on the left vertical edge of the image may be excluded from the accumulation due to the non-existence of a horizontal left-neighboring pixel, and |•| may denote absolute value.

In some embodiments of the present invention the accumulated pixel-wise vertical differences may be determined according to:

Δ V = x , y M TextBlob ( x , y ) - M TextBlob ( x , y - 1 ) ,
where ΔV may denote the accumulated pixel-wise vertical differences, x may denote the horizontal pixel-location index, y may denote the vertical pixel-location index,

x , y
may denote the summation over valid pixel locations, for example, pixel locations on the top horizontal edge of the image may be excluded from the accumulation due to the non-existence of a vertical top-neighboring pixel, and |•| may denote absolute value.

In some embodiments of the present invention the accumulated pixel-wise horizontal differences may be determined according to:

Δ H = x , y M TextBlob ( x , y ) M TextBlob ( x - 1 , y ) ,
where ⊕ may denote a logical exclusive-OR operation.

In some embodiments of the present invention the accumulated pixel-wise vertical differences may be determined according to:

Δ V = x , y M TextBlob ( x , y ) M TextBlob ( x , y - 1 ) ,
where ⊕ may denote a logical exclusive-OR operation.

The accumulated pixel-wise horizontal differences and the accumulated pixel-wise vertical differences may be compared 312, and the line orientation may be selected 314 based on the comparison of the accumulated pixel-wise horizontal differences and the accumulated pixel-wise vertical differences.

Some embodiments of the present invention may be described in relation to FIG. 4. FIG. 4 illustrates exemplary method(s) 400 for determining the line orientation in a document image. The document image may be an image generated by scanning a document page, a camera-captured image of a document page or another digital image of a document page.

A text mask associated with a document image may be received 402 in a document processing system comprising a processor. The text mask comprises a binary image, wherein one of the binary values (“one” or “zero”) corresponds to pixels that may be considered a portion of a text component in the document image and the other of the binary values corresponds to pixels that may not be considered a portion of a text component in the document image.

The text mask may be down sampled 404. In some embodiments of the present invention, the down sampling may comprise nearest-neighbor interpolation to generate a lower-resolution version of the text mask. In some embodiments of the present invention, the scaling factor used in the down sampling operation may be determined according to:

S = 640 max ( r i , w i ) ,
where S may denote the scaling factor, ri and wi may denote the height and width of the input text mask, respectively, and max(•,•) may denote a maximum operation.

From the down-sampled text mask, text blobs may be generated 406, wherein neighboring text characters in a text line may be merged into one, or more, text blobs, thereby generating a text-blob image.

In some embodiments of the present invention, text-blob generation, also considered text-blob image generation, may comprise morphological processing methods. In some of these embodiments, a dilation operation may be applied to the text mask, and a hole filling operation may be applied to the results of the dilation operation. In alternative embodiments, morphological closing, comprising a dilation operation followed by an erosion operation using the same structuring element, may be used to connect neighboring text characters in a text line. The closing operation may create text blobs that cover words and/or entire text lines, but may produce a relatively small number of cross-line merges.

In still alternative embodiments, a smearing operation may be used to merge text line components. In the smearing operation the distance between successive text pixels along a row, and a column, may be computed. When the distance between successive text pixels is less than a predetermined threshold, all non-text pixels between the successive text pixels are set to the binary value associated with text pixels. In some embodiments of the present invention, the predetermined threshold for row processing may be the same as the predetermined threshold for column processing. In some embodiments of the present invention, the predetermined threshold for row processing may be different than the predetermined threshold for column processing. In some embodiments of the present invention, a predetermined threshold value of 10 may be used for the row processing, and a predetermined threshold value of 3 may be used for column processing. In some embodiments of the present invention, the predetermined threshold values may be related to the expected language family of the document.

Pixel-wise horizontal differences may be accumulated 408 and pixel-wise vertical differences may be accumulated 410 from the text-blob image.

In some embodiments of the present invention the accumulated pixel-wise horizontal differences may be determined according to:

Δ H = x , y M TextBlob ( x , y ) - M TextBlob ( x - 1 , y ) ,
where ΔH may denote the accumulated pixel-wise horizontal differences, x may denote the horizontal pixel-location index, y may denote the vertical pixel-location index,

x , y
may denote the summation over valid pixel locations, for example, pixel locations on the left vertical edge of the image may be excluded from the accumulation due to the non-existence of a horizontal left-neighboring pixel, and |•| may denote absolute value.

In some embodiments of the present invention the accumulated pixel-wise vertical differences may be determined according to:

Δ V = x , y M TextBlob ( x , y ) - M TextBlob ( x , y - 1 ) ,
where ΔV may denote the accumulated pixel-wise vertical differences, x may denote the horizontal pixel-location index, y may denote the vertical pixel-location index,

x , y
may denote the summation over valid pixel locations, for example, pixel locations on the top horizontal edge of the image may be excluded from the accumulation due to the non-existence of a vertical top-neighboring pixel, and |•| may denote absolute value.

In some embodiments of the present invention the accumulated pixel-wise horizontal differences may be determined according to:

Δ H = x , y M TextBlob ( x , y ) M TextBlob ( x - 1 , y ) ,
where ⊕ may denote a logical exclusive-OR operation.

In some embodiments of the present invention the accumulated pixel-wise vertical differences may be determined according to:

Δ V = x , y M TextBlob ( x , y ) M TextBlob ( x , y - 1 ) ,
where ⊕ may denote a logical exclusive-OR operation.

The accumulated pixel-wise horizontal differences and the accumulated pixel-wise vertical differences may be compared 412. When the accumulated pixel-wise horizontal differences is less than the accumulated pixel-wise vertical differences 414, then the text-line orientation of the document image may be classified 416 as horizontal. When the accumulated pixel-wise horizontal differences is not less than the accumulated pixel-wise vertical differences 418, then the text-line orientation of the document image may be classified 420 as vertical.

Some embodiments of the present invention may be described in relation to FIG. 5. FIG. 5 illustrates exemplary method(s) 500 for determining the line orientation in a document image.

A document image may be received 502 in a document processing system comprising a processor. The document image may be an image generated by scanning a document page, a camera-captured image of a document page or another digital image of a document page.

A text mask may be generated 504 from the document image. The text mask comprises a binary image, wherein one of the binary values (“one” or “zero”) corresponds to pixels that may be considered a portion of a text component in the document image and the other of the binary values corresponds to pixels that may not be considered a portion of a text component in the document image.

In some embodiments, if the received document image is not a luminance image, a luminance-component image may be extracted, from the document image, as a first step in text-mask generation. In some embodiments, the luminance image may be generated by a color-space transformation. In alternative embodiments, one of the color components, in the received image, which closely approximates the luminance information may be selected, for example, the green component in a red-green-blue image, for the luminance image.

In some embodiments of the present invention, the text mask may be generated using a block-based adaptive thresholding technique as disclosed in U.S. Pat. No. 8,189,917, issued May 29, 2012, entitled “Methods and Systems for Locating Text in a Digital Image” invented by Richard John Campbell. U.S. Pat. No. 8,189,917 is hereby incorporated herein in its entirety.

In alternative embodiments of the present invention, other text-mask-generation methods known in the art may be used to generate the text mask.

The text mask may be down sampled 506. In some embodiments of the present invention, the down sampling may comprise nearest-neighbor interpolation to generate a lower-resolution version of the text mask. In some embodiments of the present invention, the scaling factor used in the down sampling operation may be determined according to:

S = 640 max ( r i , w i ) ,
where S may denote the scaling factor, ri and wi may denote the height and width of the input text mask, respectively, and max(•,•) may denote a maximum operation.

From the down-sampled text mask, text blobs may be generated 508, wherein neighboring text characters in a text line may be merged into one, or more, text blobs, thereby generating a text-blob image.

In some embodiments of the present invention, text-blob generation, also considered text-blob image generation, may comprise morphological processing methods. In some of these embodiments, a dilation operation may be applied to the text mask, and a hole filling operation may be applied to the results of the dilation operation. In alternative embodiments, morphological closing, comprising a dilation operation followed by an erosion operation using the same structuring element, may be used to connect neighboring text characters in a text line. The closing operation may create text blobs that cover words and/or entire text lines, but may produce a relatively small number of cross-line merges.

In still alternative embodiments, a smearing operation may be used to merge text line components. In the smearing operation the distance between successive text pixels along a row, and a column, may be computed. When the distance between successive text pixels is less than a predetermined threshold, all non-text pixels between the successive text pixels are set to the binary value associated with text pixels. In some embodiments of the present invention, the predetermined threshold for row processing may be the same as the predetermined threshold for column processing. In some embodiments of the present invention, the predetermined threshold for row processing may be different than the predetermined threshold for column processing. In some embodiments of the present invention, a predetermined threshold value of 10 may be used for the row processing, and a predetermined threshold value of 3 may be used for column processing. In some embodiments of the present invention, the predetermined threshold values may be related to the expected language family of the document.

Pixel-wise horizontal differences may be accumulated 510 and pixel-wise vertical differences may be accumulated 512 from the text-blob image.

In some embodiments of the present invention the accumulated pixel-wise horizontal differences may be determined according to:

Δ H = x , y M TextBlob ( x , y ) - M TextBlob ( x - 1 , y ) ,
where ΔH may denote the accumulated pixel-wise horizontal differences, x may denote the horizontal pixel-location index, y may denote the vertical pixel-location index,

x , y
may denote the summation over valid pixel locations, for example, pixel locations on the left vertical edge of the image may be excluded from the accumulation due to the non-existence of a horizontal left-neighboring pixel, and |•| may denote absolute value.

In some embodiments of the present invention the accumulated pixel-wise vertical differences may be determined according to:

Δ V = x , y M TextBlob ( x , y ) - M TextBlob ( x , y - 1 ) ,
where ΔV may denote the accumulated pixel-wise vertical differences, x may denote the horizontal pixel-location index, y may denote the vertical pixel-location index,

x , y
may denote the summation over valid pixel locations, for example, pixel locations on the top horizontal edge of the image may be excluded from the accumulation due to the non-existence of a vertical top-neighboring pixel, and |•| may denote absolute value.

In some embodiments of the present invention the accumulated pixel-wise horizontal differences may be determined according to:

Δ H = x , y M TextBlob ( x , y ) M TextBlob ( x - 1 , y ) ,
where ⊕ may denote a logical exclusive-OR operation.

In some embodiments of the present invention the accumulated pixel-wise vertical differences may be determined according to:

Δ V = x , y M TextBlob ( x , y ) M TextBlob ( x , y - 1 ) ,
where ⊕ may denote a logical exclusive-OR operation.

The accumulated pixel-wise horizontal differences and the accumulated pixel-wise vertical differences may be compared 514, and the line orientation may be selected 516 based on the comparison of the accumulated pixel-wise horizontal differences and the accumulated pixel-wise vertical differences.

Some embodiments of the present invention may be described in relation to FIG. 6. FIG. 6 illustrates exemplary method(s) 600 for determining the line orientation in a document image.

A document image may be received 602 in a document processing system comprising a processor. The document image may be an image generated by scanning a document page, a camera-captured image of a document page or another digital image of a document page.

A text mask may be generated 604 from the document image. The text mask comprises a binary image, wherein one of the binary values (“one” or “zero”) corresponds to pixels that may be considered a portion of a text component and the other of the binary values corresponds to pixels that may not be considered a portion of a text component.

In some embodiments, if the received document image is not a luminance image, a luminance-component image may be extracted, from the document image, as a first step in text-mask generation. In some embodiments, the luminance image may be generated by a color-space transformation. In alternative embodiments, one of the color components, in the received image, which closely approximates the luminance information may be selected, for example, the green component in a red-green-blue image, for the luminance image.

In some embodiments of the present invention, the text mask may be generated using a block-based adaptive thresholding technique as disclosed in U.S. Pat. No. 8,189,917, issued May 29, 2012, entitled “Methods and Systems for Locating Text in a Digital Image” invented by Richard John Campbell.

In alternative embodiments of the present invention, other text-mask-generation methods known in the art may be used to generate the text mask.

The text mask may be down sampled 606. In some embodiments of the present invention, the down sampling may comprise nearest-neighbor interpolation to generate a lower-resolution version of the text mask. In some embodiments of the present invention, the scaling factor used in the down sampling operation may be determined according to:

S = 640 max ( r i , w i ) ,
where S may denote the scaling factor, ri and wi may denote the height and width of the input text mask, respectively, and max(•,•) may denote a maximum operation.

From the down-sampled text mask, text blobs may be generated 608, wherein neighboring text characters in a text line may be merged into one, or more, text blobs, thereby generating a text-blob image.

In some embodiments of the present invention, text-blob generation, also considered text-blob image generation, may comprise morphological processing methods. In some of these embodiments, a dilation operation may be applied to the text mask, and a hole filling operation may be applied to the results of the dilation operation. In alternative embodiments, morphological closing, comprising a dilation operation followed by an erosion operation using the same structuring element, may be used to connect neighboring text characters in a text line. The closing operation may create text blobs that cover words and/or entire text lines, but may produce a relatively small number of cross-line merges.

In still alternative embodiments, a smearing operation may be used to merge text line components. In the smearing operation the distance between successive text pixels along a row, and a column, may be computed. When the distance between successive text pixels is less than a predetermined threshold, all non-text pixels between the successive text pixels are set to the binary value associated with text pixels. In some embodiments of the present invention, the predetermined threshold for row processing may be the same as the predetermined threshold for column processing. In some embodiments of the present invention, the predetermined threshold for row processing may be different than the predetermined threshold for column processing. In some embodiments of the present invention, a predetermined threshold value of 10 may be used for the row processing, and a predetermined threshold value of 3 may be used for column processing. In some embodiments of the present invention, the predetermined threshold values may be related to the expected language family of the document.

Pixel-wise horizontal differences may be accumulated 610 and pixel-wise vertical differences may be accumulated 612 from the text-blob image.

In some embodiments of the present invention the accumulated pixel-wise horizontal differences may be determined according to:

Δ H = x , y M TextBlob ( x , y ) - M TextBlob ( x - 1 , y ) ,
where ΔH may denote the accumulated pixel-wise horizontal differences, x may denote the horizontal pixel-location index, y may denote the vertical pixel-location index,

x , y
may denote the summation over valid pixel locations, for example, pixel locations on the left vertical edge of the image may be excluded from the accumulation due to the non-existence of a horizontal left-neighboring pixel, and |•| may denote absolute value.

In some embodiments of the present invention the accumulated pixel-wise vertical differences may be determined according to:

Δ V = x , y M TextBlob ( x , y ) - M TextBlob ( x , y - 1 ) ,
where ΔV may denote the accumulated pixel-wise vertical differences, x may denote the horizontal pixel-location index, y may denote the vertical pixel-location index,

x , y
may denote the summation over valid pixel locations, for example, pixel locations on the top horizontal edge of the image may be excluded from the accumulation due to the non-existence of a vertical top-neighboring pixel, and |•| may denote absolute value.

In some embodiments of the present invention the accumulated pixel-wise horizontal differences may be determined according to:

Δ H = x , y M TextBlob ( x , y ) M TextBlob ( x - 1 , y ) ,
where ⊕ may denote a logical exclusive-OR operation.

In some embodiments of the present invention the accumulated pixel-wise vertical differences may be determined according to:

Δ V = x , y M TextBlob ( x , y ) M TextBlob ( x , y - 1 ) ,
where ⊕ may denote a logical exclusive-OR operation.

The accumulated pixel-wise horizontal differences and the accumulated pixel-wise vertical differences may be compared 614. When the accumulated pixel-wise horizontal differences is less than the accumulated pixel-wise vertical differences 616, then the text-line orientation of the document image may be classified 618 as horizontal. When the accumulated pixel-wise horizontal differences is not less than the accumulated pixel-wise vertical differences 620, then the text-line orientation of the document image may be classified 622 as vertical.

Some embodiments of the present invention may be described in relation to FIG. 7. FIG. 7 illustrates exemplary systems(s) 700 for determining the line orientation in a document image. An input image 701 may be received in the orientation-determination system 700 by an image receiver 702, or an input text mask 703 may be received in the orientation-determination system 700 by a text-mask receiver 704. The input image may be an image generated by scanning a document page, a camera-captured image of a document page or another digital image of a document page.

When an input image 701 is received in the image receiver 702, the image receiver 702 may make the received image 705 available to a text-mask generator 706. The text-mask generator 706 may extract a text mask from the received image 705.

The generated text mask 707 comprises a binary image, wherein one of the binary values (“one” or “zero”) corresponds to pixels that may be considered a portion of a text component in the received image 705 and the other of the binary values corresponds to pixels that may not be considered a portion of a text component in the received image 705.

In some embodiments, if the received image 705 is not a luminance image, a luminance-component image may be extracted, from the received image 705, as a first step in text-mask generation. In some embodiments, the luminance image may be generated by a color-space transformation. In alternative embodiments, one of the color components, in the received image 705, which closely approximates the luminance information may be selected, for example, the green component in a red-green-blue image, for the luminance image.

FIG. 8A depicts an exemplary input image 802, and FIG. 8B depicts a luminance image 804 generated from the input image 802.

In some embodiments of the present invention, the text mask may be generated using a block-based adaptive thresholding technique as disclosed in U.S. Pat. No. 8,189,917, issued May 29, 2012, entitled “Methods and Systems for Locating Text in a Digital Image” invented by Richard John Campbell.

In alternative embodiments of the present invention, other text-mask-generation methods known in the art may be used to generate the text mask.

The generated text mask 707 may be received by the text-mask receiver 704.

When an input text-mask 703 is received by the text-mask receiver 704 or the generated text mask 707 is received by the text-mask receiver 704, the text-mask receiver 704 may make the text-mask 709 (either input or generated, as received) available to a text-mask down-sampler 710.

In some embodiments of the present invention, the text-mask down-sampler 710 may perform nearest-neighbor interpolation to generate a lower-resolution version 711 of the text mask 709. In some embodiments of the present invention, the scaling factor used in the down sampling operation may be determined according to:

S = 640 max ( r i , w i ) ,
where S may denote the scaling factor, ri and wi may denote the height and width of the input text mask, respectively, and max(•,•) may denote a maximum operation. A down-sampled text mask 711 may be made available to a text-blob generator 712.

FIG. 8C depicts a text mask 806 generated from the luminance image 804, and FIG. 8D depicts a down-sampled version 808 of the text mask 806.

From the down-sampled text mask 711, text blobs 713 may be generated, wherein neighboring text characters in a text line may be merged into one, or more, text blobs.

In some embodiments of the present invention, text-blob generation, also considered text-blob image generation, may comprise morphological processing methods. In some of these embodiments, a dilation operation may be applied to the text mask, and a hole filling operation may be applied to the results of the dilation operation.

FIG. 8E depicts a text-blob image 810 generated by applying a dilation operation to the down-sampled text mask 808 followed by a hole filling operation applied to the results of the dilation operation.

In alternative embodiments, morphological closing, comprising a dilation operation followed by an erosion operation using the same structuring element, may be used to connect neighboring text characters in a text line. The closing operation may create text blobs that cover words and/or entire text lines, but may produce a relatively small number of cross-line merges.

FIG. 8F depicts a text-blob image 812 generated by applying morphological closing to the down-sampled text mask 808.

In still alternative embodiments, a smearing operation may be used to merge text line components. In the smearing operation the distance between successive text pixels along a row, and a column, may be computed. When the distance between successive text pixels is less than a predetermined threshold, all non-text pixels between the successive text pixels are set to the binary value associated with text pixels. In some embodiments of the present invention, the predetermined threshold for row processing may be the same as the predetermined threshold for column processing. In some embodiments of the present invention, the predetermined threshold for row processing may be different than the predetermined threshold for column processing. In some embodiments of the present invention, a predetermined threshold value of 10 may be used for the row processing, and a predetermined threshold value of 3 may be used for column processing. In some embodiments of the present invention, the predetermined threshold values may be related to the expected language family of the document.

FIG. 8G depicts a text-blob image 814 generated by applying a smearing operation to the down-sampled text mask 808.

The text blobs 713 may be made available to an accumulator 714, and pixel-wise horizontal differences may be accumulated and pixel-wise vertical differences may be accumulated from the text-blob image 713.

In some embodiments of the present invention the accumulated pixel-wise horizontal differences may be determined according to:

Δ H = x , y M TextBlob ( x , y ) - M TextBlob ( x - 1 , y ) ,
where ΔH may denote the accumulated pixel-wise horizontal differences, x may denote the horizontal pixel-location index, y may denote the vertical pixel-location index,

x , y
may denote the summation over valid pixel locations, for example, pixel locations on the left vertical edge of the image may be excluded from the accumulation due to the non-existence of a horizontal left-neighboring pixel, and |•| may denote absolute value.

In some embodiments of the present invention the accumulated pixel-wise vertical differences may be determined according to:

Δ V = x , y M TextBlob ( x , y ) - M TextBlob ( x , y - 1 ) ,
where ΔV may denote the accumulated pixel-wise vertical differences, x may denote the horizontal pixel-location index, y may denote the vertical pixel-location index,

x , y
may denote the summation over valid pixel locations, for example, pixel locations on the top horizontal edge of the image may be excluded from the accumulation due to the non-existence of a vertical top-neighboring pixel, and |•| may denote absolute value.

In some embodiments of the present invention the accumulated pixel-wise horizontal differences may be determined according to:

Δ H = x , y M TextBlob ( x , y ) M TextBlob ( x - 1 , y ) ,
where ⊕ may denote a logical exclusive-OR operation.

In some embodiments of the present invention the accumulated pixel-wise vertical differences may be determined according to:

Δ V = x , y M TextBlob ( x , y ) M TextBlob ( x , y - 1 ) ,
where ⊕ may denote a logical exclusive-OR operation.

The accumulated pixel-wise horizontal differences, horizontal and vertical, 715 may be made available to a comparator 716, wherein the accumulated pixel-wise horizontal differences and the accumulated pixel-wise vertical differences may be compared, and the comparator results 717 may be made available to an orientation selector 718. When the accumulated pixel-wise horizontal differences is less than the accumulated pixel-wise vertical differences, then the text-line orientation 719 of the document image may be classified as horizontal by the orientation selector 718. When the accumulated pixel-wise horizontal differences is not less than the accumulated pixel-wise vertical difference, then the text-line orientation 719 of the document image may be classified as vertical by the orientation selector 718.

The text-line orientation 719 may be made available by the orientation-determination system 700 to other document-processing applications in a document workflow.

In some embodiments of the present invention described in relation to FIG. 9, a text-line orientation estimate may be used in conjunction with document language information to identify a page orientation. FIG. 9 illustrates exemplary method(s) 900 for determining the orientation of a document image. A text-line orientation estimate may be received 902 and document language information may be received 904. The document language information may comprise an indicator of the text-line orientation for the written word in a language in a language family, for example, horizontal text-line orientation for left-to-right written languages, for example, English, and vertical text-line orientation for top-to-bottom written languages, for example, Japanese. The received text-line estimate may be reconciled 906 with the received document language information to determine the page orientation.

Some embodiments of the present invention may comprise a computer program product comprising a non-transitory computer-readable storage medium having instructions stored thereon/in which may be used to program a computing system to perform any of the features and methods described herein. Exemplary computer-readable storage media may include, but are not limited to, flash memory devices, disk storage media, for example, floppy disks, optical disks, magneto-optical disks, Digital Versatile Discs (DVDs), Compact Discs (CDs), micro-drives and other disk storage media, Read-Only Memory (ROMs), Programmable Read-Only Memory (PROMs), Erasable Programmable Read-Only Memory (EPROMS), Electrically Erasable Programmable Read-Only Memory (EEPROMs), Random-Access Memory (RAMS), Video Random-Access Memory (VRAMs), Dynamic Random-Access Memory (DRAMs) and any type of media or device suitable for storing instructions and/or data.

The terms and expressions which have been employed in the foregoing specification are used therein as terms of description and not of limitation, and there is no intention in the use of such terms and expressions of excluding equivalence of the features shown and described or portions thereof, it being recognized that the scope of the invention is defined and limited only by the claims which follow.

Claims (20)

What is claimed is:
1. A method for determining a line orientation in a document image, said method comprising:
receiving, in a processor, a text mask associated with a document image;
generating a text-blob image from said text mask;
accumulating pixel-wise horizontal differences of horizontally adjacent pixel intensity values in said text-blob image;
accumulating pixel-wise vertical differences of vertically adjacent pixel intensity values in said text-blob image; and
selecting a line orientation based on a comparison of said accumulated pixel-wise horizontal differences and said accumulated pixel-wise vertical differences.
2. A method as described in claim 1, wherein said text mask is a lower resolution than said document image.
3. A method as described in claim 1 further comprising:
receiving said document image; and
generating said text mask from said document image.
4. A method as described in claim 3, wherein said generating said text mask comprises:
extracting a luminance image from said document image; and
generating said text mask from said luminance image.
5. A method as described in claim 3, wherein said generating said text mask comprises:
extracting a luminance image from said document image;
generating a first text mask from said luminance image; and
down-sampling said first text mask to form said text mask.
6. A method as described in claim 1, wherein:
prior to said generating a text-blob image, said text mask is down-sampled; and
said text-blob image is generated from said down-sampled text mask.
7. A method as described in claim 1, wherein said line orientation is horizontal when said accumulated pixel-wise horizontal differences is less than said accumulated pixel-wise vertical differences.
8. A method as described in claim 7, wherein said line orientation is vertical when said accumulated pixel-wise horizontal differences is not less than said accumulated pixel-wise vertical differences.
9. A method as described in claim 1, wherein said generating a text-blob image comprises an operation selected from the group consisting of a smearing operation, a morphological closing operation and a morphological operation comprising a dilation operation followed by a hole filling operation.
10. A method as described in claim 1 further comprising:
receiving document language information; and
determining a page orientation by reconciling said document language information and said line orientation.
11. A non-transitory computer-readable medium encoded with a computer program code for implementing a method for determining a line orientation in a document image, said method comprising:
receiving, in a processor, a text mask associated with a document image;
generating a text-blob image from said text mask;
accumulating pixel-wise horizontal differences of horizontally adjacent pixel intensity values in said text-blob image;
accumulating pixel-wise vertical differences of vertically adjacent pixel intensity values in said text-blob image; and
selecting a line orientation based on a comparison of said accumulated pixel-wise horizontal differences and said accumulated pixel-wise vertical differences.
12. A computer-readable medium as described in claim 11, said method further comprising:
receiving said document image; and
generating said text mask from said document image.
13. A computer-readable medium as described in claim 12, wherein, in said method, said generating said text mask comprises:
extracting a luminance image from said document image; and
generating said text mask from said luminance image.
14. A computer-readable medium as described in claim 12, wherein said generating said text mask comprises:
extracting a luminance image from said document image;
generating a first text mask from said luminance image; and
down-sampling said first text mask to form said text mask.
15. A computer-readable medium as described in claim 11, wherein, in said method:
prior to said generating a text-blob image, said text mask is down-sampled; and
said text-blob image is generated from said down-sampled text mask.
16. A computer-readable medium as described in claim 11, wherein, in said method, said line orientation is horizontal when said accumulated pixel-wise horizontal differences is less than said accumulated pixel-wise vertical differences.
17. A computer-readable medium as described in claim 16, wherein, in said method, said line orientation is vertical when said accumulated pixel-wise horizontal differences is not less than said accumulated pixel-wise vertical differences.
18. A computer-readable medium as described in claim 11, wherein, in said method, said generating a text-blob image comprises an operation selected from the group consisting of a smearing operation, a morphological closing operation and a morphological operation comprising a dilation operation followed by a hole filling operation.
19. A system for determining a line orientation in a document image, said system comprising, in a processor:
a text-mask receiver for receiving a text mask associated with a document image;
a text-blob generator for generating a text-blob image from said text mask;
an accumulator for accumulating pixel-wise horizontal differences of horizontally adjacent pixel intensity values in said text-blob image and for accumulating pixel-wise vertical differences of vertically adjacent pixel intensity values in said text-blob image;
a comparator for comparing said accumulated pixel-wise horizontal differences and said accumulated pixel-wise vertical differences; and
an orientation selector for selecting a line orientation based on said comparison of said accumulated pixel-wise horizontal differences and said accumulated pixel-wise vertical differences.
20. A method for determining a page orientation in a document image, said method comprising:
receiving, in a processor, a text mask associated with a document image;
generating a text-blob image from said text mask;
accumulating pixel-wise horizontal differences of horizontally adjacent pixel intensity values in said text-blob image;
accumulating pixel-wise vertical differences of vertically adjacent pixel intensity values in said text-blob image;
selecting a line orientation based on a comparison of said accumulated pixel-wise horizontal differences and said accumulated pixel-wise vertical differences;
receiving a document language information; and
selecting a page orientation based on said line orientation and said document language information.
US13753423 2013-01-29 2013-01-29 Methods, systems and apparatus for determining orientation in a document image Active 2033-10-15 US9076058B2 (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
US13753423 US9076058B2 (en) 2013-01-29 2013-01-29 Methods, systems and apparatus for determining orientation in a document image

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
US13753423 US9076058B2 (en) 2013-01-29 2013-01-29 Methods, systems and apparatus for determining orientation in a document image
JP2014012833A JP5731027B2 (en) 2013-01-29 2014-01-27 Direction determination apparatus, the direction determination method, and computer-readable recording medium

Publications (2)

Publication Number Publication Date
US20140212047A1 true US20140212047A1 (en) 2014-07-31
US9076058B2 true US9076058B2 (en) 2015-07-07

Family

ID=51223020

Family Applications (1)

Application Number Title Priority Date Filing Date
US13753423 Active 2033-10-15 US9076058B2 (en) 2013-01-29 2013-01-29 Methods, systems and apparatus for determining orientation in a document image

Country Status (2)

Country Link
US (1) US9076058B2 (en)
JP (1) JP5731027B2 (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20140325351A1 (en) * 2013-04-24 2014-10-30 Kabushiki Kaisha Toshiba Electronic device and handwritten data processing method

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
GB2514557A (en) * 2013-05-28 2014-12-03 Snell Ltd Image processing

Citations (58)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US3831146A (en) 1973-03-19 1974-08-20 Ibm Optimum scan angle determining means
US5020117A (en) 1988-01-18 1991-05-28 Kabushiki Kaisha Toshiba Handwritten character string recognition system
US5031225A (en) 1987-12-09 1991-07-09 Ricoh Company, Ltd. Character recognition method for recognizing character in an arbitrary rotation position
US5060276A (en) 1989-05-31 1991-10-22 At&T Bell Laboratories Technique for object orientation detection using a feed-forward neural network
US5077811A (en) 1990-10-10 1991-12-31 Fuji Xerox Co., Ltd. Character and picture image data processing system
US5191438A (en) 1989-12-12 1993-03-02 Sharp Kabushiki Kaisha Facsimile device with skew correction and text line direction detection
US5235651A (en) 1991-08-06 1993-08-10 Caere Corporation Rotation of images for optical character recognition
US5251268A (en) 1991-08-09 1993-10-05 Electric Power Research Institute, Inc. Integrated method and apparatus for character and symbol recognition
US5276742A (en) 1991-11-19 1994-01-04 Xerox Corporation Rapid detection of page orientation
US5319722A (en) 1992-10-01 1994-06-07 Sony Electronics, Inc. Neural network for character recognition of rotated characters
US5471549A (en) 1990-11-28 1995-11-28 Hitachi, Ltd. Method of detecting and correcting a direction of image data and document image filing system employing the same
US5508810A (en) 1991-10-17 1996-04-16 Ricoh Company, Ltd. Image recorder for properly orienting output images
US5664027A (en) 1993-04-20 1997-09-02 Lucent Technologies Inc. Methods and apparatus for inferring orientation of lines of text
US5835632A (en) 1995-03-08 1998-11-10 Canon Kabushiki Kaisha Image processing method and an image processing apparatus
US5854853A (en) * 1993-12-22 1998-12-29 Canon Kabushika Kaisha Method and apparatus for selecting blocks of image data from image data having both horizontally- and vertically-oriented blocks
US5889884A (en) 1995-05-23 1999-03-30 Minolta Co., Ltd. Image forming apparatus capable of recognizing top and bottom of document image
US5930001A (en) 1994-10-25 1999-07-27 Canon Kabushiki Kaisha Copier which rotates an image based on the recognition of an extracted character from various directions
US5987171A (en) 1994-11-10 1999-11-16 Canon Kabushiki Kaisha Page analysis system
US6011877A (en) 1993-08-26 2000-01-04 Minolta Co., Ltd. Apparatus and method for determining the directional orientation of a document image based upon the location of detected punctuation marks within the document image
JP2000083141A (en) 1998-09-03 2000-03-21 Minolta Co Ltd Image recognition device
US6101270A (en) 1992-08-31 2000-08-08 International Business Machines Corporation Neural network architecture for recognition of upright and rotated characters
US6104832A (en) 1992-09-11 2000-08-15 Canon Kabushiki Kaisha Image processing method and apparatus
US6137905A (en) 1995-08-31 2000-10-24 Canon Kabushiki Kaisha System for discriminating document orientation
US6151423A (en) 1998-03-04 2000-11-21 Canon Kabushiki Kaisha Character recognition with document orientation determination
US6169822B1 (en) 1997-07-15 2001-01-02 Samsung Electronics Co., Ltd. Method for correcting direction of document image
US6173088B1 (en) 1996-10-01 2001-01-09 Canon Kabushiki Kaisha Image forming method and apparatus
US20010013938A1 (en) 1996-09-19 2001-08-16 Akihiro Usami Image forming apparatus and method
US20010028737A1 (en) 2000-02-09 2001-10-11 Hiroyuki Takakura Image processing method and image processing device
US6304681B1 (en) 1996-09-26 2001-10-16 Canon Kabushiki Kaisha Image processing apparatus for executing image processing in correspondence with portrait and landscape types
US6320983B1 (en) 1998-03-27 2001-11-20 Fujitsu Limited Method and apparatus for character recognition, and computer-readable recording medium with a program making a computer execute the method recorded therein
US6360028B1 (en) 1996-12-27 2002-03-19 Canon Kabushiki Kaisha Image processing apparatus and method
JP2002142101A (en) 2000-11-06 2002-05-17 Sharp Corp Image processing device
US20020126904A1 (en) 1999-11-30 2002-09-12 Hiroshi Kakutani Character-recognition pre-processing apparatus and method, and program recording medium
US6501864B1 (en) 1995-07-31 2002-12-31 Fujitsu Limited Data medium handling apparatus and data medium handling method
US20030049062A1 (en) 2001-09-10 2003-03-13 Toshiba Tec Kabushiki Kaisha Image forming apparatus and method having document orientation control
US20030086721A1 (en) 2001-11-07 2003-05-08 Guillemin Gustavo M. Methods and apparatus to determine page orientation for post imaging finishing
US6574375B1 (en) 1992-04-06 2003-06-03 Ricoh Company, Ltd. Method for detecting inverted text images on a digital scanning device
US20030152289A1 (en) 2002-02-13 2003-08-14 Eastman Kodak Company Method and system for determining image orientation
US6624905B1 (en) 1998-09-29 2003-09-23 Canon Kabushiki Kaisha Image formation apparatus having image direction discrimination function
US20030210437A1 (en) 2002-05-08 2003-11-13 Toshiba Tec Kabushiki Kaisha Image forming apparatus
US20040001606A1 (en) 2002-06-28 2004-01-01 Levy Kenneth L. Watermark fonts
US20040179733A1 (en) 2003-03-11 2004-09-16 Pfu Limited Image reading apparatus
US6798905B1 (en) 1998-07-10 2004-09-28 Minolta Co., Ltd. Document orientation recognizing device which recognizes orientation of document image
US6804414B1 (en) 1998-05-01 2004-10-12 Fujitsu Limited Image status detecting apparatus and document image correcting apparatus
US20040218836A1 (en) 2003-04-30 2004-11-04 Canon Kabushiki Kaisha Information processing apparatus, method, storage medium and program
US20050041865A1 (en) 2002-04-03 2005-02-24 Li Xin Zhen Orientation determination for handwritten characters for recognition thereof
US20050163399A1 (en) 2004-01-26 2005-07-28 Aradhye Hrishikesh B. Method and apparatus for determination of text orientation
US20060018544A1 (en) 2004-07-20 2006-01-26 Yoshihisa Ohguro Method and apparatus for detecting an orientation of characters in a document image
US6993205B1 (en) 2000-04-12 2006-01-31 International Business Machines Corporation Automatic method of detection of incorrectly oriented text blocks using results from character recognition
US20060033967A1 (en) 2004-08-10 2006-02-16 Oce-Technologies B.V. Outlier detection during scanning
US7031553B2 (en) 2000-09-22 2006-04-18 Sri International Method and apparatus for recognizing text in an image sequence of scene imagery
US20060210195A1 (en) 2005-03-17 2006-09-21 Yoshihisa Ohguro Detecting an orientation of characters in a document image
US20060215230A1 (en) 2005-03-24 2006-09-28 Borrey Roland G Systems and methods of accessing random access cache for rescanning
US7151860B1 (en) 1999-07-30 2006-12-19 Fujitsu Limited Document image correcting device and a correcting method
JP2008011484A (en) 2006-06-02 2008-01-17 Nippon Telegr & Teleph Corp <Ntt> Apparatus and method for extracting character and graphic string, program for executing the method, recording medium with the program stored therein
JP2008305099A (en) 2007-06-06 2008-12-18 Sharp Corp Image processing method, image processor, image forming device, computer program and recording medium
US20080317341A1 (en) * 2007-06-21 2008-12-25 Speigle Jon M Methods and Systems for Identifying Text Orientation in a Digital Image
JP2009105541A (en) 2007-10-22 2009-05-14 Seiko Epson Corp Image processing apparatus, method and program

Family Cites Families (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP3544324B2 (en) * 1999-09-08 2004-07-21 日本電信電話株式会社 Character string information extraction apparatus and method and recording medium storing its process

Patent Citations (64)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US3831146A (en) 1973-03-19 1974-08-20 Ibm Optimum scan angle determining means
US5031225A (en) 1987-12-09 1991-07-09 Ricoh Company, Ltd. Character recognition method for recognizing character in an arbitrary rotation position
US5020117A (en) 1988-01-18 1991-05-28 Kabushiki Kaisha Toshiba Handwritten character string recognition system
US5060276A (en) 1989-05-31 1991-10-22 At&T Bell Laboratories Technique for object orientation detection using a feed-forward neural network
US5191438A (en) 1989-12-12 1993-03-02 Sharp Kabushiki Kaisha Facsimile device with skew correction and text line direction detection
US5077811A (en) 1990-10-10 1991-12-31 Fuji Xerox Co., Ltd. Character and picture image data processing system
US5471549A (en) 1990-11-28 1995-11-28 Hitachi, Ltd. Method of detecting and correcting a direction of image data and document image filing system employing the same
US5235651A (en) 1991-08-06 1993-08-10 Caere Corporation Rotation of images for optical character recognition
US5251268A (en) 1991-08-09 1993-10-05 Electric Power Research Institute, Inc. Integrated method and apparatus for character and symbol recognition
US5508810A (en) 1991-10-17 1996-04-16 Ricoh Company, Ltd. Image recorder for properly orienting output images
US5276742A (en) 1991-11-19 1994-01-04 Xerox Corporation Rapid detection of page orientation
US6574375B1 (en) 1992-04-06 2003-06-03 Ricoh Company, Ltd. Method for detecting inverted text images on a digital scanning device
US6101270A (en) 1992-08-31 2000-08-08 International Business Machines Corporation Neural network architecture for recognition of upright and rotated characters
US6104832A (en) 1992-09-11 2000-08-15 Canon Kabushiki Kaisha Image processing method and apparatus
US5319722A (en) 1992-10-01 1994-06-07 Sony Electronics, Inc. Neural network for character recognition of rotated characters
US5664027A (en) 1993-04-20 1997-09-02 Lucent Technologies Inc. Methods and apparatus for inferring orientation of lines of text
US6011877A (en) 1993-08-26 2000-01-04 Minolta Co., Ltd. Apparatus and method for determining the directional orientation of a document image based upon the location of detected punctuation marks within the document image
US5854853A (en) * 1993-12-22 1998-12-29 Canon Kabushika Kaisha Method and apparatus for selecting blocks of image data from image data having both horizontally- and vertically-oriented blocks
US5930001A (en) 1994-10-25 1999-07-27 Canon Kabushiki Kaisha Copier which rotates an image based on the recognition of an extracted character from various directions
US5987171A (en) 1994-11-10 1999-11-16 Canon Kabushiki Kaisha Page analysis system
US5835632A (en) 1995-03-08 1998-11-10 Canon Kabushiki Kaisha Image processing method and an image processing apparatus
US5889884A (en) 1995-05-23 1999-03-30 Minolta Co., Ltd. Image forming apparatus capable of recognizing top and bottom of document image
US6266441B1 (en) 1995-05-23 2001-07-24 Minolta Co., Ltd. Image forming apparatus capable of recognizing top and bottom of document image
US6501864B1 (en) 1995-07-31 2002-12-31 Fujitsu Limited Data medium handling apparatus and data medium handling method
US6137905A (en) 1995-08-31 2000-10-24 Canon Kabushiki Kaisha System for discriminating document orientation
US20010013938A1 (en) 1996-09-19 2001-08-16 Akihiro Usami Image forming apparatus and method
US6473196B2 (en) 1996-09-19 2002-10-29 Canon Kabushiki Kaisha Image forming apparatus and method
US6304681B1 (en) 1996-09-26 2001-10-16 Canon Kabushiki Kaisha Image processing apparatus for executing image processing in correspondence with portrait and landscape types
US6411743B1 (en) 1996-10-01 2002-06-25 Canon Kabushiki Kaisha Image forming method and apparatus
US6173088B1 (en) 1996-10-01 2001-01-09 Canon Kabushiki Kaisha Image forming method and apparatus
US6360028B1 (en) 1996-12-27 2002-03-19 Canon Kabushiki Kaisha Image processing apparatus and method
US6169822B1 (en) 1997-07-15 2001-01-02 Samsung Electronics Co., Ltd. Method for correcting direction of document image
US6151423A (en) 1998-03-04 2000-11-21 Canon Kabushiki Kaisha Character recognition with document orientation determination
US6320983B1 (en) 1998-03-27 2001-11-20 Fujitsu Limited Method and apparatus for character recognition, and computer-readable recording medium with a program making a computer execute the method recorded therein
US6804414B1 (en) 1998-05-01 2004-10-12 Fujitsu Limited Image status detecting apparatus and document image correcting apparatus
US6798905B1 (en) 1998-07-10 2004-09-28 Minolta Co., Ltd. Document orientation recognizing device which recognizes orientation of document image
JP2000083141A (en) 1998-09-03 2000-03-21 Minolta Co Ltd Image recognition device
US6624905B1 (en) 1998-09-29 2003-09-23 Canon Kabushiki Kaisha Image formation apparatus having image direction discrimination function
US7151860B1 (en) 1999-07-30 2006-12-19 Fujitsu Limited Document image correcting device and a correcting method
US6941030B2 (en) 1999-11-30 2005-09-06 Pfu Limited Character-recognition pre-processing apparatus and method, and program recording medium
US20020126904A1 (en) 1999-11-30 2002-09-12 Hiroshi Kakutani Character-recognition pre-processing apparatus and method, and program recording medium
US20010028737A1 (en) 2000-02-09 2001-10-11 Hiroyuki Takakura Image processing method and image processing device
US6993205B1 (en) 2000-04-12 2006-01-31 International Business Machines Corporation Automatic method of detection of incorrectly oriented text blocks using results from character recognition
US7031553B2 (en) 2000-09-22 2006-04-18 Sri International Method and apparatus for recognizing text in an image sequence of scene imagery
JP2002142101A (en) 2000-11-06 2002-05-17 Sharp Corp Image processing device
US20030049062A1 (en) 2001-09-10 2003-03-13 Toshiba Tec Kabushiki Kaisha Image forming apparatus and method having document orientation control
US6546215B2 (en) 2001-09-10 2003-04-08 Toshiba Tec Kabushiki Kaisha Image forming apparatus and method having document orientation control
US6567628B1 (en) 2001-11-07 2003-05-20 Hewlett-Packard Development Company L.P. Methods and apparatus to determine page orientation for post imaging finishing
US20030086721A1 (en) 2001-11-07 2003-05-08 Guillemin Gustavo M. Methods and apparatus to determine page orientation for post imaging finishing
US20030152289A1 (en) 2002-02-13 2003-08-14 Eastman Kodak Company Method and system for determining image orientation
US20050041865A1 (en) 2002-04-03 2005-02-24 Li Xin Zhen Orientation determination for handwritten characters for recognition thereof
US20030210437A1 (en) 2002-05-08 2003-11-13 Toshiba Tec Kabushiki Kaisha Image forming apparatus
US20040001606A1 (en) 2002-06-28 2004-01-01 Levy Kenneth L. Watermark fonts
US20040179733A1 (en) 2003-03-11 2004-09-16 Pfu Limited Image reading apparatus
US20040218836A1 (en) 2003-04-30 2004-11-04 Canon Kabushiki Kaisha Information processing apparatus, method, storage medium and program
US20050163399A1 (en) 2004-01-26 2005-07-28 Aradhye Hrishikesh B. Method and apparatus for determination of text orientation
US20060018544A1 (en) 2004-07-20 2006-01-26 Yoshihisa Ohguro Method and apparatus for detecting an orientation of characters in a document image
US20060033967A1 (en) 2004-08-10 2006-02-16 Oce-Technologies B.V. Outlier detection during scanning
US20060210195A1 (en) 2005-03-17 2006-09-21 Yoshihisa Ohguro Detecting an orientation of characters in a document image
US20060215230A1 (en) 2005-03-24 2006-09-28 Borrey Roland G Systems and methods of accessing random access cache for rescanning
JP2008011484A (en) 2006-06-02 2008-01-17 Nippon Telegr & Teleph Corp <Ntt> Apparatus and method for extracting character and graphic string, program for executing the method, recording medium with the program stored therein
JP2008305099A (en) 2007-06-06 2008-12-18 Sharp Corp Image processing method, image processor, image forming device, computer program and recording medium
US20080317341A1 (en) * 2007-06-21 2008-12-25 Speigle Jon M Methods and Systems for Identifying Text Orientation in a Digital Image
JP2009105541A (en) 2007-10-22 2009-05-14 Seiko Epson Corp Image processing apparatus, method and program

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
Chen, A knowledge-based approach for Textual Information Extraction from Mixed Text/Graphics Complex Document Images, IEEE 2010. *

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20140325351A1 (en) * 2013-04-24 2014-10-30 Kabushiki Kaisha Toshiba Electronic device and handwritten data processing method
US9378427B2 (en) * 2013-04-24 2016-06-28 Kabushiki Kaisha Toshiba Displaying handwritten strokes on a device according to a determined stroke direction matching the present direction of inclination of the device

Also Published As

Publication number Publication date Type
JP5731027B2 (en) 2015-06-10 grant
JP2014147069A (en) 2014-08-14 application
US20140212047A1 (en) 2014-07-31 application

Similar Documents

Publication Publication Date Title
Moghaddam et al. A multi-scale framework for adaptive binarization of degraded document images
US6393150B1 (en) Region-based image binarization system
US6064767A (en) Automatic language identification by stroke geometry analysis
US6473523B1 (en) Portable text capturing method and device therefor
US5539841A (en) Method for comparing image sections to determine similarity therebetween
US20050201619A1 (en) Video text processing apparatus
US20030043172A1 (en) Extraction of textual and graphic overlays from video
Chaudhuri et al. Robust detection of skew in document images
Kim et al. A fast and robust moving object segmentation in video sequences
US20080226171A1 (en) Correcting device and method for perspective transformed document images
US8437551B2 (en) Devices and methods for restoring low-resolution text images
US20060062454A1 (en) Segmenting pixels in an image based on orientation-dependent adaptive thresholds
CN101593276A (en) Video OCR image-text separation method and system
US20130071041A1 (en) High-Quality Denoising of an Image Sequence
Kapoor et al. A new algorithm for skew detection and correction
Hedjam et al. A spatially adaptive statistical method for the binarization of historical manuscripts and degraded document images
CN101599124A (en) Method and device for segmenting characters from video image
US20140023278A1 (en) Feature Extraction And Use With A Probability Density Function (PDF) Divergence Metric
US20140140626A1 (en) Edge Direction and Curve Based Image De-Blurring
US8542926B2 (en) Script-agnostic text reflow for document images
US20120294528A1 (en) Method of Detecting and Correcting Digital Images of Books in the Book Spine Area
US20050196071A1 (en) Black white image scaling for optical character recognition
US20110123114A1 (en) Character recognition device and method and computer-readable medium controlling the same
US20140029073A1 (en) Repair or editing of damaged (torn) scanned halftone prints
US20130050765A1 (en) Method and apparatus for document authentication using image comparison on a block-by-block basis

Legal Events

Date Code Title Description
AS Assignment

Owner name: SHARP LABORATORIES OF AMERICA, INC., WASHINGTON

Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:FERMAN, AHMET MUFIT;REEL/FRAME:029716/0927

Effective date: 20130129

AS Assignment

Owner name: SHARP KABUSHIKI KAISHA, JAPAN

Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:SHARP LABORATORIES OF AMERICA, INC.;REEL/FRAME:037648/0725

Effective date: 20160202