JP7106672B2 - Image processing device, control method and control program - Google Patents

Description

The present disclosure relates to an image processing apparatus, control method, and control program, and more particularly to an image processing apparatus, control method, and control program for processing an original image including high-brightness characters on a low-brightness background.

In a company where the person in charge manually converts forms such as invoices into data, the work burden on the person in charge becomes large when it is necessary to convert a huge number of forms into data. There is a growing demand for In order to improve the efficiency of the form data conversion work, it is necessary for the computer to correctly recognize the characters written on the form. In such a form, high-brightness characters such as white may be placed on a low-brightness background such as black. In general, OCR (Optical Character Recognition) software recognizes characters as if the luminance of the characters is lower than the luminance of the background. Therefore, in an image obtained by reading such a form, it is desirable to reverse the relationship between the gradation values of the low-brightness background area and the gradation values of the high-brightness character area before recognizing the characters.

A character recognition apparatus is disclosed that detects a white character portion of an image, reverses the detected white character portion, and performs character recognition processing on the reversed image portion (Patent Document 1).

JP-A-9-269970

In an image of a form in which high-brightness characters are arranged on a low-brightness background, in order to reverse the relationship between the gradation value of the low-brightness background area and the gradation value of the high-brightness character area, the low-brightness You need to correctly identify the background area. In particular, when such a form includes ruled lines, the components of the ruled lines must be removed satisfactorily in order to correctly identify the low-brightness background area in the captured image of the form.

An object of an image processing apparatus, a control method, and a control program is to make it possible to satisfactorily detect characters from an image of a document including a low-luminance background, high-luminance characters on the low-luminance background, and ruled lines. .

An image processing apparatus according to an aspect of an embodiment includes an acquisition unit that acquires a multi-valued image of a document including a low-brightness background, high-brightness characters on the low-brightness background, and a plurality of ruled lines; , a ruled-line-removed image generation unit that generates a ruled-line-removed image by removing a plurality of ruled-line components from the binary image, and black pixels in the ruled-line-removed image. and the relationship between the sizes of the plurality of connected regions detected by the region detection unit, the relationship between the positions of the plurality of connected regions, or the distance between the plurality of connected regions and a combining unit that combines a plurality of connected regions based on at least one of: It has a reversed image generation unit that generates a reversed image with a reversed relationship with the gradation value, a character detection unit that detects characters from the reversed image, and an output unit that outputs information about the detected characters.

A control method according to an aspect of the embodiment is a control method for an image processing apparatus having an output unit, wherein the image processing apparatus controls a low-brightness background, high-brightness characters on the low-brightness background, and a plurality of ruled lines. acquire a multivalued image of a document containing Detecting connected regions in which black pixels are connected in an image, and determining at least one of the size relationship of the detected connected regions, the positional relationship of the detected connected regions, or the distance between the detected connected regions Based on one, a plurality of connected regions are combined, and in a multilevel image, the relationship between the grayscale value of the low-luminance background region and the grayscale value of the high-luminance character region in the region corresponding to the combined connected region. generating a reversed image in which the is reversed; detecting characters from the reversed image; and outputting information about the detected characters from an output unit.

A control program according to one aspect of an embodiment is a control program for a computer having an output unit, and an image of a document including a low-brightness background, high-brightness characters on the low-brightness background, and a plurality of ruled lines is captured. Acquire a multi-valued image, generate a binary image by binarizing the multi-valued image, generate a ruled-line-removed image by removing multiple ruled-line components from the binary image, and connect black pixels in the ruled-line-removed image. based on at least one of the size relationship of the plurality of detected connected regions, the positional relationship of the plurality of connected regions, or the distance between the plurality of connected regions. , and reverses the relationship between the gradation values of the low-brightness background area and the gradation value of the high-brightness character area in the multivalued image corresponding to the connected areas joined by the joining part A computer is caused to generate a reversed image, detect characters from the reversed image, and output information about the detected characters from an output unit.

According to the present embodiment, the image processing apparatus, control method, and control program can satisfactorily detect characters from an image of a document including a low-luminance background, high-luminance characters on the low-luminance background, and ruled lines. becomes.

The objects and advantages of the invention may be realized and obtained by means of the elements and combinations particularly pointed out in the claims. Both the foregoing general description and the following detailed description are exemplary and explanatory, and are not limiting of the invention as claimed.

1 is a diagram showing a schematic configuration of an image processing system 1 according to an embodiment; FIG. 2 is a diagram showing a schematic configuration of a second storage device 210 and a second CPU 220; FIG. 4 is a flowchart showing the operation of image reading processing; 4 is a flowchart showing the operation of recognition processing; 5 is a schematic diagram showing an example of a multivalued image 500; FIG. 5 is a schematic diagram showing an example of a binary image 510; FIG. 6 is a schematic diagram showing an example of a binary image 600; FIG. FIG. 6 is a schematic diagram showing an example of a ruled-line-removed image 610. FIG. FIG. 7 is a schematic diagram showing an example of a ruled-line-removed image 700; FIG. 7 is a schematic diagram showing an example of a ruled-line-removed image 720; FIG. 8 is a schematic diagram showing an example of a reversed image 800; FIG. 8 is a schematic diagram showing an example of a reversed binary image 810; 2 is a block diagram showing a schematic configuration of a processing device 230; FIG.

An image processing apparatus, a control method, and a control program according to one aspect of the present disclosure will be described below with reference to the drawings. However, it should be noted that the technical scope of the present invention is not limited to those embodiments, but extends to the invention described in the claims and equivalents thereof.

FIG. 1 is a diagram showing a schematic configuration of an image processing system according to an embodiment. As shown in FIG. 1, the image processing system 1 has an image reading device 100 and an information processing device 200 .

The image reading device 100 is, for example, a scanner device. The image reading device 100 is connected to the information processing device 200 . The information processing device 200 is an example of an image processing device, such as a personal computer.

The image reading device 100 has a first interface device 101 , an imaging device 102 , a first storage device 110 , and a first CPU (Control Processing Unit) 120 .

The first interface device 101 has an interface circuit conforming to a serial bus such as USB (Universal Serial Bus), and is electrically connected to the information processing device 200 to transmit and receive image data and various information. Further, instead of the first interface device 101, a communication device having an antenna for transmitting and receiving wireless signals and a wireless communication interface circuit for transmitting and receiving signals through a wireless communication line according to a predetermined communication protocol may be used. good. The predetermined communication protocol is, for example, a wireless LAN (Local Area Network).

The imaging device 102 has a reduction optical system type imaging sensor that includes imaging elements that are CCDs (Charge Coupled Devices) linearly arranged in the main scanning direction. Further, the imaging device 102 includes a light source for irradiating light, a lens for forming an image on an imaging element, and an A/D converter for amplifying and analog/digital (A/D)-converting an electric signal output from the imaging element. have a vessel. In the image pickup apparatus 102, the image sensor picks up an image of the surface of the document being conveyed to generate and output an analog image signal, and the A/D converter converts the analog image signal into a digital image. Generate and output the input image of . The input image is a color multivalued image in which each pixel data is composed of a total of 24 bits of R (red) value, G (green) value, and B (blue) value represented by 8 bits for each color of RGB, for example. Incidentally, instead of the CCD, a CIS (Contact Image Sensor) of an equal-magnification optical system type having an imaging element of CMOS (Complementary Metal Oxide Semiconductor) may be used.

The first storage device 110 includes memory devices such as RAM (Random Access Memory) and ROM (Read Only Memory), fixed disk devices such as hard disks, or portable storage devices such as flexible disks and optical disks. The first storage device 110 also stores computer programs, databases, tables, and the like used for various processes of the image reading apparatus 100 . The computer program may be installed in the first storage device 110 from a computer-readable portable recording medium using a known setup program or the like. Examples of portable recording media include CD-ROMs (compact disk read only memory) and DVD-ROMs (digital versatile disk read only memory). Also, the first storage device 110 stores an input image and the like generated by the imaging device 102 .

The first CPU 120 operates based on a program stored in advance in the first storage device 110 . A DSP (digital signal processor), LSI (large scale integration), or the like may be used instead of the first CPU 120 . Also, instead of the first CPU 120, an ASIC (Application Specific Integrated Circuit), an FPGA (Field-Programming Gate Array), or the like may be used.

The first CPU 120 is connected to the first interface device 101, the imaging device 102, the first storage device 110, and the like, and controls these units. The first CPU 120 performs document reading control of the imaging device 102, data transmission/reception control with the information processing device 200 via the first interface device 101, and the like.

The information processing device 200 has a second interface device 201 , an input device 202 , a display device 203 , a second storage device 210 , a second CPU 220 and a processing device 230 . Each unit of the information processing apparatus 200 will be described in detail below.

The second interface device 201 has an interface circuit similar to that of the first interface device 101 of the image reading device 100 and connects the information processing device 200 and the image reading device 100 . Further, instead of the second interface device 201, a communication device having an antenna for transmitting and receiving wireless signals and a wireless communication interface circuit for transmitting and receiving signals through a wireless communication line according to a predetermined communication protocol such as a wireless LAN is provided. may be used.

The input device 202 has an input device such as a keyboard and a mouse, and an interface circuit for acquiring signals from the input device, and outputs a signal corresponding to a user's operation to the second CPU 220 .

The display device 203 is an example of an output unit. The display device 203 has a display composed of a liquid crystal, an organic EL, or the like and an interface circuit for outputting image data to the display. on the display.

The second storage device 210 has a memory device similar to the first storage device 110 of the image reading device 100, a fixed disk device, a portable storage device, and the like. The second storage device 210 stores computer programs, databases, tables, etc. used for various processes of the information processing device 200 . The computer program may be installed in the second storage device 210 from a computer-readable portable recording medium such as CD-ROM, DVD-ROM, etc. using a known setup program or the like. The second storage device 210 also stores an input image received from the image reading device 100 and various processed images obtained by performing image processing on the input image by the processing device 230 .

The second CPU 220 operates based on a program pre-stored in the second storage device 210 . A DSP, LSI, ASIC, FPGA, or the like may be used instead of the second CPU 220 .

The second CPU 220 is connected to the second interface device 201, the input device 202, the display device 203, the second storage device 210, the processing device 230, etc., and controls these units. The second CPU 220 performs data transmission/reception control with the image reading device 100 via the second interface device 201, input control of the input device 202, display control of the display device 203, image processing control of the processing device 230, and the like.

The processing device 230 performs predetermined image processing on the input image. The processing device 230 is composed of a CPU, DSP, LSI, ASIC, FPGA, or the like.

FIG. 2 is a diagram showing a schematic configuration of the second storage device 210 and the second CPU 220. As shown in FIG.

As shown in FIG. 2, the second storage device 210 stores an acquisition program 211, a binary image generation program 212, a ruled line removal image generation program 213, an area detection program 214, a combination program 215, a reverse image generation program 216, a reverse binary Programs such as an image generation program 217, a character detection program 218 and an output control program 219 are stored. Each of these programs is a functional module implemented by software running on a processor. The second CPU 220 reads each program stored in the second storage device 210 and operates according to each read program. As a result, the second CPU 220 includes an acquisition unit 221, a binary image generation unit 222, a ruled line removal image generation unit 223, an area detection unit 224, a combination unit 225, a reversed image generation unit 226, a reversed binary image generation unit 227, and a character detection unit. It functions as a unit 228 and an output control unit 229 .

FIG. 3 is a flowchart showing the operation of image reading processing by the image reading apparatus 100. As shown in FIG. The operation of the image reading process will be described below with reference to the flowchart shown in FIG. The operation flow described below is executed mainly by the first CPU 120 in cooperation with each element of the image reading apparatus 100 based on a program stored in the first storage device 110 in advance.

First, the imaging device 102 generates an input image obtained by imaging a form such as an invoice as a document, and stores it in the first storage device 110 (step S101).

This document includes at least a low-brightness background such as black, high-brightness characters such as white placed on the low-brightness background, and a plurality of ruled lines. The brightness of the low-brightness background is lower than the predetermined brightness, and the brightness of the high-brightness characters is higher than the brightness of the low-brightness background. That is, in the input image, the brightness value of the low-brightness background area is lower than a predetermined brightness value (eg, 128), and the brightness value of the high-brightness character area is higher than the brightness value of the low-brightness background area. Similarly to the luminance of the low-luminance background, the luminance of the ruled lines is lower than the luminance of the high-luminance characters. Lower.

The document may further include a high-brightness background, such as white, and low-brightness characters, such as black, placed on the high-brightness background. The brightness of the high-brightness background is higher than the predetermined brightness, and the brightness of the low-brightness characters is lower than the brightness of the high-brightness background. That is, in the input image, the brightness value of the high-brightness background area is higher than the predetermined brightness value, and the brightness value of the low-brightness character area is higher than the brightness value of the high-brightness background area.

Next, the first CPU 120 transmits the input image saved in the first storage device 110 to the information processing device 200 via the first interface device 101 (step S102), and ends the series of steps.

FIG. 4 is a flow chart showing the operation of recognition processing by the information processing apparatus 200 . The operation of the recognition process will be described below with reference to the flowchart shown in FIG. The operation flow described below is mainly executed by the processing device 230 in cooperation with each element of the information processing device 200 based on a program stored in the second storage device 210 in advance.

First, the acquisition unit 221 acquires an input image as a multivalued image from the image reading device 100 via the second interface device 201, and stores it in the second storage device 210 (step S201). Note that the acquisition unit 221 may generate a black-and-white multivalued image from the input image, which is a color multivalued image, and acquire it as a multivalued image. In that case, the acquisition unit 221 calculates the luminance value for each pixel in the input image according to the following formula, and generates a black-and-white multivalued image using the calculated luminance value as the gradation value of the pixel corresponding to each pixel. do.
Luminance value = 0.299 x (R value) + 0.587 x (G value) + 0.114 x (B value)

FIG. 5A is a schematic diagram showing an example of a multivalued image 500. FIG.

As shown in FIG. 5A, a multivalued image 500 includes a high-brightness background area 501, a low-brightness character area 502, a low-brightness background area 503, a high-brightness character area 504, and a plurality of ruled line areas 505. be The high brightness background area 501 has a white color and high brightness. The low-brightness character area 502 is black and has a lower brightness than the high-brightness background area 501 , and is arranged on the high-brightness background area 501 . The low luminance background area 503 has a black color and low luminance. The high-brightness character area 504 is white and has a higher brightness than the low-brightness background area 503 , and is placed on the low-brightness background area 503 . The ruled line area 505 has black color and low luminance, and is continuous with the low luminance background area 503 . At least part of the low-luminance character area 502 and the high-luminance character area 504 are arranged within a range surrounded by the ruled line area 505 .

Next, the binary image generator 222 generates a binary image by binarizing the acquired multivalued image (step S202). The binary image generation unit 222 sets pixels having a gradation value equal to or greater than the first threshold in the multivalued image as white pixels and pixels having a gradation value less than the first threshold as black pixels. Generate as The first threshold is set to a value between the gradation value of a pixel forming a general background and the gradation value of a pixel forming a character through preliminary experiments. As a result, in the binary image, the pixels corresponding to the low-brightness characters, the low-brightness background, and the ruled lines in the multi-valued image are black pixels, and the pixels corresponding to the high-brightness background and high-brightness characters are white pixels.

Note that the first threshold may be determined by Otsu's binarization process. In general, the brightness of the low-brightness background and high-brightness characters differs for each form. The information processing apparatus 200 determines the relationship between the luminance of the low-luminance background and the luminance of the high-luminance characters in the multivalued image by determining the first threshold value by Otsu's binarization process. Also, it is possible to distinguish well between the low-brightness background and high-brightness characters.

FIG. 5B is a schematic diagram showing an example of a binary image 510. As shown in FIG.

FIG. 5B shows a binary image 510 obtained by binarizing the multivalued image 500 shown in FIG. 5A. As shown in FIG. 5B, in a binary image 510, pixels in a region 511 corresponding to the bright background region 501 and pixels in a region 514 corresponding to the bright text region 504 are white pixels. On the other hand, in the binary image 510, pixels in a region 512 corresponding to the low-luminance character region 502, pixels in a region 513 corresponding to the low-luminance background region 503, and pixels in a region 515 corresponding to the ruled line region 505 are black. is a pixel.

Next, in the binary image, the ruled-line-removed image generation unit 223 removes white pixels from a first connected region in which black pixels are connected in the binary image and whose lengths in the horizontal and vertical directions are less than the second threshold. to remove the low-brightness character component from the binary image (step S203).

The ruled-line-removed image generator 223 specifies the first connected region by grouping black pixels in the binary image by labeling. The ruled-line-removed-image generation unit 223 extracts the first connected regions whose horizontal length and vertical length are less than the second threshold from among the specified first connected regions as removal target regions. The second threshold is set to a length on the image corresponding to the maximum horizontal and vertical lengths (for example, 40 points) of characters used in general forms. Note that the second threshold for comparison with the horizontal length of the first connected region and the second threshold for comparison with the vertical length may be set to different values. The ruled-line-removed image generation unit 223 converts each extracted removal target area (black pixel) into a white pixel in the binary image.

FIG. 6A is a schematic diagram showing an example of a binary image 600 from which low-brightness character components have been removed.

A binary image 600 shown in FIG. 6A is an image obtained by removing low-brightness character components from the binary image 510 shown in FIG. 5B. As shown in FIG. 6A, in the binary image 600, only pixels in a region 603 corresponding to the low luminance background region 503 and pixels in a region 605 corresponding to the ruled line region 505 remain as black pixels. On the other hand, in the binary image 600, pixels in an area 602 corresponding to the low-luminance character area 502 are converted to white pixels.

Next, the ruled-line-removed image generation unit 223 generates a ruled-line-removed image by removing a plurality of ruled-line components from the binary image (step S204). The ruled-line-removed image generation unit 223 converts a portion whose horizontal or vertical length is equal to or less than a predetermined value in the first connected region remaining in the binary image from which the low-brightness character component has been removed into white pixels. By doing so, a ruled-line-removed image is generated. The predetermined value is set, for example, to a length on the image corresponding to the maximum width (for example, 3 points) of a ruled line used in a general form. For example, if a white pixel exists within a range of a predetermined value from each black pixel in the first connected region, the ruled-line-removed image generation unit 223 converts the black pixel into a white pixel, thereby generating a white pixel. inflate. The ruled-line-removed image generation unit 223 can efficiently remove the ruled-line component by dilating the white pixels in the binary image, and can reduce the processing load in generating the ruled-line-removed image.

Note that the ruled-line-removed image generation unit 223 extracts a portion whose length in the horizontal or vertical direction is equal to or less than a predetermined value from each first connected region, converts each extracted portion into white pixels, and removes the ruled line. An image may be generated.

FIG. 6B is a schematic diagram showing an example of a ruled line removed image 610. As shown in FIG.

A ruled-line-removed image 610 shown in FIG. 6B is an image obtained by removing ruled-line components from the binary image 600 shown in FIG. 6A. As shown in FIG. 6B, in the ruled-line-removed image 610, only pixels in a region 613 corresponding to the low-brightness background region 503 remain as black pixels, and pixels in a region 615 corresponding to the ruled-line region 505 are a few pixels in a region 616. are converted to white pixels, except for In the ruled-line-removed image 610, an area 614 corresponding to the high-brightness character area 504 is expanded as well as the area corresponding to the high-brightness background area. In particular, in the ruled-line-removed image 610, an area 617 corresponding to the high-brightness character area is expanded, so that areas 618 and 619 corresponding to the low-brightness background area are separated.

Next, the ruled-line-removed image generator 223 removes noise components from the generated ruled-line-removed image (step S205).

For example, the ruled-line-removed image generation unit 223 extracts, as a noise component, black pixels whose continuous length in the horizontal or vertical direction is equal to or less than a predetermined number of pixels (for example, 2 pixels) in the ruled-line-removed image, and converts them into white pixels. remove by As a result, the ruled-line-removed image generation unit 223 can remove the slightly remaining ruled-line component from the ruled-line-removed image.

In addition, the ruled-line-removed image generation unit 223 generates noise for regions in which black pixels are connected in the ruled-line-removed image and whose length in the horizontal or vertical direction is equal to or less than a predetermined length (for example, a length corresponding to 5 points). It may be removed by extracting it as a component and converting it to a white pixel. This also allows the ruled-line-removed image generation unit 223 to remove the slightly remaining ruled-line component from the ruled-line-removed image.

In addition, the ruled-line-removed image generation unit 223 generates a noise component of an area in which black pixels are connected to each other in the ruled-line-removed image and in which the ratio of the number of black pixels to the total number of pixels in the area surrounded by the area is less than a predetermined ratio. , and may be removed by converting to white pixels. The predetermined ratio is set, for example, to the maximum value (for example, 33%) of the ratio of characters or pattern portions in commonly used seals. As a result, the ruled-line-removed image generation unit 223 can remove components such as a seal from the ruled-line-removed image.

FIG. 7A is a schematic diagram showing an example of a ruled-line-removed image 700 from which noise components have been removed.

A ruled-line-removed image 700 shown in FIG. 7A is an image obtained by removing noise components from the ruled-line-removed image 610 shown in FIG. 6B. As shown in FIG. 7A, in the ruled-line-removed image 700, only pixels in an area 703 corresponding to the low-brightness background area 503 remain as black pixels, and all pixels in an area 706 corresponding to the ruled-line area 505 are converted to white pixels. there is

Next, the area detection unit 224 detects a second connected area where black pixels are connected in the ruled-line-removed image (step S206). Below, the second connection region is simply referred to as the connection region. The ruled-line-removed image generator 223 detects connected regions by grouping black pixels in the ruled-line-removed image from which noise components have been removed by labeling.

Next, the combining unit 225 combines the plurality of connected regions detected by the region detecting unit 224 (step S207). The connecting part 225 connects the connecting regions based on at least one of the size relationship of the connecting regions, the positional relationship of the connecting regions, or the distance between the connecting regions.

The combiner 225 extracts all combinations of two connected regions among the detected connected regions. The combining unit 225 determines whether or not the two connected regions associated with each combination satisfy a predetermined condition for each extracted combination. The first condition among the predetermined conditions is that the length difference between two connected regions in a predetermined direction is less than or equal to a predetermined difference (for example, two pixels). The second condition among the predetermined conditions is that the distance between the center positions of the two connected regions in the predetermined direction is less than or equal to the first predetermined distance (for example, 2 pixels). The third condition among the predetermined conditions is that the two connected regions do not overlap in the direction perpendicular to the predetermined direction and the distance between the two connected regions is a second predetermined distance (for example, 3 pixels) or less. . The predetermined direction is the horizontal direction and/or the vertical direction, if the predetermined direction is the horizontal direction, the direction orthogonal to the predetermined direction is the vertical direction, and if the predetermined direction is the vertical direction, the direction orthogonal to the predetermined direction is Horizontal. The predetermined condition may be any one or two of the first condition, the second condition and the third condition.

A concatenated region may be split into two concatenated regions by dilating a bright character region within the concatenated region, in which case the distance between the two concatenated regions is less than or equal to the size of the characters in the concatenated region. is likely to be Therefore, the second predetermined distance is set, for example, to a length on the image corresponding to the minimum horizontal and vertical lengths (eg, 40 points) of characters used in general forms. Since the length of the connected area in the predetermined direction (the width of the connected area) is greater than one side of the high-brightness character area in the connected area, the second predetermined distance is set to the length of the connected area in the predetermined direction. good too.

In the example shown in FIG. 7A, the difference between the length L1 of the connection region 708 and the length L2 of the connection region 708 in the vertical direction is sufficiently small, so that the connection region 708 and the connection region 709 satisfy the first condition. Also, since the distance between the central position P1 of the connecting region 708 and the central position P2 of the connecting region 709 in the vertical direction is sufficiently small, the connecting regions 708 and 709 satisfy the second condition. Also, since the connecting regions 708 and 709 do not overlap in the horizontal direction and the distance D1 between the connecting regions 708 and 709 is sufficiently small, the connecting regions 708 and 709 satisfy the third condition. As such, connecting region 708 and connecting region 709 are joined.

On the other hand, since the difference between the length L1 of the connection region 708 and the length L3 of the connection region 710 in the vertical direction is sufficiently small, the connection region 708 and the connection region 710 satisfy the first condition. However, since the distance between the center position P1 of the connection region 708 and the center position P3 of the connection region 710 in the vertical direction is sufficiently large, the connection regions 708 and 710 do not satisfy the second condition. Moreover, since the connecting regions 708 and 710 overlap in the horizontal direction, the connecting regions 708 and 710 do not satisfy the third condition either.

Also, since the difference between the length L4 of the connection region 708 and the length L5 of the connection region 710 in the horizontal direction is sufficiently large, the connection regions 708 and 710 do not satisfy the first condition. Moreover, since the distance between the center position P4 of the connection region 708 and the center position P5 of the connection region 710 in the horizontal direction is sufficiently large, the connection regions 708 and 710 do not satisfy the second condition either. Moreover, since the distance D2 between the connecting regions 708 and 710 in the vertical direction is sufficiently large, the connecting regions 708 and 710 do not satisfy the third condition either. Therefore, connecting region 708 and connecting region 710 are not connected.

If there is a combination that satisfies a predetermined condition, the joining unit 225 joins the two connected regions related to that combination. Note that, if there are a plurality of combinations that satisfy a predetermined condition for a specific connected region, the connecting unit 225 connects the specific connected region and all connected regions related to the combination that satisfies the predetermined condition.

If there is an effective pixel within a third predetermined distance from each pixel in an image in which only pixels corresponding to black pixels included in two connected regions to be combined are effective pixels, the combining unit 225 The effective pixels are dilated by transforming them into effective pixels. The third predetermined distance is set, for example, to the same distance as the second predetermined distance set under the third condition. Further, if there is an invalid pixel within the third predetermined distance from each effective pixel in the image, the combining unit 225 shrinks the effective pixel by converting the pixel into an invalid pixel. Then, the combining unit 225 combines the two connected areas by converting the white pixels in the ruled line-removed image corresponding to the effective pixels in the image into black pixels.

Note that the combining unit 225 may combine two connected regions by converting all white pixels in a region surrounded by a circumscribing rectangle that includes the two connected regions to be combined into black pixels.

In addition, combiner 225 converts white pixels in all connected regions to black pixels. For all connected regions, the combining unit 225 determines that, in an image in which only pixels corresponding to black pixels included in each connected region are effective pixels, if there is an effective pixel within a fourth predetermined distance from each pixel, A valid pixel is dilated by converting the pixel to a valid pixel. The fourth predetermined distance is set to a length on the image corresponding to the maximum horizontal or vertical length (for example, 40 points) of characters used in general forms. Further, if there is an invalid pixel within the fourth predetermined distance from each effective pixel in the image, the combining unit 225 shrinks the effective pixel by converting the pixel into an invalid pixel. Then, the combining unit 225 converts the white pixels in the connected area into black pixels by converting the white pixels in the ruled line-removed image corresponding to the effective pixels in the image into black pixels.

Note that the combining unit 225 may convert the white pixels in the connected regions to black pixels by converting all the white pixels in the regions enclosed by the circumscribing rectangles including the connected regions into black pixels.

FIG. 7B is a schematic diagram showing an example of a ruled-line-removed image 720 in which connected areas are combined.

A ruled-line-removed image 720 shown in FIG. 7B is an image in which connected areas are combined in the ruled-line-removed image 700 shown in FIG. 7A. As shown in FIG. 7B, in the ruled-line-removed image 720, the connected regions 708 and 709 of the ruled-line-removed image 700 are combined, and the white pixels in the connected regions 708 and 709 are converted to black pixels. A connecting region 721 is formed. In the ruled-line-removed image 700, the connected region 710 is not connected to other connected regions, but in the ruled-line-removed image 720, the white pixels in the connected region 710 are converted to black pixels to form a connected region 722. there is

Next, the reversed image generation unit 226 divides a region corresponding to the connected region and including high-brightness characters of a predetermined size or larger in the multivalued image into the gradation value of the low-brightness background region and the high-brightness character region. is extracted as an inversion target area for reversing the relationship with the gradation value of (step S208). This connected area is an area in which internal white pixels are converted to black pixels in step S207, and includes both connected areas connected by the connecting unit 225 and connected areas not connected by the connecting unit 225. .

First, the reversed image generation unit 226 subtracts the gradation value of each pixel in the corresponding region corresponding to each connected region of the ruled-line-removed image from the maximum value of the gradation range in the multivalued image. A reversed transformed image is generated by transforming to the subtracted value. Next, the reversed image generator 226 generates a transformed binary image by binarizing the transformed image. Similar to the binary image generation unit 222, the reverse image generation unit 226 sets pixels in the converted image whose gradation value is equal to or greater than the first threshold as white pixels, and sets pixels whose gradation value is less than the first threshold as white pixels. An image with black pixels is generated as a converted binary image.

Next, the reversed image generation unit 226 groups black pixels in each second corresponding region by labeling for each second corresponding region corresponding to each corresponding region in the converted binary image, thereby generating a black pixel identifies the third connecting region that connects the . The reversed image generator 226 determines whether or not each second corresponding region includes a third connected region whose horizontal length and/or vertical length is greater than or equal to a third threshold. The third threshold is set to a length on the image corresponding to the minimum horizontal and vertical lengths (for example, 8 points) of characters used in general forms. Note that the third threshold for comparison with the horizontal length of the third connected region and the third threshold for comparison with the vertical length of the third connected region may be set to different values. The reversed image generation unit 226 regards the third connected area whose horizontal length and/or vertical length is equal to or greater than the third threshold as an area corresponding to a high-brightness character of a predetermined size or greater. The reversed image generation unit 226 reverses the corresponding region of the multivalued image corresponding to the second corresponding region including the third connected region whose horizontal length and/or vertical length is equal to or greater than the third threshold. Extract as target area.

Next, the reversed image generation unit 226 generates a reversed image by reversing the relationship between the gradation values of the low-luminance background region and the gradation value of the high-luminance character region in each region to be reversed in the multivalued image. (Step S209). The reversed image generation unit 226 converts the gradation value of each pixel in each region to be reversed into a subtraction value obtained by subtracting the gradation value of each pixel from the maximum value of the gradation range in the multivalued image. to generate an inverted image.

FIG. 8A is a schematic diagram showing an example of a reversed image 800. FIG.

A reversed image 800 shown in FIG. 8A is an image generated from the multivalued image 500 shown in FIG. 5A. As shown in FIG. 8A, in the reversed image 800, the relationship between the gradation values of the area 803 corresponding to the low-luminance background area 503 and the gradation value of the area 804 corresponding to the high-luminance character area 504 is reversed. . That is, in the reversed image 800, the gradation values of the area 801 corresponding to the high-brightness background area 501, the area 802 corresponding to the low-brightness character area 502, and the area 805 corresponding to the ruled line area 505 are has not changed. On the other hand, in the reversed image 800, a region 803 corresponding to the low-brightness background region 503 and a region 804 corresponding to the high-brightness character region 504 change such that the brightness value of the region 804 is lower than that of the region 803. ing.

In addition, as described above, the reversed image generation unit 226 generates a low-brightness character in the corresponding area only when a high-brightness character of a predetermined size or larger is included in the area corresponding to the connected area in the multivalued image. The relationship between the gradation value of the background area and the gradation value of the high-brightness character area is reversed. In each connected region the background is presumed to have low intensity, but in each connected region the characters can have even lower intensity than the background, such as when the background has dark gray and the characters have black. have a nature. The reversed image generation unit 226 extracts only corresponding regions containing high-brightness characters of a predetermined size or larger as inversion target regions. is extracted as the inversion target region. As a result, the reversed image generator 226 can prevent the relationship between the background gradation value and the character gradation value from being reversed in an area where the character has lower luminance than the background.

Next, the reversed binary image generator 227 generates a reversed binary image by binarizing the reversed image (step S210). Similar to the binary image generation unit 222, the reverse binary image generation unit 227 treats pixels in the reverse image whose gradation value is equal to or greater than the first threshold as white pixels, and pixels whose gradation value is less than the first threshold. is generated as an inverted binary image. In the reversed binary image, pixels corresponding to characters and ruled lines in the reversed image are black pixels, and pixels corresponding to the background are white pixels.

FIG. 8B is a schematic diagram showing an example of an inverted binary image 810. As shown in FIG.

FIG. 8B shows an inverted binary image 810 obtained by binarizing the inverted image 800 shown in FIG. 8A. As shown in FIG. 8B, in an inverted binary image 810, pixels in an area 811 corresponding to the high-luminance background area 501 of the multi-valued image 500 and pixels in an area 813 corresponding to the low-luminance background area 503 are white pixels. is. On the other hand, in the reverse binary image 810, the pixels in the region 812 corresponding to the low-luminance character region 502, the pixels in the region 814 corresponding to the high-luminance character region 504, and the pixels in the region 815 corresponding to the ruled line region 505 are Black pixels.

Next, the character detection unit 228 detects characters from the reversed binary image generated from the reversed image (step S211). The character detection unit 228 detects characters from the reverse binary image using a known OCR (Optical Character Recognition) technique.

Next, the output control unit 229 displays the detected characters on the display device 203 (step S212), and ends the series of steps. Note that the output control unit 229 may display a reversed image or a reversed binary image on the display device 203 instead of or in addition to the detected character as information about the detected character. The output control unit 229 may also transmit the detected characters, reversed image, or reversed binary image to a server or the like (not shown) via a communication device (not shown). Thus, the output control unit 229 outputs information regarding the detected characters.

Note that the ruled-line-removed image generation unit 223 may omit the process of step S203 and generate the ruled-line-removed image in step S204 from the binary image generated in step S202. Alternatively, the ruled-line-removed image generation unit 223 may omit the process of step S205, and the area detection unit 224 may detect, in step S206, connected areas from the ruled-line-removed image generated in step S204. In addition, the reversed image generating unit 226 omits the processing of step S208, and in step S209, the gradation values of the low-luminance background regions and the gradation values of the high-luminance character regions in all connected regions in the multivalued image are A reverse image may be generated by reversing the relationship. Alternatively, the reversed binary image generation unit 227 may omit the process of step S210, and the character detection unit 228 may detect characters from the reversed image in step S211.

As described in detail above, by operating according to the flowchart shown in FIG. 5, the information processing apparatus 200 extracts characters from a document image including a low-brightness background, high-brightness characters on the low-brightness background, and ruled lines. Good detection is possible.

The information processing apparatus 200 can satisfactorily identify the low-brightness background area by removing the ruled-line component, particularly the ruled-line component connected to the low-brightness background area. In addition, the information processing apparatus 200 suppresses erroneous identification of the low-luminance background region by recombining the low-luminance background region divided by the high-luminance character region expanded when removing the ruled line component. became possible. The information processing apparatus 200 can satisfactorily invert the gradation value of the low-luminance background region and the gradation value of the high-luminance character region by identifying the low-luminance background region satisfactorily.

In general, a form may contain characters with various densities. In character recognition processing, in order to be able to recognize each character with various densities, pixels corresponding to each character must be converted to black pixels in the binary image to be subjected to character recognition processing. . Therefore, binary images are preferably generated such that each pixel in the image is more likely to be converted to a black pixel. However, in the binary image so generated, some of the white characters may also be converted to black pixels. If the relationship between the white pixels and the black pixels inside is reversed, there is a high possibility that the white characters will be blurred. The information processing apparatus 200 generates a reversed image from the multivalued image and then generates a reversed binary image instead of binarizing the multivalued image and then inverting the gradation values. It is now possible to prevent characters from being blurred.

Also, general-purpose OCR software is generally configured to detect low-brightness characters placed on a high-brightness background. The information processing apparatus 200 reverses the gradation values of the low-luminance background area and the high-luminance character area to convert the original image including the high-luminance characters on the low-luminance background into the low-luminance characters arranged on the high-luminance background. Characters can be detected using OCR software configured to detect characters. Further, the information processing apparatus 200 detects only one of high-brightness characters and low-brightness characters from a document image including high-brightness characters on a low-brightness background and low-brightness characters on a high-brightness background. Characters can be detected using OCR software configured for

Further, the information processing apparatus 200 detects a connected area in which black pixels are connected in the image, and inverts the gradation values of the low-luminance background area and the high-luminance character area for each detected connected area. As a result, the information processing apparatus 200 can extract characters from a form image in which cells in which low-brightness characters are arranged on a high-brightness background and cells in which high-brightness characters are arranged on a low-brightness background are mixed. detection becomes possible.

Note that the information processing apparatus 200 calculates the gradation of the high-brightness background area from a multi-valued image obtained by inverting the relationship of the gradation values of each background, characters, and ruled lines with respect to the multi-valued image 500 shown in FIG. 5A. A reverse image may be generated by reversing the relationship between the value and the gradation value of the low-brightness character area. In this case, the ruled-line-removed image generation unit 223 generates a ruled-line-removed image by removing high-brightness ruled-line components from the binary image. To detect. A combining unit 225 combines the separated connected regions, and a reversed image generation unit 226 calculates the gradation value of the high-brightness background region and the gradation of the low-brightness character region in the region corresponding to the connected region in the multivalued image. Generates an inverted image with the relationship to values reversed.

FIG. 9 is a block diagram showing a schematic configuration of a processing device 230 in an information processing device according to another embodiment.

The processing device 230 executes recognition processing instead of the second CPU 220 . The processing device 230 includes an acquisition circuit 231, a binary image generation circuit 232, a ruled line removal image generation circuit 233, an area detection circuit 234, a combination circuit 235, a reversed image generation circuit 236, a reversed binary image generation circuit 237, and a character detection circuit 238. and an output control circuit 239 and the like.

The acquisition circuit 231 is an example of an acquisition section and has the same function as the acquisition section 221 . The acquisition circuit 231 acquires the input image as a multivalued image from the image reading device 100 via the second interface device 201 and stores it in the second storage device 210 .

The binary image generation circuit 232 is an example of a binary image generation section and has the same function as the binary image generation section 222 . The binary image generation circuit 232 reads the multivalued image from the second storage device 210 , generates a binary image from the multivalued image, and stores it in the second storage device 210 .

The ruled-line-removed image generation circuit 233 is an example of a ruled-line-removed image generation unit, and has the same function as the ruled-line-removed image generation unit 223 . The ruled-line-removed image generation circuit 233 reads the binary image from the second storage device 210 , generates a ruled-line-removed image from the binary image, and stores it in the second storage device 210 .

The area detection circuit 234 is an example of an area detection section and has the same function as the area detection section 224 . The area detection circuit 234 reads the ruled-line removed image from the second storage device 210 , detects connected areas in the ruled-line removed image, and saves the detection result in the second storage device 210 .

The coupling circuit 235 is an example of a coupling section and has the same function as the coupling section 225 . The combining circuit 235 reads out the ruled-line-removed image and the detection result of the connected regions from the second storage device 210 , combines the connected regions in the ruled-line-removed image based on the detection result, and stores them in the second storage device 210 . .

The reversed image generation circuit 236 is an example of a reversed image generation section and has the same function as the reversed image generation section 226 . The reversed image generating circuit 236 reads out the multivalued image and the ruled line removed image from the second storage device 210 , generates a reversed image from each read image, and stores the reversed image in the second storage device 210 .

The reverse binary image generation circuit 237 is an example of a reverse binary image generation section and has the same function as the reverse binary image generation section 227 . The reverse binary image generation circuit 237 reads the reverse image from the second storage device 210 , generates a reverse binary image from the reverse image, and outputs the reverse binary image to the display device 203 .

The character detection circuit 238 is an example of a character detection section and has the same function as the character detection section 228 . The character detection circuit 238 reads the reversed image or reversed binary image from the second storage device 210 , detects characters from the read image, and stores the detection result in the second storage device 210 .

The output control circuit 239 is an example of an output control section and has the same function as the output control section 229 . The output control circuit 239 reads the character detection result, the reversed image, or the reversed binary image from the second storage device 210 and outputs the read information to the display device 203 .

As described in detail above, the information processing apparatus can satisfactorily detect characters from an image of a document including a low-brightness background, high-brightness characters on a low-brightness background, and ruled lines, even when the processing apparatus 230 is used. became possible.

Although the preferred embodiments have been described above, the embodiments are not limited to these. For example, the division of functions between the image reading device 100 and the information processing device 200 is not limited to the example of the image processing system 1 shown in FIG. 200 can be changed as appropriate. Alternatively, the image reading device 100 and the information processing device 200 may be configured as one device.

For example, the first storage device 110 of the image reading device 100 stores each program stored in the second storage device 210 of the information processing device 200, and the first CPU 120 of the image reading device 100 stores the second CPU 120 of the information processing device 200. You may operate|move as each part implement|achieved by. Further, the image reading device 100 may have a processing device similar to the processing device 230 of the information processing device 200 .

In that case, the image reading device 100 has a display device similar to the display device 203 . Since the recognition process is executed by the image reading apparatus 100, the transmission/reception process of the input image in steps S102 and S201 is omitted. Each process of steps S202 to S212 is executed by the first CPU 120 of the image reading device 100 or a processing device. These processing operations are the same as those executed by the second CPU 220 or the processing device 230 of the information processing device 200 .

In the image processing system 1, the first interface device 101 and the second interface device 201 are connected via a network such as the Internet, a telephone line network (including a mobile terminal line network and a general telephone line network), an intranet, or the like. good too. In that case, the first interface device 101 and the second interface device 201 are provided with a communication interface circuit of the network to be connected. In that case, multiple information processing devices are distributed over a network so that image processing services can be provided in the form of cloud computing, and each information processing device cooperates to perform recognition processing, etc. You may make it share. As a result, the image processing system 1 can efficiently perform recognition processing on input images read by a plurality of image reading devices.

1 Image processing system 200 Information processing device 221 Acquisition unit 222 Binary image generation unit 223 Line-removed image generation unit 224 Area detection unit 225 Joining unit 226 Reversed image generation unit 227 Reversed binary image generation unit 228 Character detection unit 229 Output control unit

Claims (6)

an acquisition unit that acquires a multivalued image obtained by imaging a document including a low-brightness background, high-brightness characters on the low-brightness background, and a plurality of ruled lines;
a binary image generation unit that generates a binary image by binarizing the multivalued image such that the pixels corresponding to the low-luminance background are black pixels and the pixels corresponding to the high-luminance characters are white pixels ;
a ruled-line-removed image generating unit that generates a ruled-line-removed image by removing components of the plurality of ruled lines from the binary image;
an area detection unit that detects a connected area where black pixels are connected in the ruled-line-removed image;
Based on at least one of the size relationship of the plurality of connected regions detected by the region detection unit, the positional relationship of the plurality of connected regions, or the distance between the plurality of connected regions, a connecting portion that connects a plurality of connecting regions;
In the multi-valued image, a reversed image is generated by reversing the relationship between the gradation values of the low-luminance background region and the gradation values of the high-luminance character region in the region corresponding to the connected region combined by the combining unit. a reverse image generator;
a character detection unit that detects characters from the reversed image;
an output unit that outputs information about the detected characters;
An image processing device comprising: The ruled-line-removed image generating unit converts a portion having a horizontal or vertical length equal to or less than a predetermined value in a region where black pixels are connected in the binary image into white pixels, thereby generating the ruled-line-removed image. 2. The image processing apparatus according to claim 1, which generates Only when a region corresponding to the connected region combined by the combining unit in the multivalued image contains a high-brightness character of a predetermined size or larger, the reverse image generation unit 3. The image processing apparatus according to claim 1, wherein the relationship between the gradation value of the low-luminance background area and the gradation value of the high-luminance character area is reversed. further comprising a reversed binary image generator that generates a reversed binary image by binarizing the reversed image;
4. The image processing apparatus according to claim 1, wherein said character detection unit detects said characters from said reversed binary image. A control method for an image processing device having an output unit, the image processing device comprising:
Acquiring a multivalued image of a document containing a low-brightness background, high-brightness characters on the low-brightness background, and a plurality of ruled lines,
generating a binary image by binarizing the multivalued image such that pixels corresponding to the low-brightness background are black pixels and pixels corresponding to the high-brightness characters are white pixels ;
generating a ruled-line-removed image by removing components of the plurality of ruled lines from the binary image;
Detecting connected regions where black pixels are connected in the ruled-line-removed image;
the plurality of connected regions based on at least one of the size relationship of the detected plurality of connected regions, the positional relationship of the plurality of connected regions, or the distance between the plurality of connected regions; and
generating a reversed image by reversing the relationship between the gradation values of the low-luminance background region and the gradation value of the high-luminance character region in the region corresponding to the combined connected region in the multivalued image;
detecting characters from the reversed image;
outputting information about the detected characters from the output unit;
A control method comprising: A control program for a computer having an output unit,
Acquiring a multivalued image of a document containing a low-brightness background, high-brightness characters on the low-brightness background, and a plurality of ruled lines,
generating a binary image by binarizing the multivalued image such that pixels corresponding to the low-brightness background are black pixels and pixels corresponding to the high-brightness characters are white pixels ;
generating a ruled-line-removed image by removing components of the plurality of ruled lines from the binary image;
Detecting connected regions where black pixels are connected in the ruled-line-removed image;
the plurality of connected regions based on at least one of the size relationship of the detected plurality of connected regions, the positional relationship of the plurality of connected regions, or the distance between the plurality of connected regions; and
generating a reversed image by reversing the relationship between the gradation values of the low-luminance background region and the gradation value of the high-luminance character region in the region corresponding to the combined connected region in the multivalued image;
detecting characters from the reversed image;
outputting information about the detected characters from the output unit;
A control program characterized by causing the computer to execute:

Images