JP2019016897A - Image processing system, and computer program - Google Patents

Abstract

To identify the document area in a read image accurately.SOLUTION: An image processing system includes an image data acquisition part for acquiring read image data indicating a read image, generated by reading a document optically, and including a first image showing the document and a second image showing the area other than the document, a first image generation part for identifying multiple pixels satisfying specific conditions as multiple first pixels, and identifying multiple pixels not satisfying specific conditions as multiple second pixels, thus generating first image data showing multiple first pixels and second pixels, a second image generation part generating second image data showing multiple edge pixels in the read image, and an identification part for identifying the document area in the read image by using at least the first and second image data. The identification conditions include brilliance within a specific brilliance range, and the specific brilliance range is a range including the brilliance of pixels corresponding to the second image.SELECTED DRAWING: Figure 4

Description

  The present specification relates to image processing for specifying a document area in a read image.

  Conventionally, read image data representing a read image including a document (for example, a document image) is generated using an image reading device such as a scanner. For example, the computer disclosed in Patent Document 1 acquires an edge image from a color document image having three channels of R, G, and B using an existing edge extraction method. From the color document image, the computer has a pixel having a relatively small difference in values of the three channels R, G, and B, and a pixel having a relatively large difference in values of the three components R, G, and B; A binarized image indicating is obtained. The computer specifies a predetermined area in the color document using an image obtained by merging the edge image and the binarized image.

Japanese Patent Laid-Open No. 2006-167810

  However, in the above technique, for example, there is a possibility that the predetermined area cannot be accurately identified when the color of the pixel corresponding to outside the predetermined area is relatively close to the color of the pixel corresponding to the predetermined area.

  This specification discloses a technique that can accurately specify a document area in a read image.

  The technology disclosed in the present specification has been made to solve at least a part of the above-described problems, and can be realized as the following application examples.

Application Example 1 An image processing apparatus, an image data acquisition unit that acquires read image data generated by optically reading a document, wherein the read image data is a first indicating the document. A plurality of pixels satisfying a specific condition using the image data acquisition unit and the read image data, each of which represents a read image including an image and a second image indicating a region other than the document. The plurality of first pixels and the plurality of second pixels are identified as the first pixels and a plurality of pixels that do not satisfy the specific condition are identified as the plurality of second pixels. A first image generation unit configured to generate first image data indicating a pixel, wherein the specific condition includes a luminance within a specific luminance range, and the specific luminance range corresponds to the second image; In the range including pixel brightness A first image generation unit, a second image generation unit for generating second image data indicating a plurality of edge pixels indicating edges in the read image using the read image data, and at least the first image generation unit. An image processing apparatus comprising: a specifying unit that specifies a document area corresponding to the document in the read image using one image data and the second image data.

  According to the above configuration, the plurality of first pixels that satisfy the specific condition including having the luminance within the specific luminance range that is the luminance range including the luminance of the plurality of pixels in the second image, and the specific condition A document region in the read image is specified using first image data indicating a plurality of second pixels not satisfying the above and second image data indicating a plurality of edge pixels. As a result, the accuracy of specifying the document area in the read image can be improved.

  The technology disclosed in the present specification can be realized in various forms. For example, a multifunction device, a scanner, a printer, an image processing method, a function of these devices, or a computer for realizing the above method It can be realized in the form of a program, a recording medium recording the computer program, and the like.

It is a figure which shows the structure of an image processing system. It is a flowchart of the image processing of a present Example. It is a figure which shows an example of the image used by image processing. It is a flowchart of an original pixel specifying process. 6 is a diagram illustrating an example of an image used in document pixel specifying processing. FIG. It is a figure which shows an example of the histogram of the reduced image MI. It is a flowchart of a search range determination process. It is explanatory drawing of a search range determination process. It is a flowchart of an outline specific process. It is explanatory drawing of an outline specific process. It is a flowchart of a document area specifying process. It is a flowchart of a repair process.

A. First embodiment:
A-1. Configuration of Image Processing System FIG. 1 is a diagram illustrating a configuration of an image processing system as an embodiment. This image processing system includes an image processing apparatus 100 and a scanner 400. The image processing apparatus 100 and the scanner 400 are connected to the network NT and can communicate with each other.

  The image processing apparatus 100 is, for example, a personal computer (for example, a desktop computer or a tablet computer). The image processing apparatus 100 includes a CPU 110 as a controller of the image processing apparatus 100, a volatile storage device 120, a nonvolatile storage device 130, a display unit 140 that displays an image, an operation unit 150 that receives an operation by a user, Interface 190. These elements are connected to each other via a bus.

  The CPU 110 is an arithmetic device (processor) that performs data processing. The volatile storage device 120 is, for example, a DRAM, and the nonvolatile storage device 130 is, for example, a flash memory. The nonvolatile storage device 130 stores a program 132. The CPU 110 executes the program 132 to cause the scanner 400 to read a document and generate output image data representing the read document (details will be described later). The program 132 may be a scanner driver for controlling the scanner 400, for example. The CPU 110 temporarily stores various intermediate data used for executing the program 132 in a storage device (for example, one of the volatile storage device 120 and the nonvolatile storage device 130). The program 132 is provided by the manufacturer of the scanner 400, for example.

  The display unit 140 is a device that displays an image, and is, for example, a liquid crystal display. Instead of this, other types of devices that display images, such as LED displays and organic EL displays, may be employed. The operation unit 150 is a device that receives an operation by a user, and is, for example, a touch panel arranged on the display unit 140. Alternatively, other types of devices operated by the user, such as buttons and levers, may be employed. The user can input various instructions to the image processing apparatus 100 by operating the operation unit 150.

  The interface 190 is an interface for communicating with other devices (for example, a USB interface, a wired LAN interface, an IEEE802.11 wireless interface). In the present embodiment, the interface 190 is a wired or wireless network interface and is connected to the network NT.

  A perspective view of the scanner 400 is shown in FIG. In the figure, a first direction Dp1 and a second direction Dp2 indicate horizontal directions, and a third direction Dp3 indicates a vertically upward direction. The first direction Dp1 and the second direction Dp2 are perpendicular to each other. The third direction Dp3 is also referred to as an upward direction Dp3.

  The scanner 400 is a so-called flatbed image reading apparatus. The scanner 400 includes a main body 490 and a cover 450 attached to the upper direction Dp3 side of the main body 490 so as to be openable and closable. FIG. 1 shows the scanner 400 with the cover 450 opened in the upward direction Dp3.

  The main body 490 includes a control unit 410, an image sensor 420 that optically reads a document, a moving device 430 that moves the image sensor 420 in parallel to the first direction Dp1, and a document table 440.

  The document table 440 is provided on the upper direction Dp3 side of the main body 490 (FIG. 1). The document table 440 appears by opening the cover 450 in the upward direction Dp3. The document table 440 is a substantially rectangular table surrounded by two sides parallel to the first direction Dp1 and two sides parallel to the second direction Dp2, and is configured using a transparent plate (for example, a glass plate). Yes. The surface on the third direction Dp3 side of the document table 440 is a placement surface Us on which a document to be read is placed.

  The image sensor 420 is disposed below the document table 440 (on the opposite side of the third direction Dp3), that is, below the document table 440. The image sensor 420 is a one-dimensional image sensor that optically reads a document, and has a configuration in which a plurality of photoelectric conversion elements (also simply referred to as optical elements) such as a CCD and a CMOS are arranged in the second direction Dp2. Have. The image sensor 420 optically reads the document on the document table 440 and outputs a signal representing the read document.

  The moving device 430 includes a power source (for example, an electric motor). Using the power source, the moving device 430 moves the image sensor 420 in a direction along the placement surface Us of the document table 440 (a direction parallel to the first direction Dp1 in FIG. 1).

  The control unit 410 is a computer including, for example, a CPU 411 that performs data processing, a volatile storage device 412 (for example, a DRAM), a nonvolatile storage device 413 (for example, a flash memory), and an interface 419. Each element of the control unit 410 is connected to each other via a bus. The nonvolatile storage device 413 stores a program 414 in advance. The CPU 411 controls the image sensor 420 and the moving device 430 by executing the program 414. The interface 419 is an interface for communicating with other devices (for example, USB interface, wired LAN interface, IEEE802.11 wireless interface). In this embodiment, the interface 419 is a wired or wireless network interface, and is connected to the network NT.

  The CPU 411 controls the moving device 430 to move the image sensor 420 along the document table 440 from the position on the opposite side of the document table 440 to the first direction Dp1 to the position on the first direction Dp1 side. To move. The CPU 411 controls the image sensor 420 while moving the image sensor 420 so as to optically read the document placed on the document table 440. The CPU 411 generates read image data indicating a read image based on a signal output from the image sensor 420. The CPU 411 transmits the generated read image data to the image processing apparatus 100.

A-2. Overview of Image Processing FIG. 2 is a flowchart of image processing according to this embodiment. In this embodiment, the CPU 110 of the image processing apparatus 100 executes image processing as a scanner driver. In this image processing, read image data is acquired by causing the scanner 400 to read a document, and a repair that represents a repaired image (described later) that has been subjected to tilt correction, missing correction, or the like using the acquired read image data. Completed image data is generated. The user inputs an instruction to start image processing by operating the operation unit 150. CPU110 starts the process of FIG. 2 according to the input start instruction.

  In S10, the CPU 110 acquires read image data. Specifically, CPU 110 transmits an instruction to read a document to scanner 400. The CPU 411 of the control unit 410 of the scanner 400 generates read image data by reading a document placed on the document table 440 in response to an instruction from the image processing apparatus 100, and generates the read image data as an image. It transmits to the processing apparatus 100. CPU 110 acquires read image data transmitted from scanner 400. The data format of the read image data may be various formats. In this embodiment, the read image data is RGB image data including RGB values of a plurality of pixels or monochrome image data including luminance values of a plurality of pixels. The RGB value of one pixel indicates the color of the pixel. For example, three component values of red (R), green (G), and blue (B) (hereinafter, R value, G value, B value) Also called). In this embodiment, the number of gradations of each component value is 256 gradations. The luminance value is a value indicating the luminance of the pixel by, for example, a value of 256 gradations.

  FIG. 3 is a diagram illustrating an example of an image used in image processing. FIG. 3A shows an example of the read image OI. The read image OI has a rectangular shape.

  The read image OI is an image generated by reading a rectangular document. The read image OI includes a document image Ad indicating a document and an outer image Ab of the document image Ad. The outside image Ab indicates a region other than the document outside the document image Ad. The outer image Ab is an image located outside the original image Ad, that is, an image located between the outer edge of the original image Ad and the outer edge (four sides) of the read image OI. In the present embodiment, the outer image Ab shows a facing surface Bs (FIG. 1) that faces the document table 440 of the cover 450. In this embodiment, the facing surface Bs is painted in gray that is darker than white and lighter than black. Therefore, the outer image Ab is a gray substantially single-color image.

  The shape of the document image Ad is an incomplete substantially rectangular shape lacking a region Da corresponding to the upper right corner of the document rectangle and a region Db corresponding to the lower left corner. The lack of the upper right area Da is caused by reading the document in a state where the corresponding part of the document protrudes from the readable area on the document table 440. The lack of the lower left area Db is caused by reading the document on the document table 440 with the corresponding corner of the document folded. The substantially rectangular shape of the document image Ad is inclined with respect to the rectangular shape of the outer shape of the read image OI. This inclination is caused by the reading of the original on the original table 440 with the original inclined.

  The document image Ad includes objects Ob1 and Ob2 such as characters and drawings, and a background Bg. The background Bg is a background color area of the document.

  In S15, the CPU 110 executes a known reduction process on the read image data to generate reduced image data indicating the reduced image MI. For example, the reduced image MI has a number of pixels equivalent to 100 dpi vertical (dot per inch) × 100 dpi horizontal. When the vertical and horizontal resolutions of the read image data are 100 dpi or less, the process of S15 is omitted. FIG. 3A can also be said to be a diagram showing the reduced image MI.

  In S20, the CPU 110 executes document pixel specifying processing using the reduced image data. The document pixel specifying process is a process of specifying a plurality of document pixels PXa constituting the document image Ad in the reduced image MI and generating document pixel specifying image data indicating the document pixel specifying image CI. Since each pixel in the reduced image MI corresponds to one or more pixels in the read image OI, specifying the document pixel PXa in the reduced image MI specifies the corresponding document pixel in the read image OI. Is equivalent to Since the document area specifying process (for example, generation of simple binary image data or edge binary image data described later) is performed using the reduced image data, for example, pixels to be processed as compared with the case of using the read image data The number can be reduced. Accordingly, the processing time and memory capacity required for the document area specifying process can be reduced.

  FIG. 3B shows an example of the document pixel specifying image CI. The document pixel specifying image CI is an image composed of a plurality of pixels corresponding one-to-one with a plurality of pixels of the reduced image MI. The original pixel specific image data is binary data, and the value of each pixel of the original pixel specific image data is a non-original pixel that is different from the original pixel PXa1 in that the corresponding pixel of the reduced image MI is the original pixel PXa. Indicates whether it is PXb. Details of the document pixel specifying process will be described later. In FIG. 3B, a black region is a region where a plurality of document pixels PXa is located, and a white region is a region where non-document pixels PXb are located.

  In S25, the CPU 110 executes search range determination processing using the document pixel specifying image data. The search range determination process is a process for determining a search range SA used in the contour specifying process described later in the document pixel specifying image CI. As described above, since the document pixel specifying image CI corresponds to the read image OI and the reduced image MI, determining the search range SA in the document pixel specifying image CI is not limited to the read image OI and the reduced image. Equivalent to determining the corresponding search range in MI. In addition, since the search range determination process is performed using the document pixel specific image data generated using the reduced image data, for example, compared to the case where the document specific image data having the same number of pixels as the read image data is used, The number of pixels to be processed can be reduced. Therefore, the processing time and memory capacity required for the search range determination process can be reduced.

  FIG. 3B shows the search range SA determined in the document pixel specifying image CI. The search range SA is a rectangular range having four ends TU, TB, TL, TR. Details of the search range determination process will be described later.

  In S30, the CPU 110 executes a contour specifying process. In the contour specifying process, the contour of the document image Ad is identified by specifying a plurality of contour pixels PXo constituting the contour of the document image from the plurality of document pixels PXa within the determined search range SA. It is a process which produces | generates the contour image data which shows the contour image OLI to show.

  FIG. 3C shows an example of the contour image OLI. In this example, a plurality of contour pixels PXo constituting the contour of the document image Ad indicated by the six straight lines L1 to L6 are specified in the contour image OLI. The contour image OLI is an image composed of a plurality of pixels corresponding one-to-one with a plurality of pixels of the reduced image MI. The contour image data is binary data, and the value of each pixel of the contour image data indicates whether or not the corresponding pixel of the reduced image MI is a document constituent pixel. Since each pixel in the reduced image MI corresponds to one or more pixels in the read image OI, specifying the contour pixel PXo in the reduced image MI specifies the corresponding contour pixel in the read image OI. Is equivalent to Details of the contour specifying process will be described later.

  In S35, the CPU 110 is a process for specifying the document area AA corresponding to the document image Ad using the contour image data. As shown in FIG. 3A, the document area AA is a rectangular area obtained by supplementing the above-described areas Da and Db lacking in the document image Ad.

  In S40, the CPU 110 executes a repair process on the read image data using the result of specifying the document area AA to generate repaired image data indicating the repaired image RI. FIG. 3D shows an example of the repaired image RI. The repaired image RI includes a repaired document image Adr and a repaired outer image Abr. The document image Adr has a rectangle corresponding to the document area AA and is not inclined with respect to the outer rectangle of the repaired image RI. The outer image Abr outside the document image Adr is painted with a designated color (for example, white) designated in advance. Details of the repair process will be described later.

  In S45, the CPU 110 outputs the repaired image data. In this embodiment, the CPU 110 stores the repaired image data in a storage device (for example, the nonvolatile storage device 130) inside the image processing apparatus 100. Instead, the CPU 110 may store the repaired image data in an external storage device (for example, a USB memory not shown) connected to the image processing apparatus 100. Further, the CPU 110 may output the repaired image data in such a manner that the repaired image data is transmitted to the printer, and the repaired image RI is printed by the printer. An arbitrary format can be adopted as the data format of the repaired image data. In this embodiment, the CPU 110 outputs repaired image data in JPEG format.

  According to the image processing described above, repaired image data indicating a repaired image RI indicating a document is output using the read image data. As a result, it is possible to provide the user with image data that appropriately indicates the document.

A-3. Document Pixel Identification Process The document pixel identification process in S20 of FIG. 2 will be described. As described above, the document pixel specifying process is a process of generating document pixel specifying image data indicating the document pixel specifying image CI in FIG. FIG. 4 is a flowchart of the document pixel specifying process. FIG. 5 is a diagram illustrating an example of an image used in the document pixel specifying process.

  In S110 of FIG. 4, the CPU 110 determines whether the reduced image data is monochrome image data or color image data. For example, monochrome image data is image data including only one component value (for example, luminance), and color image data is image data including a plurality of component values (for example, R, G, B). . As described above, in the present embodiment, the read image data is either RGB image data that is color image data or monochrome image data. Therefore, the reduced image data is any one of these. It is. When the reduced image data is monochrome image data (S110: YES), in S120, the CPU 110 binarizes the reduced image data based on the luminance Y and displays the simple binary image BI. Value image data is generated. Specifically, the CPU 110 specifies a plurality of pixels having luminance within the specific luminance range YR among the plurality of pixels in the reduced image MI as the plurality of first pixels PX1. The CPU 110 identifies a plurality of pixels having luminance outside the specific luminance range YR among the plurality of pixels in the reduced image MI as the plurality of second pixels PX2. In the simple binary image data, the value of the first pixel PX1 is “0”, and the value of the second pixel PX2 is “1”.

  The specific luminance range YR is a predetermined luminance Y range. The specific luminance range YR is a range centered on the luminance Ybs of the color of the facing surface Bs (FIG. 1) of the cover 450. The specific luminance range YR is a range of luminance Y that satisfies (Ybs−ΔY) ≦ Y ≦ (Ybs + ΔY). For example, ΔY is 32 when the luminance Y is expressed by 256 gradations of 0 to 255. White (luminance 255) and black (luminance 0) are outside the specific luminance range YR.

  When the reduced image data is color image data (S110: NO), in S115, the CPU 110 binarizes the reduced image data based on the luminance Y and the saturation C, thereby obtaining a simple binary image. Generate data. Specifically, the CPU 110 sets a plurality of pixels having the luminance Y in the specific luminance range YR and the saturation C in the specific saturation range CR as the plurality of first pixels PX1. Identify. The CPU 110 identifies a plurality of pixels having luminance Y outside the specific luminance range YR and a plurality of pixels having saturation C outside the specific saturation range CR as a plurality of second pixels PX2. As a result, simple binary image data can be appropriately generated using the specific luminance range YR and the specific saturation range CR. For example, even if the brightness of the background Bg of the document image Ad is similar to the brightness Ybs of the facing surface Bs of the cover 450, the saturation of the background Bg of the document image Ad and the saturation of the facing surface Bs of the cover 450 are similar. If Cbs is sufficiently different, simple binary image data indicating a simple binary image BI in which a boundary between the document image Ad and the outer image Ab appears is generated.

  The specific saturation range CR is a predetermined saturation C range. The specific saturation range CR is a range including the saturation Cbs of the color of the facing surface Bs of the cover 450. In the present embodiment, since the color of the facing surface Bs is an achromatic color (saturation Cbs = 0), the specific saturation range CR is a range of saturation C that satisfies C ≦ Cth. The threshold value Cth is 32, for example, when the saturation C is expressed by 256 gradations of 0 to 255.

  FIG. 5A shows a simple binary image BIs as an example of the simple binary image BI. In FIG. 5A, a black portion is a portion constituted by the first pixel PX1, and a white portion is a portion constituted by the second pixel PX2. As shown in FIG. 5A, the pixels constituting the outer image Ab of the reduced image MI have a luminance within the specific luminance range YR and have a saturation within the specific saturation range CR. For this reason, the pixels constituting the outer image Ab are specified as the first pixel PX1. As described above, the specific luminance range YR is a range of the luminance Y including the luminances (for example, luminance Ybs) of a plurality of pixels in the outer image Ab, and the specific saturation range CR is a plurality of specific luminance ranges CR in the outer image Ab. It can be said that it is the range of the saturation C including the saturation (for example, the saturation Cbs) of the pixels.

  The document image Ad may include a portion having a color that is relatively different from the color of the facing surface Bs of the cover 450. Of the original image Ad of the reduced image MI, the pixel constituting the portion is specified as the second pixel PX2. The document image Ad may include a portion having a color that approximates the color of the facing surface Bs of the cover 450. Of the original image Ad of the reduced image MI, the pixel constituting the portion is specified as the first pixel PX1.

  For example, the simple binary image BIs in FIG. 5A is an example in which the color of the background Bg of the document image Ad is relatively different from the color of the facing surface Bs of the cover 450. In this example, in the reduced image MI (FIG. 3A), the pixels constituting the entire background Bg, the entire object Ob1, and a part of the object Ob2 are specified as the second pixel PX2. Yes. In the reduced image MI, the pixels constituting the entire outer image Ab and the other part of the object Ob2 are specified as the first pixel PX1. In the example of FIG. 5A, it can be seen that the boundary between the document image Ad and the outer image Ab appears in the simple binary image BIs.

  FIG. 5B shows a simple binary image BIm as another example of the simple binary image BI. This simple binary image BIm is an example in the case where the color of the background Bg of the document image Ad approximates the color of the facing surface Bs of the cover 450. In this example, in the reduced image MI, the pixels constituting the entire object Ob1 and a part of the object Ob2 are specified as the second pixel PX2. In the reduced image MI, the pixels constituting the entire outer image Ab, the entire background Bg, and the other part of the object Ob2 are specified as the first pixel PX1. In the example of FIG. 5B, it can be seen that the boundary between the document image Ad and the outer image Ab does not appear in the simple binary image BIm.

  In S125, the CPU 110 calculates the number M1 of the first pixels PX1 and the number M2 of the second pixels PX2. In S130, the CPU 110 determines whether or not the number M2 of the second pixels PX2 is larger than the number M1 of the first pixels PX1, that is, whether (M1 <M2) is satisfied. As in the case of the simple binary image BIs in FIG. 5A, when the appropriate binarization in which the boundary between the document image Ad and the outer image Ab appears is successful, a relatively large number of second pixels PX2 exist. Therefore, the number M2 of the second pixels PX2 is larger than the number M1 of the first pixels PX1. On the other hand, when the boundary between the original image Ad and the outer image Ab does not appear as in the simple binary image BIm in FIG. 5B, that is, when the appropriate binarization fails, the comparison is performed. Since only a small number of second pixels PX2 are specified, the number M2 of second pixels PX2 is equal to or less than the number M1 of first pixels PX1.

  When the number M2 of the second pixels PX2 is larger than the number M1 of the first pixels PX1 (S130: YES), the CPU 110 skips the processes of S135 to S145 described later and advances the process to S150. When the second pixel PX2 is equal to or smaller than the first pixel PX1 (S130: NO), the CPU 110 executes the processes of S135 to S145.

  In S135, the CPU 110 generates a histogram indicating the color distribution of the reduced image MI by classifying each pixel of the reduced image MI into a plurality of classes according to the color (luminance and saturation) of the pixel. To do. Specifically, a luminance histogram and a saturation histogram are generated, respectively.

  FIG. 6 is a diagram illustrating an example of a histogram of the reduced image MI. In the luminance histogram of FIG. 6A, a peak of luminance Ybs corresponding to the outer image Ab of the reduced image MI and a peak of luminance Ym corresponding to the background Bg of the document image Ad appear. This histogram is an example when the color of the background Bg of the document image Ad is close to the color of the facing surface Bs of the cover 450. For this reason, in this histogram, the peak of the luminance Ybs corresponding to the outer image Ab and the peak of the luminance Ym corresponding to the background Bg of the document image Ad are relatively close. As a result, as shown in FIG. 6A, binarization (S115, S120) using a predetermined specific luminance range YR cannot be performed appropriately, and for example, FIG. Simple binary image data indicating the simple binary image BIm of B) is generated.

  Similarly, the saturation histogram of FIG. 6B includes a peak of saturation Cbs corresponding to the outer image Ab of the reduced image MI and a peak of saturation Cm corresponding to the background Bg of the document image Ad. Appears. In this histogram, the peak of saturation Cbs corresponding to the outer image Ab and the peak of saturation Cm corresponding to the background Bg of the document image Ad are relatively close. As a result, as shown in FIG. 6B, binarization (S115) using a predetermined specific saturation range CR cannot be appropriately binarized.

  In S140, the CPU 110 determines a new binarization threshold, specifically, a new specific luminance range YRa and a specific saturation range CRa based on these histograms. If the reduced image data is monochrome image data, only the specific luminance range YRa is determined.

The specific luminance range YRa is determined as follows, for example. The CPU 110 specifies the luminance Ym corresponding to the background Bg of the document image Ad by specifying the peak of the histogram in FIG. CPU 110 determines Yth and ΔYa according to the following equations (1) and (2) so that the central value of luminance Ybs and luminance Ym is equal to threshold value Yth.
Yth = {(Ybs + Ym) / 2)) (1)
ΔYa = (Yth−Ybs) (2)
Then, based on the new ΔYa, the CPU 110 determines a range of luminance Y that satisfies (Ybs−ΔY) ≦ Y ≦ (Ybs + ΔY) as a new specific luminance range YRa.

  The specific saturation range CRa is determined as follows, for example. The CPU 110 specifies the saturation Cm corresponding to the background Bg of the document image Ad by specifying the peak of the histogram in FIG. The CPU 110 determines the central value of the saturation Cbs and the saturation Cm as a new threshold value Ctha. The CPU 110 determines a saturation C range that satisfies Y ≦ Ctha as a new specific saturation range CRa.

  In S145, the CPU 110 binarizes the reduced image data using the determined specific luminance range YRa and specific saturation range CRa, and again generates simple binary image data indicating the simple binary image BI. . Thus, even if the simple binary image BI indicated by the simple binary image data generated first in S115 or S120 is an inappropriate simple binary image BIm shown in FIG. 5B. The simple binary image BI indicated by the simple binary image data generated for the second time in this step can be highly likely to be an appropriate simple binary image BIs shown in FIG. .

  In S150, the CPU 110 performs edge extraction processing for extracting edges in the reduced image MI on the reduced image data, and generates edge extraction data. Specifically, the CPU 110 calculates the edge intensity Se by applying a so-called Sobel filter to the value of each pixel of the reduced image data. The CPU 110 generates edge extraction data having these edge strengths Se as values of a plurality of pixels.

  The calculation formula (3) of edge strength is shown below. The gradation value P (x, y) in Expression (3) represents the gradation value at a specific pixel position (x, y) in the scan image SI. A position x indicates a pixel position in the direction D1 (vertical direction), and a position y indicates a pixel position in the direction D2 (horizontal direction). The edge intensity Se (x, y) at the pixel position (x, y) in the scan image SI is calculated using the values of nine pixels in 3 rows and 3 columns adjacent to the pixel position (x, y). Is done. The first and second terms of the calculation formula are absolute values of the sum of values obtained by multiplying the gradation values of the pixels at nine positions by the corresponding coefficients. The first term is the differentiation of the gradation value in the direction D1 (that is, the differentiation in the horizontal direction), and the second term is the differentiation of the gradation value in the direction D2 (that is, the differentiation in the vertical direction). The calculated edge strength Se (x, y) is normalized to a value of 256 gradations in the range of 0 to 255.

…（２）

... (2)

  In the edge extraction image indicated by the generated edge extraction data, the position corresponding to the edge in the reduced image MI, for example, the pixel corresponding to the boundary between the document image Ad and the outer image Ab in the reduced image MI in FIG. Or, the value of the pixel corresponding to the contours of the objects Ob1 and Ob2 is larger than the values of the pixels at other positions.

  In S155, the CPU 110 binarizes the edge extraction data generated in S150, and generates edge binary image data indicating the edge binary image EI. For example, in the edge extraction data, the CPU 110 identifies a pixel having a pixel value (that is, edge strength) equal to or greater than a threshold value (for example, 128) as the edge pixel PXe, and a pixel having a pixel value less than the threshold value. This is specified as a non-edge pixel PXn. In the edge binary image data, the value of the edge pixel PXe is “1”, and the value of the non-edge pixel PXn is “0”.

  FIG. 5C shows an example of the edge binary image EI. In this edge binary image EI, it is understood that pixels corresponding to the boundary between the document image Ad and the outer image Ab in the reduced image MI and pixels corresponding to the contours of the objects Ob1 and Ob2 are specified as edge pixels PXe. . Since each pixel in the reduced image MI corresponds to one or more pixels in the read image OI, specifying the edge pixel PXe and the non-edge pixel PXn in the reduced image MI is not necessary in the read image OI. Is equivalent to identifying the corresponding edge and non-edge pixels.

  In S160, the CPU 110 synthesizes the two binary image data, that is, the simple binary image data and the edge binary image data, and the above-described original pixel specifying image CI (FIG. 3B, FIG. 5). D)) original pixel specific image data is generated. Specifically, the CPU 110 generates document pixel specifying image data by calculating the logical sum of each pixel of simple binary image data and edge binary image data. As a result, a pixel group including a plurality of second pixels PX2 specified using simple binary image data and a plurality of edge pixels PXe specified using edge binary image data, A pixel group not including pixels different from the second pixel PX2 and the edge pixel PXe is specified as a plurality of document pixels PXa. A pixel group different from the second pixel PX2 and the edge pixel PXe is specified as a plurality of non-original pixels PXb.

  As can be understood from the above description, in the original pixel specifying process of this embodiment, the reduced image data generated using the read image data is used to have a luminance within the specific luminance range YR (or the specific luminance range YRa). A plurality of pixels that satisfy a specific condition including the above are specified as a plurality of first pixels PX1, and a plurality of pixels that do not satisfy the specific condition are specified as a plurality of second pixels PX2. Binary image data is generated (S110 to S145 in FIG. 4). Further, edge binary image data indicating the edge pixel PXe is generated using the reduced image data (S150 to S155 in FIG. 4). In the image processing of this embodiment, the document area AA (FIG. 3C) corresponding to the document image Ad in the read image OI is specified using the simple binary image data and the edge binary image data. (S160 in FIG. 4, S25 to S35 in FIG. 2). As a result, the accuracy of specifying the document area AA in the read image OI can be improved.

  Assume that the document area AA is to be specified using only the edge binary image data. In the edge binary image EI indicated by the edge binary image data, as shown in FIG. 5C, the edge pixel PXe constituting the line indicating the outline of the document image Ad is specified. However, in the edge binary image EI, a broken portion is likely to occur in a thin line such as a line indicating the outline of the document image Ad. That is, the plurality of edge pixels PXe specified by using the edge binary image data do not include a part of the plurality of contour pixels PXo (FIG. 3C) constituting the contour of the document image Ad. there is a possibility. In this case, in the contour specifying process in S30 of FIG. 2, the contour of a part of the document image Ad cannot be specified, and as a result, the document area AA may not be specified appropriately. In the original pixel specifying image CI (FIG. 5D) generated by the original pixel specifying process of the present embodiment, not only the fine line indicating the outline of the original image Ad but also the background Bg of the original original image Ad is formed. The pixel is also specified as the document pixel PXa. Therefore, in this embodiment, it is possible to specify the document area AA with higher accuracy than when attempting to specify the document area AA using only the edge binary image data.

  Consider a case where the original area AA is to be specified using only simple binary image data. In the simple binary image BI indicated by the simple binary image data, in particular, when the color of the background Bg of the document image Ad approximates the color of the facing surface Bs of the cover 450, FIG. Like the simple binary image BIs shown in FIG. 5, there is a possibility that some or all of the plurality of contour pixels PXo constituting the contour of the document image Ad are not specified as the second pixel PX2. In this case, in the contour specifying process in S30 of FIG. 2, the contour of a part of the document image Ad cannot be specified, and as a result, the document area AA may not be specified appropriately. In the document pixel specifying process of the present embodiment, for example, even if some or all of the plurality of contour pixels PXo are not specified as the second pixel PX2 in the simple binary image BI, the edge binary image EI is used. If the plurality of contour pixels PXo are specified as the edge pixels PXe, the plurality of contour pixels PXo are included in the document pixel PXa specified by the document pixel specifying image CI. Therefore, in this embodiment, it is possible to specify the document area AA with higher accuracy than when attempting to specify the document area AA using only simple binary image data.

  More specifically, in the document pixel specifying process, as described above, document pixel specifying image data is generated by taking the logical sum of each pixel of simple binary image data and edge binary image data. (S160 in FIG. 4). In other words, the CPU 110 generates composite image data indicating a pixel group including a plurality of second pixels PX2 and a plurality of edge pixels PXe as document pixel specifying image data. As a result, the accuracy of specifying the document area AA in the read image OI can be further improved using the document pixel specifying image data.

  Further, in the document area specifying process, the number M2 of the plurality of second pixels PX2 of the plurality of second pixels specified by using the specific luminance range YR is less than the reference (specifically, the first pixel If the number of pixels PX1 is equal to or less than M1 (S130 in FIG. 4: NO), the CPU 110 uses a specific luminance range YRa different from the specific luminance range YR, and uses a plurality of first pixels PX1 and a plurality of first pixels. By specifying the two pixels PX2 again, simple binary image data is generated again (S145 in FIG. 4). As described above, when the number M2 of the second pixels PX2 is equal to or less than the reference, there is a high possibility that the plurality of first pixels PX1 and the plurality of second pixels PX2 are not properly specified ( For example, FIG. 5 (B)). According to the above configuration, in such a case, the plurality of first pixels PX1 and the plurality of second pixels PX2 are specified again using the specific luminance range YRa. It can be generated more appropriately. In the first time, simple binary image data is generated using a predetermined specific luminance range YR. If it can be determined that the first simple binary image data is appropriate, a histogram is displayed. It is not necessary to determine the used specific luminance range YRa. As a result, the processing time required to generate simple binary image data can be reduced.

  As can be understood from the above description, the simple binary image data in the document pixel specifying process is an example of the first image data, and the edge binary image data is an example of the second image data, and the specific luminance range YR. Is an example of the first specific luminance range, and the specific luminance range YRa is an example of the second specific luminance range.

A-4. Search Range Determination Process The search range determination process in S25 of FIG. 2 will be described. As described above, the search range determination process is a process of determining the search range SA (FIG. 3B) in the document pixel specific image CI using the document pixel specific image data. FIG. 7 is a flowchart of the search range determination process. FIG. 8 is an explanatory diagram of the search range determination process. FIG. 8 shows a document pixel specifying image CI. In FIG. 8, for the sake of easy viewing, the area painted in black in FIG. 5D is indicated by hatching.

  In S205 of FIG. 7, the CPU 110 selects one target end from the four ends corresponding to the four sides of the outer shape of the document pixel specifying image CI, that is, the upper end UE, the lower end BE, the left end LE, and the right end RE. select.

  In S210, the CPU 110 sets a position away from the target end by a specific amount ΔW as a search position. For example, when the upper end UE in FIG. 8 is the target end, the position in the direction D1 away from the upper end UE by the specific amount ΔW, that is, the position on the line P1U in FIG. 8 is set as the search position. Similarly, when the lower end BE is the target end, the position in the direction D1 away from the lower end BE by the specific amount ΔW, that is, the position on the line P1B is set as the search position. When the right end RE and the left end LE are the target ends, the lower end BE, the right end RE, and the position in the direction D2 away from the left end LE by ΔW, that is, the positions on the lines P1B and PIL are the search positions, respectively. Set to The specific amount ΔW is a preset amount, for example, the number of pixels corresponding to a length of 3 mm in the document.

  In S215, the CPU 110 determines whether there are M or more continuous document pixels PXa in the direction along the target end at the set search position. M is an integer of 2 or more, and is 20, for example. For example, when the upper end UE in FIG. 8 is the target end, it is determined whether or not M or more document pixels PXa are continuously arranged on the line P1U that is the search position. In the example of FIG. 8, when the upper end UE is the target end, it is determined that M or more document pixels PXa are arranged at the search position (position on the line P1U). For example, when the left end LE in FIG. 8 is the target end, it is determined whether or not M or more document pixels PXa are continuously arranged on the line P1L that is the search position. In the example of FIG. 8, when the left end LE is the target end, it is determined that M or more document pixels PXa are not arranged at the search position (position on the line P1L).

  When there are no M or more consecutive original pixels PXa at the search position (S215: NO), the CPU 110 performs a search while shifting the search position by one pixel toward the direction away from the target end. Specifically, the direction away from the target end is a direction perpendicular to the target end and toward the other end of the document pixel specifying image CI facing the target end.

  Specifically, in S220, CPU 110 sets the search position to a position away from the current end by one pixel from the current position. In S225, the CPU 110 determines whether or not there are M or more consecutive document pixels PXa in the direction along the target end at the new search position. If there are no M or more consecutive original pixels PXa at the search position (S225: NO), the CPU 110 returns to S220 and sets the search position to a position further away from the target end by one pixel. When there are M or more consecutive original pixels PXa at the search position (S225: YES), in S230, the CPU 110 determines the current search position as the position of the end of the search range SA corresponding to the target end. Then, the process proceeds to S260.

  In the example of FIG. 8, when the target end is the left end LE, the first search position is a position on the line P1L (S210, S215). The search position is shifted rightward (direction D2) by one pixel from the position on the line P1L, and the search is repeated (S220, S225). For example, positions on the lines P2La, P2Lb, and P2Lc in FIG. 8 are sequentially set as search positions. Then, when the search position is the line P2Lc in FIG. 8, it is determined that there are M or more continuous document pixels PXa at the search position (S225: YES), and the position on the line P2Lc is the left end of the search range SA. The TL position is determined (S230).

  When the target end is the right end RE, the search is sequentially performed from the position on the line P1R in the left direction (the direction opposite to the direction D2), and the position of the right end TR in the search range SA illustrated in FIG. It is determined (S210-S230).

  In S215, when there are M or more consecutive document pixels PXa at the search position (S215: YES), the CPU 110 performs a search while shifting the search position by one pixel toward the target end.

  Specifically, in S235, CPU 110 sets the search position to a position on the attention end side by one pixel from the current position. In S240, the CPU 110 determines whether there are M or more consecutive document pixels PXa in the direction along the target end at the new search position. If there are no M or more consecutive original pixels PXa at the search position (S240: NO), in S245, the CPU 110 corresponds to the position that is one pixel away from the target end from the current search position as the target end. The end position of the search range SA to be determined is determined, and the process proceeds to S260.

  When there are M or more consecutive original pixels PXa at the search position (S240: YES), in S250, the CPU 110 determines whether or not the current search position is the position of the target end. If the current search position is not the position of the target end (S240: NO), the CPU 110 returns to S235, and further sets the search position to the position of the target end side by one pixel. If the current search position is the position of the target end (S240: YES), in S255, the CPU 110 determines the position of the target end as the position of the end of the search range SA corresponding to the target end, The process proceeds to S260.

  In the example of FIG. 8, when the target end is the upper end UE, for example, the first search position is a position on the line P1U (S210, S215). The search position is shifted by one pixel upward from the position on the line P1U (the direction opposite to the direction D1), and the search is repeated (S235, S240, S250). For example, positions on the lines P3Ua, P3Ub, and P3Uc in FIG. 8 are sequentially set as search positions. Since the position of the line P3Uc is the position of the upper end UE when the search position is the line P3Uc in FIG. 8 (S250: YES), the position of the upper end UE of the document pixel specific image CI is within the search range SA. Is determined as the position of the upper end TU of (S255).

  When the target end is the lower end BE, the position on the line P1B and the positions on the lines P3Ba, P3Bb, and P3Bc are sequentially set as the search positions, and the search is performed (S235, S240, and S250). Then, when the search position is a line P3Bc one pixel lower than the position of the lower end TB of the search range SA shown in FIG. 8, it is determined that there are no M or more consecutive original pixels PXa at the search position. (S240: NO), a position one pixel above the line P3Bc is determined as the lower end TB of the search range SA (S245).

  In S260, the CPU 110 determines whether or not all the terminals UE, BE, LE, and RE have been processed as the target ends. If there is an unprocessed end (S260: NO), the CPU 110 returns the process to S205. If all ends have been processed (S260: YES), in S265, CPU 110 generates range information that defines search range SA, and ends the search range determination process. The range information is, for example, the distances DU, DB between the four ends UE, BE, LE, RE of the document pixel specific image CI and the corresponding ends TU, TB, TL, TR of the search range SA. , DL and DR.

  In the search range determination process of the present embodiment described above, for example, when the target end is the left end LE, the position on the line P1L that is separated from the left end LE by the specific amount ΔW continues in the direction along the left end LE. It is determined whether or not there are M document pixels PXa (S210, S215). When a plurality of document pixels PXa does not exist at the position on the line P1L (S215: NO), the plurality of document pixels PXa are sequentially arranged in the direction away from the specific end with respect to the positions on the lines P2La to P2Lc. Is determined (S220). If it is determined that there are a plurality of document pixels PXa at the position on the line P2Lc (S225: YES), the left end TL of the search range SA corresponding to the end of the left end LE based on the position on the line P2Lc. Is determined (S230). In the image processing of this embodiment, a plurality of contour pixels PXo constituting the contour of the document image Ad are specified within the search range SA determined in this way (S30 in FIG. 2).

  Shadows are likely to occur at the end of the read image OI due to the non-uniformity of light irradiation during reading. In particular, in a flat-bed type reading device such as the scanner 400, light is reflected from the side end surface portion of a glass document table 440 (also referred to as platen glass), or the document is lifted from the document table 440. Such a shadow is likely to occur because the amount of light applied to the edge of the document tends to decrease. In such a shadowed portion, the pixel density in the read image OI is high, so that even if the original pixel PXa does not exist in the portion, the original pixel PXa is erroneously specified. There is a case. According to the search range determination process of the above embodiment, as described above, it is determined whether or not there are M document pixels PXa at a position on the line P1L far from the left end LE (S210, S215). When M document pixels PXa are not present at positions on the line P1L (S215: NO), it is determined whether or not M document pixels PXa are present sequentially at positions on the lines P2La to P2Lc. (S220). As a result, when the document image Ad is at a position away from the left end LE as in the examples of FIGS. 3 and 8, for example, the left end TL of the search range SA is the read image OI (document pixel specifying image CI). ) At a position away from the left end LE. Then, a plurality of contour pixels PXo are specified within the search range SA determined in this way. Accordingly, since a shadow is generated in the vicinity of the left end LE of the read image OI, a plurality of document pixels PXa are erroneously present even though the plurality of document pixels PXa are not present in the vicinity of the left end LE. Even in such a case, the contour of the document image Ad can be specified with high accuracy.

  Further, in the search range determination process of the present embodiment, for example, when the target end is the lower end BE, is there any M document pixels PXa continuous in the direction along the lower end BE at the position on the line P1B? It is determined whether or not (S210, S215). When M document pixels PXa are present at the position on the line P1B (S215: YES), the positions on the plurality of lines P3Ba to P3Bc on the lower end BE side from the position on the line P1B are sequentially M. It is determined whether or not there are original document pixels PXa (S235, S240, S250). Then, the position of the lower end TB of the search range SA is determined based on the determination result about the positions on the plurality of lines P3Ba to P3Bc (S245, S255). Therefore, as shown in FIGS. 3 and 8, even when there are a plurality of document pixels PXa on the lower end BE side with respect to the position on the line P1B, an appropriate search range SA can be defined. The contour of the image Ad can be specified with higher accuracy.

  Further, in the search range determination process of the present embodiment, for example, when the target end is the lower end BE, the positions on the plurality of lines P3Ba to P3Bc are sequentially increased from the position on the line P1B toward the lower end BE. It is determined whether or not there are M document pixels PXa (S235, S240, S250). When it is determined that there are no M original pixels PXa at the position on the line P3Bc (S240: NO), the position of the lower end TB of the search range SA is determined based on the position on the line P3Bc. . As a result, for example, even when a shadow is generated at the lower end BE of the read image OI, the lower end BE of the search range SA can be appropriately determined between the lower end BE and the position on the line P1B. Therefore, the contour of the document image Ad can be specified with higher accuracy.

  Further, in the search range determination process of the present embodiment, the four ends UE, BE, LE, and RE corresponding to the four sides of the read image OI and the document pixel specifying image CI are used as the target ends, and S210 to S255. The four processes TU, TB, TL, TR of the search range SA corresponding to the four sides are determined (S205, S260). Therefore, since an appropriate search range can be defined for the rectangular read image OI, the contour of the document image Ad in the rectangular read image OI can be specified with high accuracy.

  Furthermore, in the present embodiment, the read image data is generated using the flatbed scanner 400, so that the outline of the document image Ad can be accurately identified in the read image in which a shadow is likely to occur at the end. .

  Further, in the present embodiment, the processes of S210 to S255 are executed using the document pixel specifying image data generated using the read image data. That is, by determining the search range SA on the document pixel specific image CI, the range on the read image OI corresponding to the search range on the document pixel specific image CI indirectly becomes the search range on the read image OI. As determined. Therefore, the search range can be determined more efficiently than the case where the search range is determined directly on the read image OI without generating document pixel specific image data.

  As can be understood from the above description, the positions in the read image OI corresponding to the positions of the lines P1U, P1B, P1R, and P1L in the document pixel specifying image CI in the search range determination process are examples of the first position. The positions in the read image OI corresponding to the positions of the lines P2La to P2Lc in the document pixel specific image CI are examples of the second position, and the lines P3Ua to P3Uc and P3Ba to P3Bc in the document pixel specific image CI are examples. The position in the read image OI corresponding to the position is an example of the third position. The document image Ad in the above embodiment is an example of the first image, and the outer image Ab is an example of the second image.

A-5. Outline Specifying Process The outline specifying process in S30 of FIG. 2 will be described. As described above, the contour specifying process is a process of specifying the contour pixel PXo within the search range SA using the document pixel specifying image data. FIG. 9 is a flowchart of the contour specifying process. FIG. 10 is an explanatory diagram of the contour specifying process.

  In S300 of FIG. 9, the CPU 110 prepares initial data of contour image data indicating the contour image OLI in the volatile storage device 120. The initial data is binary image data indicating an image having the same size as the reduced image MI and the original pixel specifying image CI, and the values of all the pixels are set to “0” as initial values.

  In S305, the CPU 110, from the four ends corresponding to the four sides of the outer shape of the search range SA, that is, the upper end TU, the lower end TB, the left end TL, and the right end TR (FIG. 10) in the document pixel specific image CI. One attention end is selected. In S305, the CPU 110 selects one target pixel from among a plurality of pixels on the target end.

  In S315, the CPU 110 searches from the target end to the inside of the search range SA using the target pixel as a starting point, and specifies the original pixel PXa to be searched first. Specifically, the direction toward the inside of the search range SA is a direction perpendicular to the target end and a direction toward the other end of the search range SA facing the target end. As shown in FIG. 10, when the pixel PXt1 positioned on the upper end TU is the target pixel, the search is performed downward, and the document pixel PXa1 to be searched first is specified. Similarly, when the pixels PXt2, PXt3, and PXt4 located on the right end TR, the lower end TB, and the left end TL are the target pixels, the search is performed toward the left side, the upper side, and the right side, respectively. Document pixels PXa2, PXa3, and PXa4 to be processed are specified.

  In S320, the CPU 110 records the specified document pixel PXa in the contour image data as the contour pixel PXo. Specifically, the value of the pixel in the contour image data corresponding to the specified document pixel PXa is changed to “1”.

  In S325, the CPU 110 determines whether all the pixels on the target end have been processed as the target pixel. When there is an unprocessed pixel (S325: NO), the CPU 110 returns the process to S305. When all the pixels have been processed (S325: YES), the CPU 110 advances the process to S330.

  In S335, the CPU 110 determines whether or not the ends TU, TB, TL, and TR of all the search ranges SA have been processed as attention ends. If there is an unprocessed end (S335: NO), the CPU 110 returns the process to S305. When all edges have been processed (S335: YES), the CPU 110 ends the contour specifying process. By the contour specifying process, contour image data indicating the contour image OLI shown in FIG. 3C is generated.

  According to the contour specifying process of the present embodiment, the original pixel PXa is searched from the end of the search range SA toward the inside of the search range SA, and the specific original pixel PXa (for example, PXa1 to PXa4) to be searched first is determined. , Identified as a contour pixel PXo. Therefore, the outline of the document image Ad can be easily and appropriately specified within the search range SA.

A-6. Document Area Specifying Process The document area specifying process in S35 of FIG. 2 will be described. As described above, the document area specifying process is a process for specifying the document area AA corresponding to the document image Ad using the contour image data. FIG. 11 is a flowchart of the document area specifying process.

  In S310 of FIG. 11, a plurality of candidate straight lines that are straight line candidates indicating the edge of the document are specified using the contour image data. Specifically, a straight line represented by the contour pixel PXo is adopted as a candidate straight line. As a method for specifying the candidate straight line, Hough transform using the contour pixel PXo is used. Since the Hough transform is a known process, its description is omitted. As a result of the Hough transform, in the example of FIG. 3C, six candidate straight lines L1 to L6 indicating the contour of the document image Ad are specified.

  In S320, the CPU 110 selects a straight line indicating the edge of the document from the six candidate straight lines L1 to L6. Specifically, the CPU 110 excludes the straight lines located at the four ends (sides) of the outline of the contour image OLI from the six candidate straight lines L1 to L6. For example, in the example of FIG. 3A, the candidate straight line L2 located at the upper end of the contour image OLI is excluded. Straight lines (for example, L2) positioned at the four ends of the contour image OLI are positioned at corresponding ends of the read image OI. As shown in FIG. 3A, such a straight line L2 corresponds to the area Da on the document that protrudes from the readable area at the time of reading, and is considered not to indicate the edge of the document. The CPU 110 extracts two sets of two straight lines parallel to each other from the remaining candidate straight lines L1, L3 to L6. In the example of FIG. 3A, a set of straight lines L3 and L6 and a set of straight lines L2 and L4 are extracted. These four straight lines L2, L3, L4, and L6 are selected as straight lines indicating the edge of the original.

  In S330, the CPU 110 identifies a rectangular internal area formed by a group of straight lines indicating the edge of the document as the document area AA. Specifically, by specifying the four intersection points CP1 to CP4 of the four straight lines L2, L3, L4, and L6 indicating the edge of the document, the document area AA defined by the four intersection points CP1 to CP4. Is identified.

  As can be understood from the above description, in the document area specifying process of the present embodiment, a straight line representing the edge of the document in the contour of the document image Ad is used using the contour image data generated using the document pixel specifying image data. A plurality of candidate straight lines L1 to L6 are identified (S310), and a document area AA defined using a part of these straight lines L2, L3, L4, and L6 is determined (S320, S330). ). Therefore, based on the document pixel specific image data, specifically, using the contour image data generated using the document pixel specific image data, an appropriate document area AA is defined from the contour of the specified document image Ad. Can be identified.

A-7. Repair Process The repair process in S40 of FIG. 2 will be described. As described above, the repair process is a process executed on the read image data using the result of specifying the document area AA. FIG. 12 is a flowchart of the repair process.

  In S410 of FIG. 12, an inclination correction process for correcting the inclination of the document image Ad is performed on the read image data. Specifically, the angle of the straight line L3 specified in the contour image OLI with respect to the vertical direction of the contour image OLI is calculated. Then, by rotating the read image OI by the angle, a rotation process is performed in which the contour line of the document image Ad in the read image OI corresponding to the straight line L3 is parallel to the vertical direction of the read image OI.

  In S420, the CPU 110 draws a missing portion of the document image Ad in the read image OI whose inclination has been corrected. Specifically, in the read image OI whose inclination has been corrected, an area outside the outline of the document image Ad specified using the outline image OLI and inside the document area AA, specifically, The areas Da and Db in FIG. 3A are specified. In the read image OI whose inclination has been corrected, the values of a plurality of pixels in the areas Da and Db are replaced with values indicating the color of the background Bg of the document image Ad. For the value indicating the color of the background Bg, for example, the average value of the values of a plurality of pixels located along the contour of the document image Ad specified using the contour image OLI and located in the document image Ad is used. .

  In S430, the CPU 110 changes the color of the outer image Ab to the designated color in the read image OI in which the missing part has been drawn. Specifically, in the read image OI whose inclination has been corrected, the values of a plurality of pixels located outside the contour of the document image Ad specified using the contour image OLI are in a predetermined color (for example, white Is replaced with a value indicating). As a result, repaired image data indicating the repaired image RI in FIG. 3D is generated.

B. Variation:
(1) In the above embodiment, document pixel specific image data is generated by taking the logical sum of simple binary image data and edge image data (S160 in FIG. 4). Instead of this, final original pixel specific image data may be generated by performing a logical sum of simple binary image data, edge image data, and another binary image data. As another binary image data, for example, binary image data of an R component obtained by binarizing the R value of each pixel included in the read image data with a predetermined threshold value can be used.

(2) In the above embodiment, when the number M2 of the second pixels PX2 is less than or equal to the number M1 of the first pixels PX1 (S130: NO in FIG. 4), the simple binary image data is generated again. (S135 to S145 in FIG. 4). The criterion for determining whether to generate simple binary image data again is not limited to this. For example, when the number M2 of the second pixels PX2 is 60% or less of the total number of pixels of the reduced image MI, the simple binary image data may be generated again.

(3) In the document area specifying process of FIG. 4, the document area AA is specified using a plurality of candidate straight lines L1 to L6 (S310 to S330 of FIG. 11), but is not limited thereto. For example, when the inclination of the document image Ad is difficult to occur, a circumscribed rectangle for the plurality of contour pixels PXo specified in the contour image OLI may be specified as the document area AA.

(4) The image processing step in FIG. 2 can be omitted as appropriate. For example, the generation of the reduced image data in S15 of FIG. 2 may be omitted. In this case, in the original pixel specifying process of S20, simple binary image data and edge binary image data having the same number of pixels as the read image data are generated using the read image data.

(5) The steps of the original pixel specifying process in FIG. 4 can be omitted as appropriate. For example, S110 and S115 in FIG. 4 may be omitted. In this case, regardless of whether the read image data is monochrome image data or color image data, simple binary processing is performed using only the specific luminance range YR in S120 without considering saturation. Image data is generated.

  Further, S125 to S145 in FIG. 4 may be omitted. In this case, the simple binary image data generated in S115 or S120 using the predetermined specific luminance range YR or specific saturation range CR is used as the final simple binary image data. Also good. Further, S110 to S130 in FIG. 4 may be omitted. In this case, S130 to S145 are always executed, and the simple binary image data generated in S145 using the specific luminance range YRa or the specific saturation range CRa based on the histogram is always final. It may be used as simple binary image data.

(6) In the search range determination process of FIG. 7, distance DU between each of the four ends of the document pixel specific image CI and the corresponding end of the search range SA as range information defining the search range SA. , DB, DL, and DR are generated (S265). For example, the range information may be coordinate information of four corners of the rectangular search range SA, two vertical coordinates indicating the upper end and the lower end of the search range SA, and the left end. And information indicating two horizontal coordinates indicating the right end.

(7) The step of the search range determination process in FIG. 7 can be omitted as appropriate. For example, the processing of S235 to S255 in FIG. When there are M consecutive document pixels PXa at positions on the lines P1U, P1B, P1L, and P1R (S215: YES), the position of the target end is set as the end of the search range SA corresponding to the target end. You may decide.

  Further, it is not necessary to perform the processing of S210 to S255 for all four edges of the document pixel specifying image CI. For example, when the end of 1 to 3 or less of the read image OI that is likely to cause a shadow is known from the structure of the scanner 400, the end corresponding to the end of 1 to 3 or less of the read image OI is S210 to S255. The remaining edge may be searched for whether there are M document pixels PXa continuous from the edge toward the inside.

(8) In the search range determination process of FIG. 7, when there are M consecutive document pixels PXa at positions on the lines P1U, P1B, P1L, P1R (S215: YES), the lines P1U, P1B, Whether or not there are M document pixels PXa continuous from P1L and P1R toward the target end is sequentially searched (S235, S240, and S250). Instead, in this case, it may be searched whether there are M document pixels PXa successively from the target end toward the lines P1U, P1B, P1L, and P1R. In this case, the position on the first line where M consecutive document pixels PXa exist is determined as the end position of the search range SA.

(9) The search range determination process in FIG. 7 indirectly determines the search range on the read image OI by analyzing the document pixel specific image CI using the document pixel specific image data. Alternatively, the search range SA may be determined by directly analyzing the read image data or the reduced image data without using the document pixel specifying image data. For example, a line corresponding to the lines P1U, P1B, P1L, and P1R is set on the read image OI, and whether or not there are M consecutive pixels on the line is determined by the pixel value of the read image data. You may decide using.

(10) A plurality of contour pixels PXo may be specified by a method different from the contour specifying process of FIG. For example, the plurality of contour pixels PXo may be specified by tracing the document pixel PXa adjacent to the non-document pixel PXb, starting from the document pixel PXa located on the end of the search range SA.

(11) The document area specifying process in FIG. 11 can be changed as appropriate. For example, the process of specifying the candidate straight line (S310) may be other various processes of specifying a straight line that can represent the edge of the document, instead of the process using the Hough transform. For example, the CPU 110 may specify a candidate straight line that approximates the arrangement of a plurality of contour pixels using the least square method. In S320, two sets of straight lines perpendicular to each other may be specified instead of the set of straight lines parallel to each other. Further, only the longest straight line may be specified as a straight line indicating the edge of the document. In this case, for example, a rectangle having a size corresponding to the paper size designated by the user may be specified as the document area AA with reference to the one straight line.

(12) The repair process in FIG. 12 can be changed as appropriate. For example, one or two of the three steps S410 to S430 may be omitted. In place of the repair process, the document area AA specification result and the contour specification result may be used for other image processing, for example, trimming processing.

(13) The search range determination process in FIG. 7 may be omitted. For example, the contour specifying process in FIG. 10 may be performed using the entire original pixel specifying image CI as the search range.

(14) The contour specifying process in FIG. 10 may be omitted. In this case, for example, the document area AA may be specified using the document pixel specifying image CI. After the contour specifying process, for example, a process of trimming an image along the specified contour of the document image Ad or a process of correcting an image in the contour of the document image Ad may be performed. For example, when it is difficult for the original image Ad to be inclined, the original pixel PXa isolated in the vicinity of the outer edge of the original pixel specific image CI is removed as noise and then specified in the original pixel specific image CI. A circumscribed rectangle for a plurality of document pixels PXa1 may be specified as the document area AA.

(15) The apparatus for optically reading an original may be various other apparatuses capable of optically reading an original instead of the scanner 400 described in FIG. For example, the scanner 400 may be a scanner that includes an automatic document feeder (Auto Document Feeder) for transporting a document and reads the transported document with the stopped image sensor 420. The CPU 110 of the image processing apparatus 100 replaces the acquisition of read image data from the scanner 400 connected to the image processing apparatus 100, and is not shown connected to the nonvolatile storage device 130 or the image processing apparatus 100. The read image data stored in advance in an external storage device (for example, a USB memory) may be acquired.

(16) The image processing of FIG. 2 may be executed by a different type of device from the personal computer. For example, the image processing may be executed by the CPU 411 of the scanner (for example, the scanner 400). The image processing may be executed by a server device connected to the network. The image processing may be executed in part by a plurality of devices (for example, computers) that can communicate with each other via a network. In this case, the whole of the plurality of apparatuses corresponds to the image processing apparatus.

  In each of the above embodiments, a part of the configuration realized by hardware may be replaced with software, and conversely, part or all of the configuration realized by software may be replaced with hardware. Also good.

  When a part or all of the functions of the present invention are realized by a computer program, the program is provided in a form stored in a computer-readable recording medium (for example, a non-temporary recording medium). be able to. The program can be used in a state where it is stored in the same or different recording medium (computer-readable recording medium) as provided. The “computer-readable recording medium” is not limited to a portable recording medium such as a memory card or a CD-ROM, but is connected to an internal storage device in a computer such as various ROMs or a computer such as a hard disk drive. An external storage device may also be included.

  As mentioned above, although this invention was demonstrated based on the Example and the modification, Embodiment mentioned above is for making an understanding of this invention easy, and does not limit this invention. The present invention can be changed and improved without departing from the spirit and scope of the claims, and equivalents thereof are included in the present invention.

    DESCRIPTION OF SYMBOLS 100 ... Image processing apparatus, 110 ... CPU, 120 ... Volatile memory device, 130 ... Nonvolatile memory device, 132 ... Program, 140 ... Display part, 150 ... Operation part, 190 ... Interface, 255 ... Luminance, 400 ... Scanner, 410: control unit, 411 ... CPU, 412 ... volatile storage device, 413 ... nonvolatile storage device, 414 ... program, 419 ... interface, 420 ... image sensor, 430 ... moving device, 440 ... document table, 450 ... cover, 490 ... Main body, SA ... Search range, AA ... Original area, EI ... Edge binary image, SI ... Scanned image, OI ... Scanned image, MI ... Reduced image, CI ... Original pixel specific image, RI ... Repaired image, BI, BIm, BIs ... simple binary image, YR, YRa ... specific luminance range, CR, CRa ... specific saturation range, NT ... network Click, Ab ... outside image, Ad ... original image, PXa ... original pixel, PXb ... non original pixel, PXe ... edge pixel, PXn ... non-edge pixel, PXO ... contour pixels

Claims (7)

An image processing apparatus,
An image data acquisition unit for acquiring read image data generated by optically reading a document, wherein the read image data includes a first image indicating the document and a second region indicating a region other than the document. The image data acquisition unit showing a read image including an image;
Using the read image data, a plurality of pixels that satisfy a specific condition are specified as the plurality of first pixels, and a plurality of pixels that do not satisfy the specific condition are specified as the plurality of second pixels. A first image generation unit that generates first image data indicating the plurality of first pixels and the plurality of second pixels, wherein the specific condition is within a specific luminance range. The first image generation unit, wherein the specific luminance range is a range including the luminance of pixels corresponding to the second image;
A second image generation unit configured to generate second image data indicating a plurality of edge pixels indicating edges in the read image using the read image data;
A specifying unit that specifies a document area corresponding to the document in the read image using at least the first image data and the second image data;
An image processing apparatus comprising: The image processing apparatus according to claim 1,
The specific part is:
Composite image data indicating a specific pixel group including the plurality of second pixels specified using the first image data and the plurality of edge pixels specified using the second image data. Generate
An image processing apparatus that identifies the document area in the read image using the composite image data. The image processing apparatus according to claim 1, wherein:
The first image generation unit includes a plurality of pixels having a luminance within the specific luminance range and having a saturation within a specific saturation range including a saturation of the second image. A plurality of pixels having a luminance outside the specific luminance range and a plurality of pixels having a saturation outside the specific saturation range are identified as the first pixels of the plurality of second pixels. An image processing apparatus that generates the first image data by specifying as follows. The image processing apparatus according to any one of claims 1 to 3,
The first image generator is
Identifying the plurality of first pixels and the plurality of second pixels using the first specific luminance range;
Determining whether the number of the plurality of second pixels specified by using the first specific luminance range is equal to or less than a reference;
The second specific luminance range different from the first specific luminance range when the number of the plurality of second pixels specified using the first specific luminance range is equal to or less than a reference. An image processing apparatus that generates the first image data by re-identifying the plurality of first pixels and the plurality of second pixels by using. The image processing apparatus according to claim 1, further comprising:
The specific part is:
Using the first image data and the second image data to generate composite image data indicating a composite image;
By analyzing the composite image data, in the composite image, a plurality of candidate straight lines that are candidates for straight lines representing edges of the document are specified,
An image processing apparatus that identifies the document area defined using at least one straight line of the plurality of candidate straight lines. An image processing apparatus according to any one of claims 1 to 5,
The first image generator is
Using the read image data, generate reduced image data indicating a reduced image obtained by reducing the read image,
An image processing apparatus that generates the first image data using the reduced image data. A computer program,
An image data acquisition function for acquiring read image data generated by optically reading a document, wherein the read image data includes a first image indicating the document and a second region indicating a region other than the document. The image data acquisition function showing a read image including an image;
Using the read image data, a plurality of pixels that satisfy a specific condition are specified as the plurality of first pixels, and a plurality of pixels that do not satisfy the specific condition are specified as the plurality of second pixels. A first image generation function for generating first image data indicating the plurality of first pixels and the plurality of second pixels, wherein the specific condition is within a specific luminance range. The first image generation function, wherein the specific luminance range is a range including the luminance of a pixel corresponding to the second image;
A second image generation function for generating second image data indicating a plurality of edge pixels indicating edges in the read image using the read image data;
A specifying function for specifying a document area corresponding to the document in the read image using at least the first image data and the second image data;
A computer program that causes a computer to realize

Images