JP6372397B2 - Image processing apparatus and computer program - Google Patents

Description

  In the present specification, a technique for generating processed image data in which a first image and a second image are arranged, and the first image and the second image represent an arranged image showing one object. About.

  A technique is known that generates arranged image data representing a arranged image in which a first image and a second image are arranged. For example, in the technique disclosed in Patent Document 1, a plurality of read images are acquired by reading a single document having a size that cannot be read at a time in a plurality of times using a scanner. An image obtained by combining the read images is generated. The overlapping position when combining a plurality of read images is determined by performing pattern matching on the plurality of read images.

JP-A-4-290066

  However, the above technique does not take into account the distribution of pixels that are determined to match between the two images. For this reason, in the image in which the first image and the second image are arranged, there is a possibility that the position where these images are arranged cannot be determined with high accuracy.

  In the present specification, a first image and a second image are arranged, and the first image and the second image represent a arranged image showing one object (for example, one original). Disclosed is a technique for accurately determining an arrangement position between a first image and a second image when generating arranged image data.

  The technology disclosed in this specification can be implemented as the following application examples.

Application Example 1 First image data representing a first image showing a part of one object, second image data representing a second image showing another part of the object, An image acquisition unit for acquiring
A reference area determination unit that determines a reference area that is a partial area in the first image;
A first determination unit that determines whether the reference region is similar to one candidate region of the plurality of candidate regions in the second image;
When the reference region and the one candidate region are similar, the reference region and the one candidate region among a plurality of pixels in one region of the reference region and the one candidate region Among the plurality of similar pixels that are similar to the corresponding pixel in the other region and the plurality of dissimilar pixels that are not similar to the corresponding pixel. A second determination unit to
When the bias of the distribution of one of the plurality of similar pixels and the plurality of dissimilar pixels is equal to or less than a reference, the first image and the one candidate region are based on the reference region and the one candidate region. A position determining unit that determines a relative position with respect to the second image;
The first image and the second image are arranged at the determined relative position, and one arranged already representing the processed image showing the object by the first image and the second image An image generation unit for generating image data;
An image processing apparatus comprising:

  According to the above configuration, when it is determined that the reference region and the one candidate region are similar, and the bias of one distribution of the plurality of similar pixels and the plurality of dissimilar pixels is equal to or less than the reference, Based on the reference area and the one candidate area, a relative position between the first image and the second image is determined. As a result, when the processed image data is generated, the arrangement positions of the first image and the second image can be determined with high accuracy.

Application Example 2 An image processing apparatus,
An image acquisition unit that acquires first image data representing a first image and second image data representing a second image;
Of the plurality of pixels in the first image, one of a plurality of similar pixels that are similar to the corresponding pixels in the second image and a plurality of dissimilar pixels that are not similar to the corresponding pixels. A pixel specifying part to be specified;
A plurality of pixels in the first image are classified into a plurality of first classes based on positions in a first direction, and the plurality of similar pixels and the plurality of similar pixels for each of the plurality of first classes A histogram generator for generating a first histogram obtained by counting one of the plurality of dissimilar pixels;
A similarity determination unit that determines whether or not the first image and the second image are similar by using a first feature amount indicating a feature of the shape of the first histogram;
An image processing apparatus comprising:

  In this way, it is possible to easily determine whether or not the first image and the second image are similar using the first histogram.

  The technology disclosed in the present specification can be realized in various forms. For example, an image processing method and an image processing apparatus, a computer program for realizing the functions of the method or the apparatus, and the like It can be realized in the form of a recording medium (for example, a non-temporary recording medium) on which a computer program is recorded.

It is a block diagram which shows the structure of an image processing system. FIG. 11 is a sequence diagram showing an operation of the image processing system 1000. It is explanatory drawing of a scanning process. It is a figure which shows an example of a scanned image and a arranged image. It is a flowchart of an arrangement position determination process. It is a figure which shows an example of reference | standard area | region SP, candidate area | region NP, and judgment mask image MK. It is a flowchart of a distribution judgment process. It is a flowchart of the process which sets the division area PA of 2nd Example. It is explanatory drawing of the process which sets the division area PA of 2nd Example. It is a flowchart of the distribution judgment process of 3rd Example. It is explanatory drawing of the distribution judgment process of 3rd Example. It is explanatory drawing of the scan data of a modification.

A. First embodiment:
A-1: Configuration of Image Processing System 1000 FIG. 1 is a block diagram showing the configuration of the image processing system. The image processing system 1000 includes a server 400 as an image processing apparatus and a multifunction device 200. The server 400 is connected to the Internet 70, and the multifunction device 200 is connected to the Internet 70 via a LAN (abbreviation of local area network) 80. As a result, the server 400 and the multifunction device 200 can communicate with each other via the LAN 80 and the Internet 70. Further, the personal computer 500 of the user of the multifunction device 200 may be connected to the LAN 80.

  The server 400 includes a CPU 410 as an example of a controller of the server 400, a volatile storage device 420 such as a DRAM, a nonvolatile storage device 430 such as a hard disk drive and a flash memory, and an interface for connecting to a network such as the Internet 70. Including a communication unit 480. The volatile storage device 420 is provided with a buffer area 421 for temporarily storing various intermediate data generated when the CPU 410 performs processing. The nonvolatile storage device 430 stores a computer program 431 and a UI data group 433.

  The computer program 431 and the UI data group 433 are installed in the server 400 by being uploaded to the server 400 via the Internet 70 by an administrator of the server 400, for example. Alternatively, the computer program 431 and the UI data group 433 may be provided in a form stored in, for example, a DVD-ROM, and may be installed in the server 400 by the administrator of the server 400. The CPU 410 implements image processing to be described later by executing the computer program 431.

  The multifunction device 200 includes a CPU 210 as an example of a controller of the multifunction device 200, a volatile storage device 220 such as a DRAM, a nonvolatile storage device 230 such as a flash memory and a hard disk drive, a printer unit 240, and a scanner unit 250. , An operation unit 260 such as a touch panel and buttons, a display unit 270 such as a liquid crystal display, and a communication unit 280 that communicates with an external device. For example, the communication unit 280 includes an interface for connecting to a network such as the LAN 80 and an interface for connecting to an external storage device such as a USB memory.

  The volatile storage device 220 is provided with a buffer area 221 for temporarily storing various data generated when the CPU 210 performs processing. A control program 231 is stored in the non-volatile storage device 230. The control program 231 can be provided by being stored in advance in the nonvolatile storage device 230 when the printer 100 is manufactured. Instead, the control program 231 can be provided, for example, in a form downloaded from a server connected via the Internet, or in a form recorded on a CD-ROM or the like.

  The printer unit 240 executes printing using a printing method such as an inkjet method or a laser method. The scanner unit 250 generates scan data representing a color image or a gray image by optically reading a document using a photoelectric conversion element such as a CCD or a CMOS. The scanner unit 250 includes a so-called flatbed type document table 255 described later.

  The CPU 210 executes control of the multifunction device 200 by executing the control program 231. For example, the CPU 210 controls the printer unit 240 and the scanner unit 250 to execute a copy process, a print process, a scan process, and the like. Further, the CPU 210 can access the server 400 and use a service provided by the server 400.

A-2: Operation of Image Processing System 1000 FIG. 2 is a sequence diagram showing the operation of the image processing system 1000. The processing in the sequence diagram is started when the multifunction device 200 receives an instruction to use an image generation service provided by the server 400 from a user. Although this image generation service will be described in detail later, a plurality of images represented by a plurality of scan data are arranged, and one object (specifically, a manuscript 10 described later) is composed of a plurality of images. This is a service for generating arranged image data representing an arranged image indicating the above. The details of the plurality of scan data are described later. For example, the plurality of scan data are generated by reading a document having a size larger than the size that can be read at one time in a plurality of times.

  When the process is started, in S5, the CPU 210 of the multifunction device 200 transmits a service start request to the server 400. Upon receiving the service start request, the CPU 410 of the server 400 selects UI data necessary for providing the image generation service from the UI data group 433 shown in FIG. 1, and the UI data is sent to the multifunction device 200 in S10. To send. Specifically, the UI data includes screen data representing a user interface screen (hereinafter also referred to as UI screen) and control data. This control data includes, for example, various data necessary for the MFP 200 to perform predetermined processing such as the scan processing in S15 described later using the UI screen. For example, the control data includes information necessary for performing processing to be executed by the multifunction device 200, for example, a transmission destination address of an image file. The process to be executed by the multifunction device 200 includes, for example, a process of transmitting an image file to the server 400 in S20 described later.

  In S15, the CPU 210 executes a scan process for generating a plurality of scan data based on the received UI data. In the scan process, the CPU 210 generates two pieces of scan data by reading the document prepared by the user in two steps. The scan data of this embodiment is, for example, RGB image data including component values of RGB components represented by 256 gradation values from 0 to 255 for each pixel.

  FIG. 3 is an explanatory diagram of the scanning process. FIG. 3A shows an example of a document used in this embodiment. The size of the document 10 is A3 size in this embodiment. The line CL is a line that is located in the center of the A3 size original 10 in the longitudinal direction and is parallel to the short side of the original 10. The left area 10L is a left half area of the document 10, that is, an area on the left side of the line CL. The right region 10R is a half region on the right side of the document 10, that is, a region on the right side of the line CL of the document 10. The size of the left region 10L and the size of the right region 10R are A4 size. The A3 size and the A4 size are paper dimensions defined by ISO (abbreviation of International Organization for Standardization) 216.

  The length in the longitudinal direction of the document table 255 (FIGS. 3B and 3C) of the scanner unit 250 is slightly longer (for example, several centimeters) than 297 mm, which is the length in the longitudinal direction of the A4 size. The length of the manuscript table in the short-side direction is the length of the letter-size short-side direction, which is the paper size defined by ANSI / ASME (Abbreviation of American National Standards Institute / American Society of Mechanical Engineers) Y14.1. It is slightly longer (for example, several centimeters) than the length 215.9 mm. That is, in this embodiment, the maximum size of a document that can be read at one time is larger than the A4 size and smaller than the A3 size. Therefore, in this embodiment, the A3 size original 10 is read in two steps.

  In the first reading, left scan data indicating an image in an area including the left area 10L and a partial area CAR in contact with the left area 10L in the right area 10R is generated. Specifically, the CPU 210 prompts the user to place the document 10 on the document table 255 in the state shown in FIG. 3B by displaying a UI screen (not shown) on the display unit 270. As shown in FIG. 3B, the user faces the document surface on which the image of the document 10 is placed toward the document table 255, and the left area 10L of the document surface is the back side (the upper side in FIG. 3B). ), The document 10 is placed on the document table 255 so that the right area 10R of the document surface is positioned on the near side (lower side in FIG. 3B). When the user inputs a reading instruction to the multifunction device 200, the CPU 210 controls the scanner unit 250 to read the left region 10L and the partial region CAR, thereby generating left scan data.

  In the second reading, right scan data indicating an image in an area including the right area 10R and a partial area CAL in contact with the right area 10R in the left area 10L is generated. Specifically, after generating the left scan data, the CPU 210 displays a UI screen (not shown) on the display unit 270, thereby placing the document 10 on the document table 255 in the state shown in FIG. Prompt the user to do. As shown in FIG. 3C, the user faces the document surface on which the image of the document 10 is arranged to the document table 255 side, and the right area 10R of the document surface is the back side (the upper side of FIG. 3C). ), The document 10 is placed on the document table 255 so that the left area 10L of the document surface is positioned on the near side (lower side in FIG. 3C). When the user inputs a reading instruction to the multifunction device 200, the CPU 210 controls the scanner unit 250 to read the right region 10R and the partial region CAL, thereby generating right scan data.

  FIG. 3D shows a left scan image IL1 represented by left scan data and a right scan image IR1 represented by right scan data. The left-side scanned image IL1 includes a left-side document image HIL that indicates the left area 10L and the partial area CAR of the document 10, and a margin WBL. The right-side scan image IR1 includes a right-side original image HIR indicating the right area 10R and the partial area CAL, and a margin WBR. These document images HIL and HIR respectively include images CIL and CIR representing the center portion CA in the lateral direction of the document 10 (that is, the region composed of the above-described partial regions CAL and CAR).

  These scanned images IL1 and IR1 have a rectangle corresponding to the shape of the document table 255 in FIGS. 3B and 3C. The scanned images IL1 and IR1 are arranged on the document table 255 with the direction along the long side of the document table 255 as the vertical direction and the left direction of the document table 255 in FIGS. 3B and 3C as the upward direction. Represents an image. An upward direction of the original 10 with respect to the original table 255 at the time of the first reading (left direction in FIG. 3B) and an upward direction of the original 10 with respect to the original table 255 at the time of the second reading (FIG. 3C). Is the opposite direction. For this reason, the left side original image HIL of the left side scan image IL1 and the right side original image HIR of the right side scan image IR1 are reversed in the vertical direction.

  In the scanning process of S15, the left side scan data and the right side scan data are each converted into a file of a predetermined format, for example, an image file of JPEG (Joint Photographic Experts Group) format.

  In S <b> 20 of FIG. 2, the CPU 210 transmits the left scan data image file (also referred to as the left image file) and the right scan data image file (also referred to as the right image file) to the server 400. As a result, in S25, the CPU 410 of the server 400 acquires these two image files. The CPU 410 acquires left scan data and right scan data from these image files and stores them in the buffer area 421. For example, when an image file in JPEG format is used, scan data converted into RGB image data is acquired from the image file by a predetermined conversion process and stored in the buffer area 421.

  In S30, the CPU 410 performs rotation processing on the left side scan data and the right side scan data, respectively. Specifically, the CPU 410 performs known processing such as Hough transform and edge detection processing on the left scan data to detect the upper end of the left original image HIL in the scan image IL1. The CPU 410 rotates the left scan image IL1 so that the upper end of the left original image HIL and the horizontal direction of the image are parallel to generate rotated left scan data representing the rotated left scan image IL2. . As a result, for example, it is possible to correct the inclination of the original image HIL generated due to the original 10 being inclined with respect to the original table 255. Further, the CPU 410 corrects the inclination of the right original image HIR by executing the same processing as the left scan data for the right scan data. Further, the CPU 410 rotates the right scan image IR1 by 180 degrees in order to make the vertical direction of the right original image HIR coincide with the vertical direction of the left original image HIL, and shows the rotated right side scan image IR2. Generate scan data.

  FIG. 4 is a diagram illustrating an example of the scanned image and the arranged image. FIG. 4A shows an example of the rotated left scan image IL2 and the rotated right scan image IR2. Hereinafter, scan data representing the rotated left scan image IL2 is also simply referred to as left scan data, and scan data representing the rotated right scan image IR2 is also simply referred to as right scan data.

  In S35, the CPU 410 determines a reference area SP that is a partial area of the right scan image IR2 in the right scan image IR2. As shown in FIG. 4A, the reference region SP is determined to be a rectangular region having a predetermined size arranged at a predetermined position in the right-side scan image IR2. Further, the reference area SP is arranged in an image CIR representing the central portion CA (FIG. 3A) of the document 10 in the right side scan image IR2. For example, the reference region SP is arranged along an end (left end in the present embodiment) along the image CIR representing the center CA among the four ends of the right-side scan image IR2. The horizontal length of the reference region SP is, for example, 50 to 150 pixels, and the vertical length of the reference region SP is 1/4 to 1/2 of the vertical length of the right-side scan image IR2. It is.

  In S40, the CPU 410 determines a search area SA shown in FIG. 4A in the left scan image IL1. The search area SA is a predetermined area. The length in the vertical direction of the search area SA in FIG. 4A is, for example, equal to the total length in the vertical direction of the left-side scan image IL2. The horizontal length of the search area SA is, for example, 20% to 50% of the horizontal length of the left scan image IL2. Search area SA preferably includes an image CIL representing central portion CA (FIG. 3A) of document 10. The search area SA is an area including the right end of the left scan image IL2, for example.

  In S45, the CPU 410 executes an arrangement position determination process. The layout position determination process is a process for determining the relative positions of the left document image HIL and the right document image HIR in a placed image BI (described later) to be generated. Briefly, a similar region CP similar to the reference region SP in the right scan image IR2 is determined from the search region SA in the left scan image IL2. The position where the reference area SP in the right original image HIR and the similar area CP in the left original image HIL overlap is determined as a relative position between the left original image HIL and the right original image HIR. For example, a part of the document 10 shown in the reference area SP in the right-side scan image IR2 in FIG. 4A is set as a specific part SPT shown in FIG. The similar area CP is an area representing the specific portion SPT of the document 10 in the left side scan image IL2. Details of the arrangement position determination processing will be described later.

  In S50, the CPU 410 executes an arranged image generation process. In the arranged image generation process, the CPU 410 uses the right scan data and the left scan data to arrange the right document image HIR in the right scan image IR2 and the left document image HIL in the left scan image IL2. Arranged image data representing the completed image BI is generated. Then, the CPU 410 generates an image file representing the arranged image BI by converting the arranged image data into an image file of a predetermined format.

  FIG. 4B is a diagram illustrating an example of the arranged image BI. As shown in FIG. 4B, in the arranged image BI, the right original image HIR and the left original image HIL are arranged at the relative positions determined in the arrangement position determining process. That is, in the arranged image BI, the right original image HIR and the left original image HIL are arranged such that the reference area SP in the right original image HIR and the similar area CP in the left original image HIL overlap. . In the arranged image BI, for example, the value of the pixel in the right original image HIR (that is, the right scanned image IR2) is preferentially given to the value of the pixel in the region where the right original image HIR and the left original image HIL overlap each other. Adopted. The arranged image BI is a right original image HIR and a left original image HIL, and the right original image HIR and the left original image HIL are one image showing the original 10 in FIG.

  In S <b> 55 of FIG. 2, the CPU 410 transmits the generated arranged image file to the multifunction device 200. When receiving the arranged image file, the CPU 210 of the multifunction device 200 stores the received arranged image file in the nonvolatile storage device 230 and notifies the user that the arranged image file has been received. The arranged image file is provided for use by the user. For example, the multifunction device 200 can print the arranged image BI using the arranged image file based on a user instruction.

  According to the image processing system 1000 described above, a plurality of image files obtained by reading different areas from one original 10 shown in FIG. 3, specifically, an image file of right-side scan data, Using the left scan data image file, an arranged image file representing one original 10 can be generated.

A-3. Arrangement position determination processing:
The arrangement position determination process in S45 of FIG. 2 will be described. FIG. 5 is a flowchart of the arrangement position determination process.

  In S205, the CPU 410 specifies one attention candidate area in the search area SA in the left-side scan image IL2 in FIG. FIG. 4A shows one candidate area NP1 in the search area SA. The size shape and size of one candidate region are the same as the size and shape of the reference region SP in the right-side scan image IR2 in FIG. Each pixel in the reference area SP and each pixel in one candidate area NP have a one-to-one correspondence. It can be seen that a plurality of candidate areas NP can be arranged in the search area SA by moving the candidate area NP1 of FIG. 4A in the vertical direction or the horizontal direction in increments of one pixel. In S205, one area is identified as a target candidate area in a predetermined order from all candidate areas NP that can be arranged in the search area SA.

  In S210, CPU 410 initializes determination mask image MK. Although an example of the judgment mask image MK will be described later, the judgment mask image MK is a binary image having the same shape and size as the reference region SP and the target candidate region. That is, the number of pixels in the reference region SP and the attention candidate region is the same as the number of pixels in the determination mask image MK, and each pixel in the reference region SP and the attention candidate region and each pixel in the determination mask image MK are one-to-one. It corresponds with.

  In S215, the CPU 410 selects one target pixel from the plurality of pixels in the target candidate area. In S220, the CPU 410 calculates a difference ΔVP between the value of the target pixel in the target candidate region (for example, RGB value) and the value of the pixel in the reference region SP corresponding to the target pixel. A pixel in the reference region SP corresponding to the target pixel is a pixel in the reference region SP that overlaps the target pixel when the image in the target candidate region and the image in the reference region SP are overlapped. The values of the two pixels for which the difference ΔVP is to be calculated are (R1, G1, B1) and (R2, G2, B2). The difference ΔVP is represented by the sum of absolute values of differences between the three types of component values. That is, the difference ΔVP is represented by the sum of the absolute value of (R1−R2), the absolute value of (G1−G2), and the absolute value of (B1−B2).

  In S225, the CPU 410 determines whether or not the calculated difference ΔVP is less than or equal to a predetermined reference value THv. When the difference ΔVP is equal to or smaller than the predetermined reference value THv (S225: YES), in S230, the CPU 410 determines that the target pixel is a similar pixel, and records the target pixel as a similar pixel. Specifically, the CPU 410 sets the pixel value of the determination mask image MK corresponding to the target pixel to “ON”. When the difference ΔVP is larger than the predetermined reference value THv (S225: NO), the CPU 410 determines that the target pixel is a dissimilar pixel in S235, and records the target pixel as a dissimilar pixel. Specifically, the CPU 410 sets the pixel value of the determination mask image MK corresponding to the target pixel to “OFF”.

  In S240, the CPU 410 determines whether or not all pixels in the attention candidate region have been processed as the attention pixel. When there is an unprocessed pixel (S240: NO), the CPU 410 returns to S215 and selects the unprocessed pixel as a target pixel. When all the pixels have been processed (S240: YES), the CPU 410 advances the process to S245. At the time of shifting to S245, all the pixels in the target candidate region are classified as either a similar pixel or a dissimilar pixel.

  In S245, the CPU 410 calculates the similar pixel ratio R1 of the candidate region of interest. The ratio R1 is a value obtained by dividing the number SC of similar pixels in the target candidate region by the total number Nt of pixels in the target candidate region (R1 = (SC / Nt)).

  In S250, the CPU 410 determines whether or not the similar pixel ratio R1 is equal to or greater than the threshold value TH1. When the ratio R1 of similar pixels is equal to or greater than the threshold value TH1 (S250: YES), the CPU 410 executes a distribution determination process in S255. The distribution determination process is a process for determining whether or not the deviation of the distribution of similar pixels in the target candidate region is below a reference. If the distribution of similar pixels is less than the reference, the distribution of similar pixels is determined to be uniform. If the distribution of similar pixels is greater than the reference value, the distribution of similar pixels is biased. It is judged. The distribution determination process will be described later.

  If the distribution of similar pixels is uniform (S260: YES), the CPU 410 determines the attention candidate area as a similar area CP similar to the reference area SP in S265, and the CPU 410 in S265. In S270, CPU 410 determines the relative position of left original image HIL and right original image HIR based on the current candidate region of interest determined as similar region CP and reference region SP. Then, the arrangement position determination process is terminated. Specifically, as described above with reference to FIG. 4B, the reference area SP in the right original image HIR, the current candidate area for attention (ie, the similar area CP) in the left original image HIL, The arrangement position is determined so that the two overlap.

  When the ratio R1 of similar pixels is less than the threshold value TH1 (S250: NO), and when the distribution of similar pixels is not uniform but uneven (S260: NO), the CPU 410, in S275, It is determined whether all candidate areas in the search area SA have been processed as attention candidate areas. If there is an unprocessed candidate area (S275: NO), the CPU 410 returns to S205 and identifies the unprocessed candidate area as the attention candidate area.

  If all the candidate areas have been processed (S275: YES), the CPU 410 determines the relative position of the left original image HIL and the right original image HIR as the default position in S280, and arranges them. The position determination process ends. The default position is, for example, a position where the right end of the left scan image 20L simply touches the left end of the right scan image 20R in the entire length.

  According to the arrangement position determination process described above, in S250, it is determined whether or not the reference area SP and the target candidate area are similar based on whether the similar pixel ratio R1 is equal to or greater than the threshold value TH1. When the reference area SP and the target candidate area are similar (S250: YES), the CPU 410 executes a distribution determination process in S255. Then, if the distribution of similar pixels in the target candidate area is uniform (S260: YES), the CPU 410 makes a relative between the left original image HIL and the right original image HIR based on the reference area SP and the target candidate area. A specific position is determined (S270). As a result, when the arranged image data is generated in S50 of FIG. 2, the arrangement positions of the two original images HIL and HIR can be accurately determined.

  This will be described in more detail. Since the original images HIL and HIR vary depending on the original 10 to be read, the images in the reference area SP also vary. For example, an image in the reference area SP may have a relatively large edge amount or a relatively small edge amount. The number of noise pixels included in the document images HIL and HIR varies depending on the amount of edges in the image and the characteristics of the scanner unit 250 for reading the document 10. The noise pixel is a pixel that does not indicate the color to be represented on the document 10. For example, an edge portion in a scan image tends to include noise pixels depending on the positional relationship between the edge and each cell of the image sensor of the scanner. For this reason, for example, the similarity region CP similar to the reference region SP is determined only by the similarity determination based on the number of similar pixels. May not be determined properly.

  FIG. 6 is a diagram illustrating an example of the reference region SP, the candidate region NP, and the determination mask image MK. The reference region SP1 in FIG. 6A is an image having a relatively large number of noise pixels. The candidate area NP1 in FIG. 6A is a candidate area similar to the reference area SP1. That is, the candidate area NP1 in FIG. 6A is a candidate area to be determined as the similar area CP. The determination mask image MK1 in FIG. 6A is a determination mask image showing similar and dissimilar pixels between the reference region SP1 and the candidate region NP1.

  The reference area SP2 in FIG. 6B is an image having a relatively large number of noise pixels. The candidate area NP2 in FIG. 6B is a candidate area that is not similar to the reference area SP2. That is, the candidate area NP2 in FIG. 6B is a candidate area that should not be determined to be the similar area CP. A judgment mask image MK2 in FIG. 6B is a judgment mask image showing similar and dissimilar pixels in the reference area SP2 and the candidate area NP2.

  In the judgment mask images MK1 and MK2, the black portion is a region where dissimilar pixels exist, and the white portion is a region where similar pixels exist. Among the black dissimilar pixels, the first dissimilar pixel Pn is a non-similar pixel due to at least one of the two pixels corresponding to each other of the reference region SP and the candidate region NP being a noise pixel. The dissimilar pixels determined to be similar are shown. Among the black dissimilar pixels, the second type dissimilar pixel Pd is determined to be dissimilar, although both of the two corresponding pixels of the reference region SP and the candidate region NP are not noise pixels. The dissimilar pixels are shown. The second type of dissimilar pixel Pd is a pixel constituting a different area representing an image substantially different from the corresponding area of the candidate area NP in the reference area SP. Noise pixels are distributed relatively uniformly throughout the image. For this reason, there is a possibility that the first type dissimilar pixels Pn are distributed relatively uniformly in the entire determination mask image MK (that is, the entire reference region SP and the entire candidate region NP). high. Since the second type dissimilar pixel Pd forms a different area as described above, the second type dissimilar pixel Pd is biased to the different area in the determination mask image MK (that is, in the reference area SP and the candidate area NP). Probability is high.

  In the example of FIG. 6A, since the reference region SP1 is similar to the candidate region NP1, almost all the dissimilar pixels are the first type dissimilar pixels Pn. Therefore, in this example, as can be understood from the judgment mask image MK1, dissimilar pixels are distributed and distributed relatively uniformly over the entire reference region SP and the entire candidate region NP. On the other hand, in the example of FIG. 6B, since the reference region SP2 is not similar to the candidate region NP2, the dissimilar pixels are the first type dissimilar pixel Pn and the second type dissimilarity. Pixel Pd. In the determination mask image MK2, the first type dissimilar pixels Pn are uniformly distributed throughout the image, and the second type dissimilar pixels Pd are different regions DFA between the reference region SP2 and the candidate region NP2, It is biased toward DFb.

  Here, the amount of noise in the reference region SP1 is larger than the amount of noise in the reference region SP2. For this reason, the number of first-type dissimilar pixels Pn in the determination mask image MK1 is larger than the number of first-type dissimilar pixels Pn in the determination mask image MK2. For this reason, the total number of similar pixels Pn of the determination mask image MK1 (that is, the sum of the first type of dissimilar pixels Pn and the second type of dissimilar pixels Pd) is the total number of dissimilar pixels of the determination mask image MK2. It is almost the same. In such a case, it can be understood that the similar region CP similar to the reference region SP cannot be appropriately determined from the plurality of candidate regions only by the similarity determination based on the number of similar pixels.

  Here, in this embodiment, all the pixels in the reference region SP and the candidate region NP are classified as either similar pixels or dissimilar pixels. For this reason, when dissimilar pixels are distributed throughout the reference region SP and the candidate region NP, it may be determined that similar pixels are also uniformly distributed throughout the reference region SP and the candidate region NP. . Conversely, when similar pixels are distributed throughout the reference region SP and candidate region NP, it may be determined that dissimilar pixels are also uniformly distributed throughout the reference region SP and candidate region NP. Therefore, whether dissimilar pixels and similar pixels are uniformly distributed or unevenly distributed throughout the reference region SP and the candidate region NP may be determined based on dissimilar pixels. You may judge. In the present embodiment, the determination is made based on similar pixels, as will be described later in the description of the distribution determination process.

  Therefore, in the arrangement position determination process (FIG. 5) according to the present embodiment, as described above, whether the distribution of similar pixels is uniform in the attention candidate region determined to be similar by the similarity determination based on the number of similar pixels. Judging whether it is biased. When the distribution of similar pixels is uniform, the candidate region of interest is determined to be a similar region CP. When the distribution of similar pixels is not uniform, the candidate region of interest is a similar region CP. Not determined. As a result, a similar region CP similar to the reference region SP can be appropriately determined. Therefore, the arrangement positions of the two original images HIL and HIR can be determined with high accuracy.

A-4. Distribution judgment processing:
The distribution determination process in S255 of FIG. 5 will be described. FIG. 7 is a flowchart of the distribution determination process.

  In S300, the CPU 410 determines a specific range for determination. The specific range SR for determination is a specific range for the ratio R2 of similar pixels in a divided area PA described later. In this embodiment, the specific range SR is determined based on the similar pixel ratio R1 calculated in S245 of FIG. Specifically, the specific range SR is a range having a predetermined width centered on the ratio R1 of similar pixels in the entire determination mask image MK. In this embodiment, the specific range SR is set to (R1−ΔR) or more and (R1 + ΔR). ΔR is, for example, 10%.

  In S305, the CPU 410 divides the judgment mask image MK into a plurality of pieces, and sets a plurality of divided areas PA in the judgment mask image MK. For example, the determination mask image MK is divided into a checkered pattern, so that a rectangular divided area PA having a predetermined size (for example, 32 pixels vertically × 32 pixels horizontally) is set in the determination mask image MK. In the examples of FIGS. 6A and 6B, a total of 22 divided areas PA of 11 vertical × 2 horizontal are set in the determination mask images MK1 and MK2. As described above, the size and shape of the reference region SP and the candidate region for attention are equal to the size and shape of the determination mask image MK. For this reason, setting a plurality of divided areas PA in the judgment mask image MK is synonymous with setting a plurality of divided areas PA in the reference area SP and the target candidate area.

  In S310, the CPU 410 selects one target divided region from the plurality of divided regions PA. In S315, the CPU 410 calculates the similar pixel ratio R2 of the target divided region. The ratio R1 is a value obtained by dividing the number SCp of similar pixels in the target divided region by the total number Ntp of pixels in the target divided region (R2 = (SCp / Ntp)).

  In S320, the CPU 410 determines whether the similar pixel ratio R2 is within the specific range SR determined in S300. In other words, it is determined whether or not the number SCp of similar pixels is within the range of (R1−ΔR)% to (R1 + ΔR)% of the total number Ntp. When the ratio R2 of similar pixels is within the specific range SR (S320: YES), the CPU 410 records the attention divided region as a uniform region in S325. When the ratio R2 of similar pixels is not within the specific range SR (S320: NO), the CPU 410 records the attention divided region as a biased region in S330. The uniform region is a divided region PA in which the internal similar pixel ratio R2 is relatively approximate to the similar pixel ratio R1 of the entire determination mask image MK. The biased area is a divided area PA in which the ratio R2 of the similar pixels inside is relatively far from the ratio R1 of the similar pixels in the entire determination mask image MK.

  For example, if the judgment mask image MK to be processed is the judgment mask image MK1 in FIG. 6A, similar pixels and dissimilar pixels are distributed throughout the judgment mask image MK1, so that the judgment mask image MK1 A relatively high proportion of the divided areas PA is determined to be a uniform area.

  On the other hand, when the judgment mask image MK to be processed is the judgment mask image MK2 in FIG. 6B, similar pixels and dissimilar pixels are not distributed over the whole judgment mask image MK2, and therefore, the judgment mask image MK2 A relatively high proportion of the divided areas PA is determined to be a biased area. For example, in the divided area PA where the second type dissimilar pixel Pd of the determination mask image MK2 is unevenly distributed, the ratio R2 of similar pixels in the divided area PA is relatively lower than the ratio R1 of the entire similar pixels. Further, in the divided area PA where the second type dissimilar pixel Pd of the determination mask image MK2 is not unevenly distributed, the ratio R2 of similar pixels in the divided area PA is relatively higher than the ratio R1 of the entire similar pixels.

  In S335, the CPU 410 determines whether or not all the divided areas PA in the determination mask image MK have been processed as the target divided areas. When there is an unprocessed divided area PA (S335: NO), the CPU 410 returns to S310 and identifies the unprocessed divided area PA as a target divided area. When all the divided areas PA have been processed (S335: YES), the CPU 410 advances the process to S340. When the process proceeds to S340, all the divided areas PA of the determination target mask image MK are classified as either a uniform area or a biased area.

  In S340, the CPU 410 calculates a uniform area ratio R3 of the determination mask image MK. The ratio R3 is a value obtained by dividing the number RC of uniform areas in the judgment mask image MK by the total number PNt of the divided areas PA in the judgment mask image MK (R3 = (RC / PNt)).

  In S345, the CPU 410 determines whether or not the uniform area ratio R3 is equal to or greater than the threshold value TH3. The threshold value TH3 is 90%, for example. When the uniform area ratio R3 is equal to or greater than the threshold value TH3 (S345: YES), the CPU 410 determines in S350 that the distribution of similar pixels in the determination mask image MK is uniform. In other words, the reference area SP and the candidate area corresponding to the determination mask image MK are determined to be areas where the distribution deviation of similar pixels is equal to or less than the reference. If the uniform area ratio R3 is less than the threshold value TH3 (S345: NO), the CPU 410 determines that the distribution of similar pixels in the determination mask image MK is biased in S355. In other words, the reference region SP and the candidate region corresponding to the determination mask image MK are determined to be regions where the deviation of the distribution of similar pixels is larger than the reference.

  According to the distribution determination process of the first embodiment described above, the divided area PA is set in the determination mask image MK (S305). That is, a plurality of divided areas PA are set in the reference area SP and the target candidate area. Then, it is determined whether each of the plurality of divided areas PA is a uniform area based on the number of similar pixels (S320 to S330). Then, using the ratio R3, which is a feature amount based on the number of uniform regions, it is determined whether or not the distribution of similar pixels in the reference region SP and the candidate region of interest is below the reference (S345 to S355). As a result, it is possible to accurately determine whether or not the deviation of the distribution of similar pixels in the reference region SP and the candidate region of interest is below the reference. For example, in the reference area SP1 and the candidate area NP1 in FIG. 6A, it is determined that the distribution deviation of similar pixels is below the reference. In the reference area SP2 and the candidate area NP2 in FIG. If the distribution bias is larger than the reference, it is determined appropriately.

  In S215 to S250 of FIG. 5 of the first embodiment, it is determined whether the reference area SP is similar to the target candidate area using the number SC of similar pixels in the determination mask image MK. Then, in the distribution determination process of FIG. 7, it is determined whether or not the distribution deviation of similar pixels used in S215 to S250 of FIG. As a result, it is possible to efficiently execute the primary similarity determination between the reference region SP and the target candidate region in S215 to S250 in FIG. 5 and the determination regarding the distribution of similar pixels in the distribution determination process.

  Furthermore, in S215 to S250 in FIG. 5, when the ratio R1 indicating the number SC of similar pixels to the total number Nt of pixels in the reference region SP is equal to or greater than the threshold value TH1 (S250: YES), the reference region SP and the candidate region of interest Are judged to be similar. In the distribution determination process of FIG. 7, when the ratio R2 indicating the number of similar pixels SCp with respect to the total number Ntp of pixels in the target divided area is the specific range SR based on the total ratio R1, the target divided area is a uniform area. (S340 to S355). As a result, since the ratio R3 can be calculated appropriately, it can be determined with higher accuracy whether or not the distribution deviation of the plurality of similar pixels is equal to or less than the reference.

B. Second embodiment:
The second embodiment is different from the first embodiment in the process of setting the divided area PA in S305 of FIG. Processes other than S305 in FIG. 7 are the same as those in the first embodiment. FIG. 8 is a flowchart of a process for setting the divided area PA of the second embodiment. FIG. 9 is an explanatory diagram of a process for setting the divided area PA of the second embodiment. In the second embodiment, a plurality of types of divided areas PA having different sizes can be set. Specifically, in the second embodiment, a large divided area PAb, a medium divided area PAm having a smaller size than the large divided area PAb, and a small divided area PAs having a smaller size than the middle divided area PAm can be set.

  In S405, as shown in FIG. 9A, the CPU 410 divides the entire judgment mask image MK (that is, the entire reference region SP and the entire candidate region of interest) into 4 pieces of 2 in the vertical direction and 2 in the horizontal direction. Thus, four large divided areas PAb are set.

  In S410, one target divided area is selected from the set divided areas PA. In the first S410, one target divided area is selected from the four large divided areas PAb. Note that in S410 after the medium divided area PAm and the small divided area PAs are set in S430, which will be described later, in addition to the large divided area PAb, all the divided areas including these medium divided area PAm and small divided area PAs are included. One unprocessed divided area PA is selected as a target divided area from the PAs.

  In S415, the CPU 410 calculates a variance value σpa of pixel values in the target divided area. For example, the variance value of the luminance value Y of each pixel in the target divided region of the reference region SP is calculated as the variance value σpa of the target divided region. The variance value σpa is a feature amount indicating variation in pixel values in the target divided region, and indicates that the variation in pixel values increases as the variance value σpa increases.

  In S420, the CPU 410 determines whether or not the calculated variance value σpa is greater than or equal to the threshold value THpa. When the variance value σpa is equal to or greater than the threshold value THpa (S420: YES), the CPU 410 determines whether or not the attention divided region is the minimum size in S425. The threshold value THpa is, for example, smaller than the variance value of the area representing the object of the document 10 (the house in the example of FIG. 3A), and the solid background of the document 10 (for example, the paper portion on which the document object is not printed) ) Is experimentally set in advance so as to be a value larger than the variance value of the region representing). In the present embodiment, the minimum-size divided area that can be set is the small divided area PAs, and therefore it is determined whether or not the target divided area is the small divided area PAs.

  If the target divided region is not the minimum size (S425: NO), the CPU 410 divides the target divided region into four of 2 vertical × 2 horizontal in S430, and the target divided region is divided into the target divided regions. A divided area PA having a size smaller than the area is set. For example, when the target divided area is the upper right large divided area PAb in FIG. 9A, as shown in FIG. 9B, there are four medium divided areas PAm in the large divided area PAb. Is set. For example, when the target divided area is the lower left middle divided area PAm among the four middle divided areas PAm in FIG. 9B, as shown in FIG. 9C, the middle divided area PAm Four subdivided areas PAs are set therein.

  When the calculated variance value σpa is less than the threshold THpa (S420: NO), or when the target divided area is the minimum size (S425: YES), the CPU 410 processes the target divided area in S435. Recorded as a completed area. Since the divided area PA recorded as the processed area is not selected as the attention divided area in the subsequent S410, it is not divided smaller than the current size.

  In S440, CPU 410 determines whether there is an unprocessed divided area PA that is not recorded as a processed area. If there is an unprocessed divided area PA (S440: YES), the CPU 410 returns to S410 and selects the unprocessed divided area PA as a target divided area. If there is no unprocessed divided area PA (S440: NO), the CPU 410 ends the process of FIG.

  According to the processing in FIG. 8 described above, when the variance value σpa of the attention divided area is equal to or greater than the threshold value THpa (S420: YES), the attention division area is determined in S430 as long as the attention division area is not the minimum size. It is further divided into four. When the variance value σpa of the attention divided area is less than the threshold value THpa (S420: NO), the attention division area is not divided smaller than the current one (S435). That is, a relatively small divided area (for example, a small divided area PAs) is set in an area where the variation in pixel values is relatively large in the reference area SP. A relatively large divided area (for example, a large divided area PAb) is set in an area where the variation in pixel values is relatively small.

  In the process of FIG. 7, it is determined whether each divided area PA is a uniform area or a biased area regardless of the size (S315 to S330). In S340 of FIG. 7, when calculating the uniform region ratio R3, the number of uniform regions RC is counted depending on the size of the uniform region. That is, even if the uniform area is any one of the small divided area PAs, the middle divided area PAm, and the large divided area PAb, it is counted as one uniform area.

  According to the second embodiment described above, in this way, the uniform area ratio R3, which is the feature amount used for the determination in the distribution determination process of FIG. 7, can be appropriately calculated. Therefore, in the distribution determination process of FIG. It is possible to more appropriately determine whether or not the deviation of the distribution of the plurality of similar pixels is below the reference. As a result, since the similar area CP similar to the reference area SP can be determined with higher accuracy, the relative position between the left original image HIL and the right original image HIR can be determined with higher accuracy.

  When comparing the similarity between the reference area SP and the attention candidate area, the reference area SP or the attention candidate area has a relatively small variation in pixel values, such as an area representing the background of the document 10. It is difficult to determine similarity even when comparing regions without features. When comparing the similarity between the reference area SP and the attention candidate area, the reference area SP or the attention candidate area is an area having a relatively large variation in pixel values, for example, an area representing an object of the document 10. Similarity can be judged with higher accuracy by comparing areas with particular features. For this reason, when determining whether the distribution of similar pixels is uniform or biased, it is preferable to focus on an area where the variation in pixel values is relatively large. In this embodiment, when calculating the ratio R3 of the uniform area, the contribution ratio of the area where the pixel value variation is relatively large in the reference area SP is the contribution ratio of the area where the pixel value variation is relatively small. Get higher. As a result, when determining whether the distribution of similar pixels is uniform or biased, it is possible to make a determination with emphasis on an area where the variation in pixel values is relatively large. Therefore, as described above, the similar region CP similar to the reference region SP can be determined with higher accuracy.

C. Third Embodiment In the third embodiment, the distribution determination process is different from the first embodiment and the second embodiment. FIG. 10 is a flowchart of the distribution determination process of the third embodiment. FIG. 11 is an explanatory diagram of the distribution determination process of the third embodiment. Processing other than the distribution determination processing is the same as that of the first embodiment.

  In S505, a projection histogram Hv in the vertical direction of the determination mask image MK is generated. Specifically, the CPU 410 classifies the plurality of pixels in the determination mask image MK into a plurality of classes based on the vertical positions (that is, the vertical positions of the reference region SP and the candidate region of interest). To do. In the present embodiment, a plurality of pixels having the same vertical position, that is, a plurality of pixels on a line of one pixel extending in the horizontal direction are classified into one class. For example, when the size of the judgment mask image MK is vertical P pixels × horizontal Q pixels, a plurality of pixels in the judgment mask image MK are classified into P classes and pixels belonging to one class. The number is Q. Then, for each of the P classes, the CPU 410 creates a vertical projection histogram Hv by counting the number of similar pixels among the Q pixels belonging to each class.

  FIG. 11A shows a vertical projection histogram Hv1 that is created when the judgment mask image MK to be processed is the judgment mask image MK1 of FIG. FIG. 11B shows a vertical projection histogram Hv2 created when the judgment mask image MK to be processed is the judgment mask image MK2 of FIG. The vertical axis of the projection histograms Hv1 and Hv2 indicates the pixel position in the vertical direction, and the horizontal axis indicates the number of similar pixels.

  Instead, when creating the projection histogram Hv1, a plurality of pixels on a plurality of continuous lines may be classified into one class. For example, a plurality of pixels in the judgment mask image MK having a size of vertical P pixels × horizontal Q pixels are classified into (P / 2) classes, and the number of pixels belonging to one class is (2 × Q ) It may be a piece.

  In S510, the CPU 410 calculates the size Sv of the circumscribed rectangle Bv of the vertical projection histogram Hv. 11A and 11B show circumscribed rectangles Bv1 and Bv2 of the projection histograms Hv1 and Hv2 in the vertical direction. The size Sv of the circumscribed rectangle Bv of the projection histogram Hv is calculated by multiplying the maximum value Mv among the P count values counted for each of the P classes by the number P of classes (Sv = Mv × P). FIGS. 11A and 11B illustrate that the projection histogram Hv1 takes a maximum value Mv1 at a specific class Cv1, and the projection histogram Hv2 takes a maximum value Mv2 at a specific class Cv2. Yes.

  In S515, the CPU 410 calculates a first evaluation value Ev that is an evaluation value based on the vertical projection histogram Hv. The first evaluation value Ev is a ratio of the number SC of similar pixels in the judgment mask image MK to the size Sv of the circumscribed rectangle Bv of the vertical projection histogram Hv (Ev = (SC / Sv)). The number SC of similar pixels is equal to the total value of the count values of P classes.

  In S520, a horizontal projection histogram Hh of the determination mask image MK is generated. Specifically, the CPU 410 determines a plurality of pixels in the determination mask image MK based on a horizontal position that is a direction intersecting the vertical direction (that is, the horizontal position of the reference area SP and the attention candidate area). And classify them into multiple classes. In the present embodiment, a plurality of pixels having the same horizontal position, that is, a plurality of pixels on a line of one pixel extending in the vertical direction are classified into one class. For example, when the size of the judgment mask image MK is vertical P pixels × horizontal Q pixels, a plurality of pixels in the judgment mask image MK are classified into Q classes and pixels belonging to one class. The number is P. Then, for each of the Q classes, the CPU 410 creates a horizontal projection histogram Hh by counting the number of similar pixels among the P pixels belonging to each class.

  FIG. 11A shows a projection histogram Hh1 in the horizontal direction that is created when the judgment mask image MK to be processed is the judgment mask image MK1 in FIG. FIG. 11B shows a horizontal projection histogram Hh2 created when the judgment mask image MK to be processed is the judgment mask image MK2 of FIG. The horizontal axis of the projection histograms Hh1 and Hh2 indicates the pixel position in the horizontal direction, and the vertical axis indicates the number of similar pixels.

  Instead, when creating the projection histogram Hh1, a plurality of pixels on a plurality of continuous lines may be classified into one class. For example, a plurality of pixels in the judgment mask image MK having a size of vertical P pixels × horizontal Q pixels are classified into (Q / 2) classes, and the number of pixels belonging to one class is (2 × P ) It may be a piece.

  In S525, the CPU 410 calculates the size Sh of the circumscribed rectangle Bh of the horizontal projection histogram Hh. 11A and 11B show circumscribed rectangles Bh1 and Bh2 of the horizontal projection histograms Hh1 and Hh2. The size Sh of the circumscribed rectangle Bh of the projection histogram Hh is calculated by multiplying the maximum value Mh among the Q count values counted for each of the Q classes by the number Q of classes (Sh = Mh × Q). FIGS. 11A and 11B illustrate that the projection histogram Hh1 takes the maximum value Mh1 for a specific class Ch1, and the projection histogram Hh2 takes the maximum value Mh2 for a specific class Ch2. Yes.

  In S530, the CPU 410 calculates a second evaluation value Eh that is an evaluation value based on the horizontal projection histogram Hh. The second evaluation value Eh is a ratio of the number SC of similar pixels in the judgment mask image MK to the size Sh of the circumscribed rectangle Bh in the horizontal projection histogram Hh (Eh = (SC / Sh)). The number SC of similar pixels is equal to the total value of the count values of Q classes.

  In S535, the CPU 410 determines whether or not the first evaluation value Ev is greater than or equal to the threshold value THe and the second evaluation value Eh is greater than or equal to the threshold value THe. The threshold value THe is 90%, for example.

  When the first evaluation value Ev is greater than or equal to the threshold value THe and the second evaluation value Eh is greater than or equal to the threshold value THe (S535: YES), the CPU 410 distributes similar pixels in the judgment mask image MK, that is, the reference It is determined that the distribution of similar pixels in the region SP and the target candidate region is uniform. When the first evaluation value Ev is less than the threshold value THe and the second evaluation value Eh is at least one less than the threshold value (S535: NO), the CPU 410 distributes the similar pixels of the determination mask image MK, that is, the reference It is determined that the distribution of similar pixels in the region SP and the candidate region of interest is biased.

  When the determination mask image MK to be processed is an image in which similar pixels are uniformly distributed, for example, in the case of the determination mask image MK1 in FIG. 11A, in the vertical projection histogram Hv1, P The P count values calculated for each of the classes are relatively close to each other. Therefore, the change of the vertical projection histogram Hv1 of the determination mask image MK1 with respect to the vertical pixel position is relatively small. As a result, since the shape of the vertical projection histogram Hv1 of the determination mask image MK1 is relatively close to the shape of the circumscribed rectangle Bv1, the first evaluation value Ev calculated for the vertical projection histogram Hv1 is a relatively large value. For example, the value tends to be close to 100%. If similar pixels are uniformly distributed in the judgment mask image MK to be processed, the second evaluation value Eh of the horizontal projection histogram Hh is also a relatively large value, for example, 100, for the same reason. It tends to be close to%.

  When the judgment mask image MK to be processed is an image in which similar pixels are distributed unevenly, for example, in the case of the judgment mask image MK2 in FIG. 11B, P in the vertical projection histogram Hv2. There is a high possibility that the P count values calculated for each of the classes include a value significantly smaller than the maximum value Mv2. For example, the count value at the position PL in FIG. 11B is significantly smaller than the maximum value Mv2. Therefore, the change of the vertical projection histogram Hv2 of the determination mask image MK2 with respect to the vertical pixel position is relatively large. As a result, since the shape of the vertical projection histogram Hv2 of the judgment mask image MK2 is different from the shape of the circumscribed rectangle Bv2, the first evaluation value Ev calculated for the vertical projection histogram Hv2 is a relatively small value. Prone. If similar pixels are biased in the judgment mask image MK to be processed, the second evaluation value Eh of the horizontal projection histogram Hh tends to be a relatively small value for the same reason.

  As can be understood from the above description, the first evaluation value Ev is a feature amount indicating the feature of the shape of the vertical projection histogram Hv, and the second evaluation value Eh is the feature of the shape of the horizontal projection histogram Hh. It can be said that it is a feature amount to be shown.

  As can be seen from the above description, in the third actual example, a vertical projection histogram Hv (also referred to as a first histogram) is generated (S505), which is a feature amount indicating the shape of the vertical projection histogram Hv. Using one evaluation value Ev, it is determined whether the distribution of similar pixels is biased or uniform, that is, whether the bias of the distribution of similar pixels is below a reference (S510, S515, S535). As a result, it can be easily determined whether or not the distribution of similar pixels is less than the reference.

  More specifically, a specific value Sv obtained by multiplying the maximum count value Mv counted for each of the plurality of classes and the number of classes is calculated (S510), and similar pixels for the specific value Sv are calculated. The ratio of the number Sc is calculated as the first evaluation value Ev (S515). Then, when the first evaluation value Ev is equal to or greater than the threshold value THe, it can be determined that the distribution deviation of similar pixels is equal to or less than the reference (S535). As a result, it is possible to more appropriately determine whether or not the distribution bias of one of the plurality of similar pixels and the plurality of dissimilar pixels is equal to or less than the reference using the vertical projection histogram Hv. .

  Further, a horizontal projection histogram Hh is generated (S520), and the second evaluation value Eh, which is a feature amount indicating the shape of the horizontal projection histogram Hh, and the above-described first evaluation value Ev are used for similarity. It is determined whether or not the deviation of the pixel distribution is below the reference (S535). As a result, by evaluating the distribution of similar pixels from two directions, it can be determined with higher accuracy whether or not the deviation of the distribution of similar pixels is below the reference.

  In the third embodiment, whether or not the image in the reference area SP and the image in the target candidate area are similar using the feature amounts Ev and Eh indicating the shape characteristics of the projection histograms Hv and Hh. It can be said that it is judging. More specifically, in the third embodiment, among the plurality of pixels in the target candidate region, a plurality of similar pixels similar to the corresponding pixels in the reference region SP are specified (S215 to S215 in FIG. 5). S235). Then, the ratio R1 of similar pixels in the target candidate region is the threshold value TH1 (S250: YES in FIG. 5), and the distribution of similar pixels in the target candidate region is less than the reference value based on the feature amounts Ev and Eh. If it is determined that the reference area SP and the candidate area of interest are similar (S265).

D. Modification (1) In the third embodiment, it is determined whether or not the reference region SP is similar to the target candidate region using the feature amounts Ev and Eh indicating the shape features of the projection histograms Hv and Hh. . Not limited to this, it is possible to determine whether any one image is similar to another image using the feature amounts Ev, Eh indicating the features of the projection histograms Hv, Hh. For example, it may be determined whether two scanned images are similar, or it may be determined whether two photographic images are similar. In particular, an image generated by optically reading an object such as a scanned image or a photographic image tends to include noise pixels. For this reason, when the feature quantities Ev and Eh indicating the features of the projection histograms Hv and Hh are used for the determination of the similarity of these two images, the determination accuracy of the similarity can be further improved.

(2) In the third embodiment, feature amounts Ev and Eh indicating features of the shapes of the two projection histograms Hv and Hh are used. Instead, only one of the two projection histograms Hv and Hh is generated, and only the feature quantity Ev indicating the feature of the shape of the one projection histogram is used, and the deviation of the distribution of similar pixels is the reference value. It may be determined whether or not:

(3) The specific range SR set in S300 of FIG. 7 is not limited to the range based on the ratio R1 of the similar pixels in the entire determination mask image MK. The specific range SR may be, for example, a range greater than or equal to the threshold TH1 used in S250 of FIG. That is, in S320, when the ratio R2 is equal to or greater than the threshold value TH1, it is determined that the attention candidate area is a uniform area. When the ratio R2 is less than the threshold value TH1, the attention candidate area is a biased area. It may be judged.

(4) In S320 to S330 in FIG. 7, whether or not the target divided region is a similar divided region in which the image in the target divided region of the reference region SP and the image in the target divided region of the target candidate region are similar. May be determined. Then, in S340 of FIG. 7, it is determined whether or not the ratio of the similar divided areas to the total number of the divided areas PA is equal to or greater than the threshold value TH3. It may be determined that the distribution deviation of similar pixels in the candidate region is equal to or less than a reference value. A known method may be used to determine whether or not the image in the target divided region of the reference region SP is similar to the image in the target divided region of the target candidate region. For example, two images may be used. A method based on whether or not the histograms are similar may be used.

(5) In S250 of FIG. 5, a primary determination is made as to whether or not the reference region SP is similar to the target candidate region based on the similar pixel ratio R1. Instead, using a known method, for example, a method based on whether or not the histograms of two images are similar, a primary determination is made as to whether or not the reference region SP is similar to the candidate region of interest. It may be done.

(6) In the third embodiment, the evaluation values Ev and Eh are used as the feature quantities indicating the features of the projection histograms Hv and Hh. Instead, the difference value ΔV (Mv−Lv) between the maximum value Mv and the minimum value Lv among the P count values of the vertical projection histogram Hv and the Q counts of the horizontal projection histogram Hh. A difference value ΔH (Mh−Lh) between the maximum value Mh and the minimum value Lh among the values may be used as the feature amount. In this case, for example, when the difference value ΔV is equal to or smaller than a predetermined threshold and the difference value ΔH is equal to or smaller than the predetermined threshold, it may be determined that the distribution deviation of similar pixels is equal to or smaller than the reference. .

(7) The document 10 for generating two pieces of scan data may be read in a mode different from the reading mode described with reference to FIG. FIG. 12 is an explanatory diagram of scan data according to a modification. As shown in FIGS. 12A and 12B, the document 10 may be read by the scanner unit 250 in a state where the document 10 is folded in half along a line CL in FIG. In the first reading in FIG. 12A, left scan data representing the left region 10L is generated, and in the second reading in FIG. 12B, right scan data representing the right region 10R is generated.

  The left original image HIL represented by the two generated scan data and the right original image HIR do not have an area on the original 10 that appears in both the two images HIL and HIR. For this purpose, the CPU 410 uses the two scan data, for example, as shown in FIG. 12C, at least the left scan image IL2b from which the right margin is removed from the left document image HIL, and at least the left scan image. The right-side scan image IR2b from which the margin is removed is generated. Then, the CPU 410 restores a plurality of pixels in the area AL outside the right end of the left original image HIL along the right end of the left original image HIL. The restoration of the pixels in the area AL is performed, for example, by a complementary process using a plurality of pixels in the area inside the left original image HIL and along the right end of the left original image HIL. Similarly, the CPU 410 restores a plurality of pixels in the area AR outside the left end of the right original image HIR along the left end of the right original image HIR. The restoration of the pixels in the area AR is performed, for example, by a complementing process using a plurality of pixels in the area inside the right original image HIR and along the left end of the right original image HIR. The areas AL and AR are very thin linear areas, and the horizontal width ΔW of the areas AL and AR is, for example, 3 pixels. Then, the CPU 410 determines the area AR configured by the restored pixels as the reference area SP, and determines the area including the area AL configured by the restored pixels as the search area SA. Thereafter, the CPU 410 may perform the same process as in the above embodiment. Only one of the areas AL and AR may be restored.

(8) In each actual example described above, various determinations are made based on similar pixels, but various determinations may be made based on dissimilar pixels. For example, in S250 of FIG. 5, it is determined whether or not the similar pixel ratio R1 is greater than or equal to the threshold TH1, but the dissimilar pixel ratio (1-R1) is less than a predetermined threshold (1-TH1). It may be determined whether or not there is. In S320 of FIG. 7, it is determined whether or not the similar pixel ratio R2 is within the specific range SR ((R1−ΔR) ≦ R2 ≦ (R1 + ΔR)), but the dissimilar pixel ratio (1−R2). May be determined within a specific range ((1-R1-ΔR) ≦ (1-R2) ≦ (1-R1 + ΔR)). Further, in the distribution determination process of FIG. 10, a projection histogram of dissimilar pixels is generated, and for the projection histogram of dissimilar pixels, (number of dissimilar pixels / size of circumscribed rectangle of projection histogram of dissimilar pixels) is It may be used as an evaluation value.

(9) In the first and second embodiments, a plurality of divided areas PA are set in the judgment mask image MK. As described above, this is equivalent to setting the divided area PA in the reference area SP and the target candidate area. In processing, the reference area SP may be divided to set the divided area PA, or the attention candidate area may be divided to set the divided area PA.

(10) In the above embodiment, arranged image data representing an arranged image in which the left original image HIL and the right original image HIR are arranged in the horizontal direction is generated. Instead of this, arranged image data representing an arranged image in which one original image and another original image are arranged in the vertical direction may be generated. In this case, for example, the reference area SP is determined along the lower end of one original image, and the search area SA is determined along the upper end of another original image.

(11) In the arrangement position determination process of FIG. 5, the CPU 410 has a similar pixel ratio R1 of one candidate region of interest equal to or greater than the threshold TH1 (S250: YES), and the distribution determination process determines the distribution of similar pixels. If it is determined that the region is uniform (S260: YES), the candidate region of interest is determined as a similar region CP (S265), and the remaining candidate candidate regions are not determined. Instead, the CPU 410 may determine whether or not the ratio R1 is greater than or equal to the threshold value TH1 for all attention candidate areas, and may determine a plurality of attention candidate areas whose ratio R1 is greater than or equal to the threshold value TH1. Then, the CPU 410 performs distribution determination processing one by one in descending order of the ratio R1 for a plurality of target candidate areas whose ratio R1 is greater than or equal to the threshold TH1, and the candidate candidates determined that the distribution of similar pixels is uniform. The region may be determined as the similar region CP. Alternatively, a distribution determination process is performed for each of a plurality of attention candidate areas whose ratio R1 is equal to or greater than the threshold TH1, and among the plurality of attention candidate areas, the attention candidate area having the most uniform distribution of similar pixels is similar. The region CP may be determined. The attention candidate region having the most uniform distribution of similar pixels is, for example, the attention candidate region having the largest uniform region ratio R3 or the attention candidate region having the largest sum of the first evaluation value Ev and the second evaluation value Eh. is there.

(12) In the above embodiment, arranged image data representing an arranged image in which two document images are arranged is generated using two image data. However, the present invention is not limited to this, and one arranged image data may be generated using an arbitrary number of image data. For example, arranged image data representing an arranged image in which four document images are arranged may be generated using four image data.

(13) In the above embodiment, the two pieces of image data used for generating the arranged image data are generated by the scanner unit 250 of the multi-function device 200. The present invention is not limited to this, and various image data representing an optically read image can be employed. For example, two pieces of image data may be generated by optically reading both sides of the document 10 folded in half by photographing with a digital camera. Further, these image data are not limited to image data generated by a reading device (scanner or digital), but may be image data generated using an application program such as drawing creation or document creation.

(14) The processing (for example, the processing of S25 to S50 in FIG. 2) executed by the CPU 410 of the server 400 in the above embodiment may be executed by the CPU 210 of the multifunction device 200, for example. In this case, the server 400 is not necessary in this case, and the multi-function device 200 may execute the processing of FIG. 2 alone. Further, the processing executed by the CPU 410 of the server 400 may be executed by the CPU (not shown) of the personal computer 500 (FIG. 1) connected to the multifunction device 200. For example, the CPU of the personal computer 500 may execute these processes by executing a scanner driver program installed in the personal computer 500. Further, the server 400 may be configured by one computer as in the present embodiment, or may be configured by a computer system including a plurality of computers that can communicate with each other.

(15) In the above embodiment, the server 400 acquires the scan data in the image file format in S25, and outputs (transmits) the arranged image data in the image file format in S55. Instead, for example, when the CPU 210 of the multifunction device 200 executes the processes of S25 to S50 as in the above-described modification, the CPU 210 converts the scan data generated using the scanner unit 250 into an image file. You may acquire as it is, without converting. Further, the CPU 210 may output (specifically, print or the like) as it is without converting the arranged image data into an image file.

(16) In the above embodiment, a part of the configuration realized by hardware may be replaced with software, and conversely, a part of the configuration realized by software may be replaced with hardware. Also good.

  As mentioned above, although this invention was demonstrated based on the Example and the modification, Embodiment mentioned above is for making an understanding of this invention easy, and does not limit this invention. The present invention can be changed and improved without departing from the spirit and scope of the claims, and equivalents thereof are included in the present invention.

  70 ... Internet, 80 ... LAN, 100 ... Printer, 200 ... Multifunction device, 210 ... CPU, 220 ... Volatile storage device, 221 ... Buffer area, 230 ... Non-volatile storage device, 231 ... control program, 240 ... printer unit, 250 ... scanner unit, 255 ... original table, 260 ... operation unit, 270 ... display unit, 280. ..Communication unit, 400 ... server, 410 ... CPU, 420 ... volatile storage device, 421 ... buffer area, 430 ... nonvolatile storage device, 431 ... computer program, 433 ... UI data group, 480 ... communication unit, 500 ... personal computer, 1000 ... image processing system

Claims (12)

An image for obtaining first image data representing a first image showing a part of one object and second image data representing a second image showing another part of the object. An acquisition unit;
A reference area determination unit that determines a reference area that is a partial area in the first image;
A first determination unit that determines whether the reference region is similar to one candidate region of the plurality of candidate regions in the second image;
When the reference region and the one candidate region are similar, the reference region and the one candidate region among a plurality of pixels in one region of the reference region and the one candidate region Among the plurality of similar pixels that are similar to the corresponding pixel in the other region and the plurality of dissimilar pixels that are not similar to the corresponding pixel. A second determination unit to
When the bias of the distribution of one of the plurality of similar pixels and the plurality of dissimilar pixels is equal to or less than a reference, the first image and the one candidate region are based on the reference region and the one candidate region. A position determining unit that determines a relative position with respect to the second image;
The first image and the second image are arranged at the determined relative position, and one arranged already representing the processed image showing the object by the first image and the second image An image generation unit for generating image data;
An image processing apparatus comprising: The image processing apparatus according to claim 1,
The second determination unit includes:
Set a plurality of divided areas to divide the one area,
For each of the plurality of divided regions, it is determined whether or not the number of one of the similar pixels and the dissimilar pixels included therein is a specific divided region within a specific range,
Using a first feature amount based on the number of the specific divided areas, it is determined whether or not a distribution bias of one of the plurality of similar pixels and the plurality of dissimilar pixels is equal to or less than a reference. , Image processing device. The image processing apparatus according to claim 2,
The first determination unit determines whether the reference region and the one candidate region are similar using the number of one of the similar pixels and the dissimilar pixels,
The second determination unit is an image processing apparatus that determines whether or not a distribution bias of one of the similar pixel and the dissimilar pixel used by the first determination unit is equal to or less than a reference. The image processing apparatus according to claim 2, wherein:
When the first ratio indicating the number of similar pixels with respect to the number of pixels in the reference region is equal to or greater than a first threshold, the first determination unit is similar to the reference region and the one candidate region. Judging
In the case where the second ratio indicating the number of the similar pixels to the number of pixels in the target divided area among the plurality of divided areas is within the specific range based on the first ratio. In addition, the image processing apparatus determines that the target divided area is the specific divided area. The image processing apparatus according to any one of claims 2 to 4,
The second determination unit includes:
Within the one area, the relatively small divided area is set in an area where the pixel value variation is relatively large, and the relatively large divided area is set in an area where the pixel value variation is relatively small. And
An image processing apparatus that calculates the first feature amount by counting the number of the specific divided areas of the plurality of divided areas without depending on the size of the specific divided areas. The image processing apparatus according to claim 1,
The second determination unit includes:
The plurality of pixels in the one region are classified into a plurality of first classes based on positions in a first direction, and the plurality of similar pixels and the plurality of similar pixels for each of the plurality of first classes Generating a first histogram obtained by counting the number of one of a plurality of dissimilar pixels;
Whether or not a distribution bias of one of the plurality of similar pixels and the plurality of dissimilar pixels is equal to or less than a reference by using a second feature amount indicating a feature of the shape of the first histogram. An image processing apparatus for determining The image processing apparatus according to claim 6,
The second determination unit includes:
Calculating a first value obtained by multiplying the maximum count value counted for each of the plurality of first classes and the number of the plurality of first classes;
Calculating a ratio of the total value of the count value to the first value as the second feature amount;
An image processing apparatus that, when the second feature amount is equal to or greater than a threshold value, determines that a deviation in distribution of one of the plurality of similar pixels and the plurality of dissimilar pixels is equal to or less than a reference. The image processing apparatus according to claim 6 or 7, wherein
The second determination unit includes:
A plurality of pixels in the one region are classified into a plurality of second classes based on positions in a second direction intersecting the first direction, and each of the plurality of second classes is classified. Generating a second histogram obtained by counting the number of one of the similar pixels and the dissimilar pixels;
Using the second feature amount indicating the shape of the first histogram and the third feature amount indicating the shape feature of the second histogram, the plurality of similar pixels and the plurality of the plurality of similar pixels An image processing apparatus that determines whether or not a deviation in distribution of one of dissimilar pixels is equal to or less than a reference. An image processing apparatus,
An image acquisition unit that acquires first image data representing a first image and second image data representing a second image;
Of the plurality of pixels in the first image, one of a plurality of similar pixels that are similar to the corresponding pixels in the second image and a plurality of dissimilar pixels that are not similar to the corresponding pixels. A pixel specifying part to be specified;
A plurality of pixels in the first image are classified into a plurality of first classes based on positions in a first direction, and the plurality of similar pixels and the plurality of similar pixels for each of the plurality of first classes A histogram generator for generating a first histogram obtained by counting one of the plurality of dissimilar pixels;
A similarity determination unit that determines whether or not the first image and the second image are similar by using a first feature amount indicating a feature of the shape of the first histogram;
An image processing apparatus comprising: The image processing apparatus according to claim 9,
The similarity determination unit
Calculating a first value obtained by multiplying the maximum count value counted for each of the plurality of first classes and the number of the plurality of first classes;
Calculating a ratio of the number of one of the similar pixels and the non-similar pixels to the first value as the first feature amount;
An image processing apparatus that determines that the first image and the second image are similar when the first feature amount is equal to or greater than a threshold value. The image processing apparatus according to claim 9 or 10, wherein:
The similarity determination unit
The plurality of pixels in the first image are classified into a plurality of second classes based on positions in a second direction intersecting the first direction, and each of the plurality of second classes Generating a second histogram obtained by counting the number of one of the similar and non-similar pixels for
Using the first feature amount indicating the shape of the first histogram and the second feature amount indicating the shape feature of the second histogram, the first image and the second image are used. An image processing apparatus that determines whether or not the two are similar to each other. A computer program,
An image for obtaining first image data representing a first image showing a part of one object and second image data representing a second image showing another part of the object. An acquisition unit;
A reference area determination function for determining a reference area which is a partial area in the first image;
A first determination function for determining whether the reference region is similar to one candidate region of the plurality of candidate regions in the second image;
When the reference area and the one candidate area are similar, out of a plurality of pixels in one area of the reference area and the one candidate area, the reference area and the one candidate area It is determined whether the distribution bias of one of a plurality of similar pixels similar to the corresponding pixel in the other region and a plurality of dissimilar pixels not similar to the corresponding pixel is equal to or less than a reference. A second determination function;
When the bias of the distribution of one of the plurality of similar pixels and the plurality of dissimilar pixels is equal to or less than a reference, the first image and the one candidate region are based on the reference region and the one candidate region. A position determining function for determining a relative position with respect to the second image;
The first image and the second image are arranged at the determined relative position, and one arranged already representing the processed image showing the object by the first image and the second image An image generation function for generating image data;
A computer program that causes a computer to realize

Images