JP2017011523A - Image processing device and computer program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To reduce the processing time when arranged image data representing one object by a first object image and a second object image are generated.SOLUTION: An image processing device determines whether or not the number of first pixels contained between an end in a first direction of a first target image and an end in a first direction of a first image containing the first target image is equal to or more than a reference, determines whether or not the number of second pixels contained between an end in a second direction of the first target image and an end in a second direction of the first image is equal to or more than the reference, and executes first search processing for searching a first type arrangement position where an end image along one end of the first target image overlaps with an end image along one end of the second target image, thereby determining arrangement positions of the two target images. When the number of the first pixels is less than the reference, the first search processing is executed using the end image along the end in the first direction of the first target image and at least one of the end images along the first and second directions of the second target image.SELECTED DRAWING: Figure 9

Description

  In this specification, the first target image and the second target image are arranged, and the arranged image data representing the arranged image showing one object by the first target image and the second target image. It relates to the technology to generate.

  Patent Document 1 discloses a technique for generating image data representing an image obtained by joining a plurality of scanned originals using original data obtained by sequentially scanning a plurality of originals to be joined. It is disclosed. In this technique, a combination process is executed for a combination of a plurality of sides that can connect the originals, and the originals are connected at the sides having the same characteristics. Alternatively, in this technique, when a user inputs a procedure for joining documents, the joining of documents is executed according to the procedure.

JP-A-6-164894

  However, in the above technique, for example, when a document joining procedure is not input, a joining process is performed for each of combinations of a plurality of sides that can be joined. Therefore, a process for generating image data is performed. The time could be excessively long.

  In the present specification, when the first image and the second image are arranged, and the first target image and the second target image are used to generate image data representing an arranged image indicating one target object. Disclosed is a new technology that can reduce the processing time.

  The technology disclosed in this specification can be implemented as the following application examples.

Application Example 1 Image acquisition for acquiring first image data representing a first image including a first target image and second image data representing a second image including a second target image A first pixel included in the first direction of the first target image and the end of the first image in the first direction is greater than or equal to a reference. A second determination unit that is included between the first determination unit that performs the second determination and the second direction end opposite to the first direction of the first target image and the second direction end of the first image. A second determination unit that determines whether or not the number of pixels is equal to or greater than a reference, and an end image along one end of the first direction and the second direction of the first target image; A first search is made for a first type arrangement position where the first image of the second target image and an end image along one end of the second direction overlap. By executing a search process, a first position determination unit that determines a relative arrangement position between the first target image and the second target image, and the first position determination unit at the determined arrangement position An image generation unit configured to generate arranged image data representing a arranged image in which a target image and the second target image are arranged, and the first target image and the second target image represent one target object; The first position determination unit, when the number of the first pixels is less than a reference, the end image along the end in the first direction of the first target image, Using at least one of an end image along an end in the first direction of the second target image and an end image along an end in the second direction of the second target image; And when the number of the first pixels is equal to or greater than a reference, the first target image When the first search process is executed without using the end image along the end in the first direction, and the number of the second pixels is less than the reference, the first of the first target image An end image along the end in two directions, an end image along the end in the first direction of the second target image, and an end image along the end in the second direction of the second target image. When the first search process is executed using at least one of the two and the number of the second pixels is equal to or larger than a reference, the first target image is along the end in the second direction. An image processing apparatus that executes the first search process without using an end image.

  According to the above configuration, when the number of first pixels is greater than or equal to the reference, the first search process is executed without using the end image along the end in the first direction of the first target image, When the number of the second pixels is equal to or larger than the reference, the first search process is executed without using the edge image along the edge in the second direction of the first target image. As a result, the processing time of the first search process can be reduced. Therefore, it is possible to reduce the processing time when generating image data representing the arranged image.

Application Example 2 Image acquisition for acquiring first image data representing a first image including a first target image and second image data representing a second image including a second target image Each of a plurality of combinations of a portion, any one of a plurality of ends of the first target image, and any one of a plurality of ends of the second target image A histogram generation unit that generates a histogram of two end images along two ends constituting a combination, and the histogram is used to relatively high the similarity of the histograms of the two end images. Order determination for determining the order of the plurality of combinations such that the order of the search processing for the combinations precedes the order of the search processing for the combinations having a relatively low similarity between the histograms of the two end images. Part The search process is a process of searching for a position that associates an end image along one end of the first target image with an end image along one end of the second target image. The order determination unit, a position determination unit that determines a relative arrangement position of the first target image and the second target image by executing the search process, and the determined arrangement position The first target image and the second target image are arranged, and arranged image data representing a arranged image showing one object is generated by the first target image and the second target image. An image generator;
The position determination unit executes the search process for the plurality of combinations according to the determined order, and determines the arrangement position for a specific combination among the plurality of combinations. In addition, the image processing apparatus does not execute the search process for a combination after the determined order after the specific combination among the plurality of combinations.

  According to the above configuration, the search process is executed according to the order determined so that the order of the search process with respect to the combination having a high degree of similarity between the histograms of the two edge images is first, and the arrangement position of the specific combination is determined. When it is determined, the search process is not executed for the combination whose order determined after the specific combination is a plurality of combinations, so that the processing time of the first search process can be reduced. Therefore, it is possible to reduce the processing time when generating image data representing the arranged image.

  The present invention can be realized in various forms, for example, an image processing method and an image processing apparatus, a computer program for realizing the function of the method or apparatus, and a recording in which the computer program is recorded. It can be realized in the form of a medium (for example, a recording medium that is not temporary).

It is a block diagram which shows the structure of an image processing system. FIG. 11 is a sequence diagram showing an operation of the image processing system 1000. It is a figure which shows an example of a manuscript. 6 is a first explanatory diagram of a scan pattern A. FIG. FIG. 10 is a second explanatory diagram of a scan pattern A. It is a figure which shows an example of a scanned image. It is a figure which shows an example of the arranged image. It is a flowchart of a margin amount determination process. It is a flowchart of the process target determination process of 1st Example. It is a flowchart of the arrangement position determination process of 1st Example. It is a flowchart of a 1st search process. It is explanatory drawing of a 1st search process. 6 is an explanatory diagram of a scan pattern B. FIG. 6 is a diagram illustrating an example of a scan image generated with a scan pattern B. FIG. It is a flowchart of the 2nd search process. It is explanatory drawing of a 2nd search process. It is a flowchart of a pixel restoration process. FIG. 17 is an enlarged view near the reference range REb in FIG. It is a flowchart of the process target determination process of 2nd Example. It is a flowchart of the arrangement position determination process of 2nd Example. It is a flowchart of the arrangement position determination process of 3rd Example. It is a flowchart of the process order determination process of 3rd Example. It is a figure which shows an example of the histogram of R component of two edge part images. It is a flowchart of the arrangement | positioning order determination process of 4th Example.

A. First embodiment:
A-1: Configuration of Image Processing System 1000 FIG. 1 is a block diagram showing the configuration of the image processing system. The image processing system 1000 includes a server 400 as an image processing apparatus and a multifunction device 200. The server 400 is connected to the Internet 70, and the multifunction device 200 is connected to the Internet 70 via a LAN (abbreviation of local area network) 80. As a result, the server 400 and the multifunction device 200 can communicate with each other via the LAN 80 and the Internet 70. The personal computer 500 which is a user terminal device may be connected to the LAN 80.

  The server 400 includes a CPU 410 as an example of a controller of the server 400, a volatile storage device 420 such as a DRAM, a nonvolatile storage device 430 such as a hard disk drive and a flash memory, and an interface for connecting to a network such as the Internet 70. Including a communication unit 480. The volatile storage device 420 is provided with a buffer area 421 for temporarily storing various intermediate data generated when the CPU 410 performs processing. The nonvolatile storage device 430 stores a computer program 431 and a UI data group 433.

  The computer program 431 and the UI data group 433 are installed in the server 400 by being uploaded to the server 400 via the Internet 70 by an administrator of the server 400, for example. Alternatively, the computer program 431 and the UI data group 433 may be provided in a form stored in, for example, a DVD-ROM, and may be installed in the server 400 by the administrator of the server 400. The CPU 410 implements image processing to be described later by executing the computer program 431.

  The multifunction device 200 includes a CPU 210 as an example of a controller of the multifunction device 200, a volatile storage device 220 such as a DRAM, a nonvolatile storage device 230 such as a flash memory and a hard disk drive, a printer unit 240, and a scanner unit 250. , An operation unit 260 such as a touch panel and buttons, a display unit 270 such as a liquid crystal display, and a communication unit 280 that communicates with an external device. For example, the communication unit 280 includes an interface for connecting to a network such as the LAN 80 and an interface for connecting to an external storage device such as a USB memory.

  The volatile storage device 220 is provided with a buffer area 221 for temporarily storing various data generated when the CPU 210 performs processing. A control program 231 is stored in the non-volatile storage device 230. The control program 231 may be provided by being stored in advance in the non-volatile storage device 230 at the time of manufacture of the multifunction device 200. Instead, the control program 231 can be provided, for example, in a form downloaded from a server connected via the Internet, or in a form recorded on a CD-ROM or the like.

  The printer unit 240 executes printing using a printing method such as an inkjet method or a laser method. The scanner unit 250 generates scan data representing a color image or a gray image by optically reading a document using a photoelectric conversion element such as a CCD or a CMOS. The scanner unit 250 includes a so-called flatbed type document table 255 described later.

  The CPU 210 executes control of the multifunction device 200 by executing the control program 231. For example, the CPU 210 controls the printer unit 240 and the scanner unit 250 to execute a copy process, a print process, a scan process, and the like. Further, the CPU 210 can access the server 400 and use a service provided by the server 400.

A-2: Operation of Image Processing System 1000 FIG. 2 is a sequence diagram showing the operation of the image processing system 1000. The processing in the sequence diagram is started when the multifunction device 200 receives an instruction to use an image generation service provided by the server 400 from a user. Although this image generation service will be described in detail later, a plurality of images represented by a plurality of scan data are arranged, and one object (specifically, a manuscript 10 described later) is composed of a plurality of images. This is a service for generating arranged image data representing an arranged image indicating the above. The details of the plurality of scan data are described later. For example, the plurality of scan data are generated by reading a document having a size larger than the size that can be read at one time in a plurality of times.

  When the process is started, in S5, the CPU 210 of the multifunction device 200 transmits a service start request to the server 400. Upon receiving the service start request, the CPU 410 of the server 400 selects UI data necessary for providing the image generation service from the UI data group 433 shown in FIG. 1, and the UI data is sent to the multifunction device 200 in S10. To send. Specifically, the UI data includes screen data representing a user interface screen (hereinafter also referred to as UI screen) and control data. This control data includes, for example, various data necessary for the MFP 200 to perform predetermined processing such as the scan processing in S15 described later using the UI screen. For example, the control data includes information necessary for performing processing to be executed by the multifunction device 200, for example, a transmission destination address of an image file. The process to be executed by the multifunction device 200 includes, for example, a process of transmitting an image file to the server 400 in S20 described later.

  In S15, the CPU 210 executes a scan process for generating a plurality of scan data based on the received UI data. In the scan process, the CPU 210 generates two pieces of scan data by reading the document prepared by the user in two steps. The scan data of this embodiment is, for example, RGB image data including component values of RGB components represented by 256 gradation values from 0 to 255 for each pixel.

  FIG. 3 is a diagram illustrating an example of a document. The size of the document 10 is A3 size in this embodiment. The line CL is a line that is located in the center of the A3 size original 10 in the longitudinal direction and is parallel to the short side of the original 10. The left area 10L is a left half area of the document 10, that is, an area on the left side of the line CL. The right region 10R is a half region on the right side of the document 10, that is, a region on the right side of the line CL of the document 10. The size of the left region 10L and the size of the right region 10R are A4 size. The A3 size and the A4 size are paper dimensions defined by ISO (abbreviation of International Organization for Standardization) 216.

  The arrows with the reference numerals 10L and 10R in FIG. 3 indicate the positions and orientations of the regions 10L and 10R. That is, it means that these regions 10L and 10R are arranged at the positions where these arrows are arranged and in the directions indicated by these arrows. In the figure (for example, FIG. 4) mentioned later, illustration of the image in area | region 10L, 10R is abbreviate | omitted, and the position and orientation of these area | regions 10L, 10R may be illustrated only with a signed arrow.

  4 and 5 are explanatory diagrams of the scan pattern A. FIG. The length in the longitudinal direction of the document table 255 (FIGS. 4A and 4B) of the scanner unit 250 is slightly longer (for example, several centimeters) than 297 mm, which is the length in the longitudinal direction of the A4 size. The length of the manuscript table in the short-side direction is the length of the letter-size short-side direction, which is the paper size defined by ANSI / ASME (Abbreviation of American National Standards Institute / American Society of Mechanical Engineers) Y14.1. It is slightly longer (for example, several centimeters) than the length 215.9 mm. That is, in this embodiment, the maximum size of a document that can be read at one time is larger than the A4 size and smaller than the A3 size. Therefore, in this embodiment, the A3 size original 10 is read in two steps.

  In the scanning process of the first embodiment, the scanning pattern A in FIGS. 4 and 5 is used to perform two readings to generate two pieces of scan data.

(Scan Pattern A)
In the scan pattern A, the document 10 is placed on the document table 255 without being bent. On the other hand, as shown in FIG. 4, the scan pattern A is read twice, and an image in an area including the left area 10L and a partial area CAR in contact with the left area 10L in the right area 10R (hereinafter, left side). (Also referred to as a document image AIL) is read, and left scan data indicating the left document image AIL is generated. On the other side of the two scan pattern A readings, as shown in FIG. 5, an image in an area including the right area 10R and a partial area CAL in contact with the right area 10R in the left area 10L (hereinafter, right side) (Also referred to as a document image AIR) is read, and right scan data indicating the right document image AIR is generated. Which of the left original image AIL and the right original image AIR is read first depends on the user, and the multi-function device 200 and the server 400 cannot recognize which was read first.

  There are two types of modes shown in FIGS. 4A and 4B for reading the left original image AIL. Which mode is used depends on the user, and the multifunction device 200 and the server 400 cannot recognize the used mode. In aspect 1, as shown in FIG. 4A, the user faces the document surface on which the image of the document 10 is placed facing the document table 255 side, and the left area 10L of the document surface is the back side (FIG. The document 10 is placed on the document table 255 so that the right area 10R of the document surface is positioned on the near side (lower side in FIG. 4A) on the upper side of A). At this time, since the document 10 is arranged on the document table 255 so that the entire left region 10L is positioned on the document table 255, at least a part of the right region 10R is on the front side of the document table 255. Stick out.

  In mode 2, as shown in FIG. 4 (B), the user faces the document table 255 on which the image of the document 10 is placed, and the left area 10L of the document surface is on the near side (FIG. 4 (B). The document 10 is placed on the document table 255 so that the right area 10R of the document surface is located on the back side (upper side in FIG. 4B). At this time, since the document 10 is arranged on the document table 255 so that the entire left region 10L is positioned on the document table 255, at least a part of the right region 10R is located behind the document table 255. Stick out.

  FIGS. 4C to 4F show an example of the left side scan image represented by the left side scan data generated in the reading modes 1 and 2 of the left side original image AIL. These left-side scan images ALa to ALd and right-side scan images ARa to ARb described later have a rectangle corresponding to the shape of the document table 255. These scanned images are arranged on the document table 255 with the direction along the long side of the document table 255 as the vertical direction and the left direction of the document table 255 in FIGS. 4A and 4B as the upper direction. Represents an image. The upward direction of the document 10 with respect to the document table 255 in the mode 1 (the direction of the arrow in FIG. 4A) and the upward direction of the document 10 with respect to the document table 255 in the mode 2 (the direction of the arrow in FIG. 4B). , Reverse. Therefore, the left original image AIL in the left scan images ALa and ALb (FIGS. 4C and 4D) generated in the mode 1 and the left scan images ALc and ALd generated in the mode 2 (FIG. 4). The left and right original images AIL in (E) and (F)) are upside down.

  In these left-side scan images ALa to ALd, the margin MG does not appear at least at the end corresponding to the partial area CAR near the center of the document 10 among the left and right ends. This is because, as described with reference to FIGS. 4A and 4B, the right region 10 </ b> R of the document 10 protrudes to one side of the document table 255 during reading.

  In the left-side scan images ALa to ALd, there are cases where a margin MG appears at the end corresponding to the left end of the document 10 among the left and right ends, and when the margin MG does not appear. For example, in aspect 1 (FIG. 4A), between the left end of the document 10 (the upper end of FIG. 4A) and the corresponding end of the document table 255 (the upper end of FIG. 4A), When there is no gap, the margin MG does not appear at the end corresponding to the left end of the document 10 as in the left-side scan image ALa in FIG. On the other hand, when there is a gap, a margin MG appears at the end corresponding to the left end of the document 10 as in the left-side scan image ALb in FIG. The same applies to aspect 2 (FIG. 4B) as shown in FIGS. 4E and 4F.

  As with the left original image AIL, there are two types of modes shown in FIGS. 5A and 5B for reading the right original image AIR. Which mode is used depends on the user, and the multifunction device 200 and the server 400 cannot recognize the used mode. In aspect 1 of the right original image AIR, as shown in FIG. 5A, the original 10 is arranged on the original table 255 in the same direction as in aspect 1 of the left original image AIL (FIG. 4A). . At this time, since the document 10 is arranged on the document table 255 so that the entire right region 10R is positioned on the document table 255, at least a part of the left region 10L is located behind the document table 255. Stick out.

  In aspect 2 of the right original image AIR, in aspect 2 of the right original image AIR, as shown in FIG. 5B, the original 10 is oriented in the same direction as in aspect 2 of the left original image AIL (FIG. 4B). Is placed on the document table 255. At this time, since the document 10 is arranged on the document table 255 so that the entire right region 10R is positioned on the document table 255, at least a part of the left region 10L is on the front side of the document table 255. Stick out.

  FIGS. 5C to 5F show an example of the right side scan image represented by the right side scan data generated in the modes 1 and 2 for reading the right side original image AIR. Similarly to the left-side scan images ALa to ALd described above, the right-side original image AIR in the right-side scan images ARa and ARb (FIGS. 5C and 5D) generated in the mode 1 and the right-side generated in the mode 2 The right and left original image AIR in the scanned images ARc and ARd (FIGS. 5E and 5F) is upside down. Similarly to the left-side scan images ALa to ALd described above, in the right-side scan images ARa to ARd, the margin MG does not appear at least at the end corresponding to the partial area CAL near the center of the document 10, There are cases where a margin MG appears at the end corresponding to the right end of the document 10 and cases where no margin MG appears.

  As can be seen from the above description, the scan image (left scan image AL or right scan image AR) generated by the scan pattern A is a double-sided marginless image (for example, FIG. ), (E) images ALa and ALc, and images ARa and ARc in FIGS. 5C and 5E), and there is no margin MG at one of the left and right ends, and a margin at the other end. One side marginal image with MG (for example, images ALb and ALd in FIGS. 4D and 4F and images ARb and ARd in FIGS. 5D and 5F).

  The scan data generated by the first reading in S15 of FIG. 2 is referred to as first scan data, and the scan data generated by the second reading is second scan data. As can be seen from the above description, the first scan data is one of the left scan data and the right scan data, and the second scan data is the other of the left scan data and the right scan data. In the scan process of S15, the first scan data and the second scan data are each converted into a file of a predetermined format, for example, an image file of JPEG (Joint Photographic Experts Group) format.

  In S <b> 20 of FIG. 2, the CPU 210 transmits the image file of the first scan data (also referred to as the first image file) and the image file of the second scan data (also referred to as the second image file) to the server 400. To do. As a result, in S25, the CPU 410 of the server 400 acquires these two image files. The CPU 410 acquires first scan data and second scan data from these image files and stores them in the buffer area 421. For example, when an image file in JPEG format is used, scan data converted into RGB image data is acquired from the image file by a predetermined conversion process and stored in the buffer area 421.

  FIG. 6 is a diagram illustrating an example of first and second scan images represented by acquired first and second scan data. In this example, the first scan image I11 in FIG. 6A is a left-side scan image generated by the aspect 1 in FIG. 4A, and the second scan image I21 in FIG. It is a right side scan image produced | generated by the aspect 2 of (B). Therefore, the first scan image I11 includes the left original image AIL and the margin MG outside the left original image AIL. The second scanned image I21 includes a right original image AIR and a margin MG outside the right original image AIR. Since the margin MG is an area obtained by reading the document table 255, it has a value (RGB value) indicating the color of the document table 255. The color of the margin MG is white in this embodiment, but may be black or gray. In the scan images I11 and I21, the document images AIL and AIR are inclined because the document 10 placed on the document table 255 is inclined when the scan data is generated.

  In S30, the CPU 410 performs an inclination correction process on the first scan data and the second scan data, respectively. Specifically, the CPU 410 performs known processing such as Hough transform and edge detection processing on the scan data to be processed, and the upper end of the document image (AIL or AIR) in the scan image (I11 or I21). Is detected. The CPU 410 rotates the scan image (I11 or I21) so that the upper end of the original image (AIL or AIR) and the upper end of the scan image (I11 or I21) are parallel to each other, and the corrected scan image is corrected. The corrected scan data representing is generated. Here, the end of the document image to be detected may be the lower end instead of the upper end, but is preferably not the right end or the left end. This is because one of the left and right edges of the document image (AIL or AIR) in the scanned image (I11 or I12) does not represent the edge of the document 10.

  In the margin amount determination process to be described later, since a margin line is determined for each vertical line, the original image is inclined in the scan image to be processed by executing the inclination correction in advance as described above. In addition, it is possible to accurately determine whether the margin amount is equal to or larger than the reference amount by the subsequent margin amount determination process.

  FIG. 6B shows corrected first scan image I12 and second scan image I22. In the corrected scanned images I12 and I22, an additional area AA constituted by additional pixels is added outside the original image (AIL or AIR) and the margin MG. This is because, for convenience of processing, the corrected scanned images I12 and I22 are handled as rectangular images whose upper and lower ends are parallel to the horizontal direction and whose left and right ends are parallel to the vertical direction. The CPU 410 generates a mask image (not shown) in which the pixels in the additional area AA and the pixels in the other area are binarized so that the additional area AA can be identified separately from the other areas. And stored in the buffer area 421. Hereinafter, the first and second scan data that have been subjected to the tilt correction processing are also simply referred to as first scan data and second scan data.

  In S35 of FIG. 2, the CPU 410 executes a margin amount determination process at the right end of the first scan image I12. In S40, the CPU 410 executes a margin amount determination process at the left end of the first scan image I12. In S45, the CPU 410 executes a margin amount determination process at the right end of the second scan image I22. In S50, the CPU 410 executes a margin amount determination process at the left end of the second scan image I22. These margin amount determination processes are processes for determining whether or not the margin amount (the margin amount at the right end or the left end) at the processing target end of the scan image to be processed is greater than or equal to the reference. Specifically, the processing target of the scanned image from the processing target end (right end or left end) of the original image (AIL or AIR) in the processing target scanning image of the scanned images I12 and I22 in FIG. 6B. It is determined based on a predetermined determination condition whether or not the amount of blank space up to the edge is equal to or greater than a reference. The margin amount determination process will be described later.

  In S55 of FIG. 2, the CPU 210 executes a process target determination process. The processing target determination process is selected from the first end selected from the left end LT1 and the right end RT1 of the document image in the first scan image I12, and the left end LT2 and the right end RT2 of the document image in the second scan image I22. Of determining a combination of two ends to be processed in a search process (described later) for determining an arrangement position in the arrangement position determination process of S65, among a plurality of combinations of the second end to be performed It is. In the example of the scan images I12 and I22 in FIG. 6B, the right end RT1 of the document image in the first scan image I12 and the left end LT2 of the document image in the second scan image I22 are joined to each other. It is clear that 10 can be reproduced. However, in reality, one of the first scan image I12 and the second scan image I22 may be one of the left-side scan images ALa to ALd in FIGS. There is a possibility that any of the right-side scan images ARa to ARd in FIGS. Therefore, any one of the left end LT1 and the right end RT1 of the document image in the first scan image I12, and any one of the left end LT2 and the right end RT2 of the document image in the second scan image I22; In this stage, the CPU 410 cannot recognize whether it is appropriate to join the end portions in order to reproduce the document 10. In the process target determination process, a combination of two ends to be processed is determined using the result of the blank amount determination process in S35 to S50. The process target determination process will be described later.

  In S65, the CPU 410 executes an arrangement position determination process. In the arrangement position determination process, a first search process (described later) for determining the first type arrangement position is executed, and the two original images AIL and AIR that can appropriately represent the original 10 are relative to each other. This is a process for determining the correct position. The first type arrangement position is a relative position where a part of the left original image AIL and a part of the right original image BIR overlap. The arrangement position determination process is executed using the result of the process target determination process. The arrangement position determination process will be described later.

  In S70, the CPU 410 executes a placed image generation process. In the arranged image generation process, arranged image data representing the arranged image in which the left original image AIL and the right original image AIR are arranged is generated using the first scan data and the second scan data. . In the arranged image, the right original image AIR and the left original image AIL are arranged at the relative positions determined in the arrangement position determining process. As a result, arranged image data representing one arranged image representing the document 10 in FIG. 3 is generated by the two document images AIL and AIR.

  FIG. 7 is a diagram illustrating an example of the arranged image. In the arranged image OIA of FIG. 7, the document images AIL and AIR in the scan images I12 and I22 are arranged at the first type arrangement position determined by the arrangement position determination process.

  In the arranged image OIA, the pixel value (that is, the RGB value) in the area where the left original image AIL and the right original image AIR overlap with each other is the value of one original image, for example, the pixel in the right original image AIR. Is preferentially adopted. The additional pixels are appropriately deleted in the arranged image generation process so that the additional pixels in the additional area AA of the scanned images I12 and I22 are not arranged in the area representing the document 10 in the arranged image OIA.

  The CPU 410 generates a placed image file representing the placed image by converting the created placed image data into an image file of a predetermined format. For example, the arranged image data that is RGB image data is JPEG-compressed to generate a arranged image file in the JPEG format. In the modified example, a placed image file in PDF format may be generated from the placed image data.

  In S <b> 75 of FIG. 2, the CPU 410 transmits the generated arranged image file to the multifunction device 200. When receiving the arranged image file, the CPU 210 of the multifunction device 200 stores the received arranged image file in the nonvolatile storage device 230 and notifies the user that the arranged image file has been received. The arranged image file is provided for use by the user. For example, the multifunction device 200 can print the arranged image OIA using the arranged image file based on a user instruction.

  Note that the processing order determination process of S60 indicated by a broken line in FIG. 2 is not executed in the first embodiment. The physical order determination process is executed in the third and fourth embodiments, and will be described in the third and fourth embodiments.

A-3. Margin amount judgment processing:
The four margin amount determination processes in S35 to S50 will be described. The four margin amount determination processes are the same process except that the scan image to be processed is different from the end of the process target.

  FIG. 8 is a flowchart of margin amount determination processing. In S100, the CPU 410 determines a processing start position STP. The process start position STP is a specific position in the horizontal direction of the scan image to be processed. The processing start position STP is determined in the vicinity of the processing target end of the processing target scan image, in the present embodiment, in the vicinity of the right end and the left end of the scan images I12 and I22 in FIG. Specifically, when the scan image to be processed is the first scan image I12 and the end of the process target is the right end, the process start position STP is set to a value before tilt correction as shown in FIG. Among the upper right and lower right vertices of the scan image I11, the vertex in the horizontal direction is determined to be away from the right end of the first corrected scan image I12. When the scan image to be processed is the second scan image I22 and the end of the process target is the left end, the process start position STP is a scan image before tilt correction as shown in FIG. Among the upper left and lower left vertices of I21, the position is determined to be the horizontal position of the vertex that is away from the left end of the second scanned image I22 that has been tilt corrected.

  In S105, the CPU 410 selects one line of interest from among a plurality of vertical pixel lines in the scan image. The lines of the plurality of vertical pixels each include a plurality of pixels that are equal in the horizontal direction and arranged along the vertical direction. The first attention line is a pixel line at the processing start position STP. The line of interest is a direction away from the processing target end of the scanned image from the processing start position STP, that is, when the processing target end is the right end, the left direction, and when the processing target end is the left end. Are selected sequentially in the right direction.

  In S110, the CPU 410 selects one target pixel from a plurality of pixels included in the target line. The target pixel is sequentially selected one by one from the upper end to the lower end of the target line.

  In S115, the CPU 410 determines whether or not the target pixel is a specific color pixel having a specific color that should be included in the margin. The specific color that the margin should have is the color of the document table 255, which is white in this embodiment. When the target pixel has a pixel value (RGB value) within a predetermined range indicating a specific color, the CPU 410 determines that the target pixel is a specific color pixel, and the value of the pixel outside the range indicating the specific color If the pixel of interest is a non-specific color pixel having a color different from the specific color.

  If the target pixel is a specific color pixel (S115: YES), the CPU 410 adds 1 to the pixel counter WC in S120. The pixel counter WC is a counter for counting the number of specific color pixels continuous in the vertical direction. In S125, the CPU 410 initializes the noise counter NC, that is, sets the initial value to “0”. The noise counter NC is a counter for counting the number of non-specific color pixels continuous in the vertical direction.

  When the target pixel is a non-specific color pixel (S115: NO), the CPU 410 adds 1 to the noise counter NC in S130. In S135, the CPU 410 determines whether or not the noise counter NC is greater than or equal to a predetermined threshold value THn. The threshold value THn is, for example, 2 to 5.

  If the noise counter NC is greater than or equal to the predetermined threshold value THn (S135: YES), the CPU 410 determines in S140 whether or not the current pixel counter WC is greater than the current maximum continuous number WCm. When the pixel counter WC is larger than the maximum continuous number WCm (S140: YES), the CPU 410 updates the value of the maximum continuous number WCm to the value of the pixel counter WC in S145, and proceeds to S150. . When the pixel counter WC is less than the maximum continuous number WCm (S140: NO), the CPU 410 skips S145 and proceeds to S150. In S150, the CPU 410 initializes the value of the pixel counter WC.

  When the noise counter NC is less than the predetermined threshold value THn (S135: NO), the CPU 410 skips the above-described S140 to S150 and proceeds to S155.

  In S155, it is determined whether or not all the pixels in the target line have been processed as the target pixel. When there is an unprocessed pixel (S155: NO), the CPU 410 returns to S110 and selects an unprocessed pixel as a target pixel. When all the pixels have been processed (S155: YES), the CPU 410 executes the same processing as S140 and S145 described above. That is, in S160, it is determined whether or not the current pixel counter WC is larger than the current maximum continuous number WCm. If the pixel counter WC is larger than the maximum continuous number WCm (S160: YES), in S165. The value of the maximum continuous number WCm is updated to the value of the pixel counter WC. When the pixel counter WC is less than the maximum continuous number WCm (S160: NO), S165 is skipped.

  The maximum continuous number WCm at the time when the process proceeds to S170 indicates the maximum value of the number of specific color pixels continuous in the vertical direction in the target line. As will be understood from the above description, not only when one specific color pixel and another specific color pixel are actually continuous in the vertical direction, but also between one specific color pixel and another specific color pixel. Even if there is a non-specific color pixel below the threshold THn, it is determined that the one specific color pixel and the other specific color pixel are continuous in the vertical direction. This is because the non-specific color pixels can be included as noise in the margin portion of the scanned image due to various causes, for example, the characteristics of the image sensor of the scanner unit 250 and dirt on the document table 255. is there. In other words, in this embodiment, the fact that there is no non-specific color pixel equal to or greater than the threshold THn between one specific color pixel and another specific color pixel is that It can be said that it is a continuous condition for determining that the specific color pixel is continuous in the vertical direction. As a modification, the continuous condition may be that the specific color pixels are actually continuous in the vertical direction.

  In S170, CPU 410 determines whether or not maximum continuous number WCm is equal to or greater than threshold value THm. The threshold value THm is set to, for example, a value that is 50% of the number of pixels in the vertical direction of the scan image to be processed. When the maximum continuous number WCm is equal to or greater than the threshold value THm, it can be determined that the target line is a blank line constituting the blank. Therefore, when the maximum continuous number WCm is greater than or equal to the threshold value THm (S170: YES), the CPU 410 adds 1 to the margin counter WL for counting the number of margin lines in S175.

  When the maximum continuous number WCm is less than the threshold value THm, it can be determined that the target line is not a margin line constituting a margin but a line constituting the right end of the document image. Therefore, when the maximum continuous number WCm is less than the threshold value THm (S170: NO), the CPU 410 sets the current margin counter WL value as the margin line number WLN of the scan image to be processed in S190. Then, the process proceeds to S195. The margin line number WLN is an index value indicating the horizontal length of the right margin of the scanned image.

  In S180, the CPU 410 determines whether all lines in the scan image to be processed have been processed as the target line. Normally, when the target line reaches the right end of the document image, the margin amount determination process is terminated via S190 described above. Accordingly, the processing of all the lines does not occur except in an irregular case where, for example, a scanned image to be processed does not include a document image. If there is an unprocessed line in the scanned image (S180: NO), the CPU 410 initializes the pixel counter WC, the noise counter NC, and the maximum continuous number WCm in S185, and then returns to S105 to The next attention line of the attention line of is selected. The next target line is the line adjacent to the left side of the current target line when the end of the processing target is the right end, and the right side of the current target line when the end of the processing target is the left end. Adjacent line. When all the lines in the scan image have been processed (S180: YES), the CPU 410 determines the value of the current margin counter WL as the number of margin lines WLN of the scan image to be processed in S190, The process proceeds to S195.

  In S195, the CPU 410 determines whether or not the number of blank lines WLN is greater than or equal to the threshold value THw. When the number of margin lines WLN is equal to or greater than the threshold value THw (S195: YES), the CPU 410 determines in S197 that the margin amount of the scan image to be processed is greater than or equal to the reference, and ends the margin amount determination processing. . When the number of margin lines WLN is less than the threshold value THw (S195: NO), the CPU 410 determines in S198 that the margin amount of the scan image to be processed is less than the reference, and ends the margin amount determination processing. . The threshold value THw is, for example, 20 when the horizontal resolution of the scan image to be processed is 300 dpi.

  According to the margin amount determination process described above, the CPU 410 determines that a pixel whose pixel value satisfies a specific determination condition, specifically, a pixel whose pixel value is within a predetermined range indicating a specific color. (S115), the number of specific color pixels satisfying the continuous condition indicating that the specific color pixels continue in the vertical direction, that is, the maximum continuous number WCm is calculated (S120 to S165). Then, CPU 410 determines whether or not the margin amount is equal to or greater than the reference using maximum continuous number WCm (S170 to S198). As a result, it is possible to appropriately determine whether or not the margin amount is greater than or equal to the reference. This is because the margin is considered to be composed of continuous specific color pixels.

  Specifically, the CPU 410 has a maximum continuous number WCm of a plurality of vertical lines that is equal to or greater than a threshold value THm, and the processing target end (right end or left end) of the document image in the processing target scan image. And a margin line number WLN indicating the number of margin lines positioned between the scan image processing end and the edge of the scanned image (S170 to S190), and using the margin line number WLN, the margin amount is equal to or greater than the reference. Judge whether there is.

  As described above, in the margin amount determination process of the present embodiment, the number of blank lines WLN, which is the number of blank lines including a relatively large number of specific color pixels continuous in the vertical direction, is equal to or greater than the threshold value THw. Whether or not the margin amount is equal to or greater than the reference. This determination condition is a condition indicating that the number of a plurality of specific color pixels that are continuous between the right end of the document image and the right end of the scanned image in the vertical direction or the horizontal direction is equal to or greater than a reference. I can say that.

  When the scanned image to be processed is a blank image with no margins (for example, images ALa and ALc in FIGS. 4C and 4E and images ARa and ARc in FIGS. 5C and 5E). Is determined that both the left margin amount and the right margin amount of the scanned image are less than the reference. When the scan image to be processed is a single-side marginless image (for example, images ALb and ALd in FIGS. 4D and 4F and images ARb and ARd in FIGS. 5D and 5F). Is determined that the margin amount at one end of the left and right ends of the scanned image is less than the reference, and the other end is determined that the margin amount is greater than or equal to the reference.

A-4. Processing target decision processing:
The process target determination process in S55 of FIG. 2 will be described. FIG. 9 is a flowchart of the processing target determination process according to the first embodiment. In S205, the CPU 410 selects a two-end combination of interest from the following two two-end combinations (1) to (4).
(1) The left end LT1 of the document image in the first scan image I12 and the left end LT2 of the document image in the second scan image I22
(2) The left end LT1 of the document image in the first scan image I12 and the right end RT2 of the document image in the second scan image I22
(3) The right end RT1 of the document image in the first scan image I12 and the left end LT2 of the document image in the second scan image I22
(4) The right end RT1 of the document image in the first scan image I12 and the right end RT2 of the document image in the second scan image I22
By joining two scanned images I12 and I22 at two ends constituting any one of these two end combinations, the document 10 can be appropriately reproduced.

  In S210, the CPU 410 determines whether or not the margin amounts at the two ends constituting the combination of the two ends to be noticed are less than the reference. Whether the margin amount at the two ends is equal to or greater than the reference or less than the reference has been determined in the margin amount determination processing in S35 to S50 in FIG.

  If both margins at the two ends are less than the reference (S210: YES), in S215, the CPU 410 determines that the combination of the two ends to be noted is the processing target of the first search process. The first search process will be described later, but an end image along one end of the first scan image I12 constituting a combination of two ends to be processed and a second scan image constituting a combination of the two ends to be processed. This is a process of searching for a first type arrangement position where an end image along one end of I22 overlaps. Since the edges to be joined in order to properly reproduce the document 10 are the center CA of the document 10 in FIG. 3, that is, the edge corresponding to the partial area CAL or the partial area CAR, the margins in the scanned images I12 and I22 are present. Because there is almost no end.

  If at least one of the margins at the two ends is equal to or greater than the reference (S210: NO), in S220, the CPU 410 determines that the combination of the two ends to be noticed is not the target of the first search process. decide. This is because the edge where a certain amount of margin exists in the scanned images I12 and I22 is not the edge to be joined in order to appropriately reproduce the document 10.

  In S225, it is determined whether or not all the two-end combinations are selected as the two-end combinations to be noticed. When there is an unselected two-end combination (S225: NO), the CPU 410 returns to S205 and selects the two-end combination as a focused two-end combination. When all the combinations at the two ends have been selected (S225: YES), the processing target determination process ends.

  In the processing target determination process described above, when both the first scan image I12 and the second scan image I22 are both-side marginless images, all the two end points (1) to (4) described above are used. It is determined that the combination is a processing target of the first search process.

  If one of the first scan image I12 and the second scan image I22 is a one-sided marginless image and the other is a two-sided marginless image, the amount of margin among the left and right edges of the one-sided marginless image It is determined that the combination of the two ends including the end of which is less than the reference is the processing target of the first search process, and the combination of the two ends including the end of which the margin amount is equal to or greater than the reference is the first search process. Determined not to be processed. For example, when the first scan image I12 is the one-sided marginless image ALd in FIG. 4F and the second scan image I22 is the two-sided marginless image ARa in FIG. The combination of the two ends of 1) and (2) is determined to be the target of the first search process, and the combination of the two ends of (2) and (4) is not the target of the first search process Is determined.

  When both the first scan image I12 and the second scan image I22 are single-sided marginless images, the edge where the margin amount of one one-sided marginless image is less than the reference and the other one-sided marginless image It is determined that a combination of two ends composed of an end having a margin amount less than the reference and a target of the first search process is determined as a combination of two ends, and a combination of the other two ends is a process of the first search process. It is determined that it is not the target. For example, when the first scan image I12 is the one-sided marginless image ALd in FIG. 4F and the second scan image I22 is the one-sided marginless image ARb in FIG. The combination of the two ends of 1) is determined to be the processing target of the first search process, and the combination of the two ends of (2) to (4) is determined not to be the processing target of the first search process. .

A-5. Arrangement position determination processing:
The arrangement position determination process in S65 of FIG. 2 will be described. FIG. 10 is a flowchart of the arrangement position determination process of the first embodiment. In S230, the CPU 410, among the combinations of the two ends of (1) to (4) described above, has one or more two-ends determined to be processing targets of the first search processing in the processing target determination processing. One combination is selected from the combinations. In S235, the CPU 410 executes a first search process for the selected combination of the two ends.

  FIG. 11 is a flowchart of the first search process. FIG. 12 is an explanatory diagram of the first search process. In FIG. 12, the combination of the right end RT1 of the original image in the first scan image I12 and the right end RT2 of the original image in the second scan image I22 in FIG. 6B is a combination of two ends to be processed. A case is shown as an example. Hereinafter, the ends constituting the combination of the two ends to be processed (for example, the right end RT1 of the document image in the first scan image I12 in FIG. 12 and the right end RT2 of the document image in the second scan image I22) are each scanned image. It is also called the end of the processing target.

  In S300, the CPU 410 executes blank deletion and rotation on the scanned images I12 and I22. Specifically, the margin at the end of the processing target of each scanned image is removed. Specifically, the line determined to be a margin line in the margin amount determination process at the edge to be processed and the portion outside the margin line (that is, the portion outside the processing start position STP) are scanned. Delete from the images I12 and I22. Further, when the first scan image I12 and the second scan image I22 are arranged side by side, the processing target end of the first scan image I12 and the processing target end of the second scan image I22 face each other. Rotate the scanned image so that For example, in the example of FIG. 12, the margin of the right end in the first scan image I12 and the right end in the second scan image I22 is deleted, and the second scan image I22 is such that these two ends face each other. It has been rotated 180 degrees. The margin amount at the end of the processing target is less than the reference, and there is substantially no margin. Therefore, only a slight portion on the right side of the processing start position STP (FIG. 6B) is substantially deleted. It is. In FIG. 12, illustration of the inside of the document images AIL and AIR is omitted, and only a signed arrow is shown.

  In S302, the CPU 410 determines a reference area SPa that is a partial area of the second scan image I22 in the second scan image I22. As shown in FIG. 12, the reference area SPa is determined to be a rectangular area having a predetermined size arranged at a predetermined position in the second scan image I22. The reference area SPa is arranged in the vicinity of the processing target end of the second scan image I22 along the processing target end. For example, the horizontal length of the reference area SPa is, for example, 50 to 150 pixels, and the vertical length of the reference area SPa is ¼ to the vertical length of the second scan image I22. 1/2.

  In S305, the CPU 410 determines one candidate region of interest from among a plurality of candidate regions determined based on a predetermined reference range REa for determining a candidate region. For example, as illustrated in FIG. 12, the reference range REa is a range indicated by a broken line in the vicinity of the processing target end of the first scan image I12. The reference range REa is a rectangular range of M1 vertical pixels × N1 horizontal pixels (M1 and N1 are integers of 2 or more). N1 is several hundreds, for example, and M1 is several tens to several hundreds, for example. For example, the right end of the reference range REa matches the right end of the first scan image I12. The horizontal width of the reference range REa has a length corresponding to the width of the central portion CA in the document 10, for example. The position of the center in the vertical direction of the reference range REa in the first scan image I12 is the same as the upper end of the reference region SPa in the second scan image I22.

  The CPU 410 selects one target pixel position from (M1 × N1) pixel positions within the reference range REa. When the reference area SPa is arranged such that the upper left pixel Pd (FIG. 12) of the reference area SPa is positioned at the target pixel position, the CPU 410 has an area in the first scan image I12 that overlaps the reference area SPa. It is specified as the attention candidate area. The search area SAa, which is an area where the candidate area can be set, is an area along the processing target end of the first scan image I12 as shown in FIG.

  When one attention candidate area is specified, in S310, the CPU 410 calculates a similarity ratio (SC / Nt) of the attention candidate area. The similarity rate (SC / Nt) of the attention candidate area is an index value indicating the degree of similarity between the attention candidate area and the reference area SPa.

  The calculation method of the similarity ratio (SC / Nt) is as follows. First, the CPU 410 selects a plurality of pixels in the target candidate region one by one as the target pixel, and determines whether the target pixel is a similar pixel or a dissimilar pixel. Specifically, the CPU 410 calculates a difference ΔVP between the component value of the target pixel in the target candidate region and the component value of the pixel in the reference region SPa corresponding to the target pixel. A pixel in the reference region SPa corresponding to the target pixel is a pixel in the reference region SPa that overlaps the target pixel when the target candidate region and the reference region SPa are overlapped. The component values of the two pixels for which the difference ΔVP is to be calculated are (R1, G1, B1) and (R2, G2, B2). The difference ΔVP is represented by, for example, the sum of absolute values of differences between three types of component values. That is, the difference ΔVP is represented by the sum of the absolute value of (R1−R2), the absolute value of (G1−G2), and the absolute value of (B1−B2).

  The CPU 410 determines that the target pixel is a similar pixel when the calculated difference ΔVP is equal to or smaller than the predetermined reference value TH1, and when the calculated difference ΔVP is larger than the predetermined reference value TH1. , It is determined that the target pixel is a dissimilar pixel. When the difference ΔVP is equal to or smaller than the predetermined reference value TH1, it can be determined that the color of the target pixel in the target candidate region and the color of the pixel in the reference region SPa corresponding to the target pixel are similar. is there.

  The CPU 410 calculates the similarity ratio (SC / Nt) of the target candidate region by dividing the number SC of similar pixels in the target candidate region by the total number Nt of pixels in the target candidate region. The similarity ratio (SC / Nt) is a ratio of the number SC of similar pixels to the total number Nt of pixels in the target candidate region. The larger the similarity ratio (SC / Nt), the more similar the reference area SPa and the attention candidate area.

  In S315, the CPU 410 determines whether or not the similarity ratio (SC / Nt) is equal to or higher than a threshold value TH2. That is, the CPU 410 determines whether or not the current candidate region for attention is similar to the reference region SPa.

  When the similarity ratio (SC / Nt) is equal to or greater than the threshold value TH2, that is, when it is determined that the current candidate region of interest is similar to the reference region SPa (S315: YES), the CPU 410 proceeds to S320. The current candidate region for attention is stored as a similar region similar to the reference region SPa, and the process proceeds to S325.

  When the similarity ratio (SC / Nt) is less than the threshold value TH2 (S315: NO), the CPU 410 skips S320 and proceeds to S325.

  In S325, the CPU 410 determines whether or not all (M1 × N1) candidate areas corresponding to the (M1 × N1) pixel positions in the reference range REa have been processed as the target candidate areas. When there is an unprocessed candidate area (S325: NO), the CPU 410 returns to S305 and identifies the unprocessed candidate area as the attention candidate area. If all candidate areas have been processed (S325: YES), the CPU 410 advances the process to S330.

  In S330, the CPU 410 determines whether or not there are one or more similar regions in the (M1 × N1) candidate regions. That is, it is determined whether there are one or more candidate areas stored as similar areas in S320. If there is one or more similar areas (S330: YES), in S335, the CPU 410 has the highest similarity ratio (SC / Nt) calculated in S310 among the one or more similar areas. One similar area CPa is determined as the final one similar area CPa in the combination of the two ends to be processed. When one similar region CPa is determined, the first search process is terminated.

  If there is no one or more similar regions (S330: NO), the CPU 410 skips S335 and ends the first search process. Therefore, in this case, the similar region CPa is not determined.

  When the first search process is completed, it is determined in S240 of FIG. 10 whether or not all the combinations of the two ends determined to be the target of the first search process have been selected. If there is an unselected two-end combination (S240: NO), the CPU 410 returns to S230 and selects the two-end combination. When all the combinations of the two ends to be processed have been selected (S240: YES), the CPU 410 advances the process to S245.

  In S245, the CPU 410 determines whether there is a combination of two ends for which the similar area CPa is determined in S335 of the first search process in FIG. 11 among the combinations of the two ends to be processed. Usually, when a combination of two ends that can correctly reproduce the document 10 by joining them together is a processing target, it is highly likely that the similar region CPa is determined in the first search process, and the document 10 is When a combination of two ends that can be correctly reproduced is not a processing target, there is a high possibility that the similar region CPa is not determined in the first search process.

  If there are one or more two-end combinations for which the similar region CPa is determined (S245: YES), in S250, the CPU 410 determines the similarity ratio (SC / Nt) among the one or more two-end combinations. The arrangement position of the first type is determined by the combination of the two ends where the similar region CPa having the highest) is determined. That is, the relative relationship between the first scan image I11 and the first scan image I12 is such that the reference region SPa used in the first search process for the combination of the two ends overlaps the determined similar region CPa. Is determined (that is, a relative arrangement position between the left original image AIL and the right original image AIR). For example, in the example of the arranged image OIA in FIG. 7 described above, the right end RT1 of the document image in the first scan image I12 in FIG. 6B and the right end RT2 of the document image in the second scan image I22. The combination is a two-end combination whose arrangement position is to be determined. In this arranged image OIA, the similar region CPa along the right end RT1 of the document image in the first scan image I12 and the reference region SPb along the right end RT2 of the document image in the second scan image I22 overlap. An arrangement position is determined.

  If there is no combination of the two ends for which the similar area CPa is determined (S245: NO), in S255, the CPU 410 compares the two scanned images I12 and I22 (two original images AIL and AIR) with each other. The default position is determined. The default position is, for example, a position where the right end of the first scan image I12 and the left end of the second scan image I22 are in contact with each other over the entire length.

  According to the first embodiment described above, the first scan data representing the first scan image I12 including the document image (AIL or AIR) and the second scan image I22 including the document image (AIL or AIR) are represented. Second image data is acquired (S25 and S30 in FIG. 2). Then, it is determined whether or not the margin amounts at the left and right ends of the first scan image I12 are greater than or equal to the reference (S35 and S40 in FIG. 2). Then, the combination of the two ends to be processed is determined as shown in FIG. 9 using whether or not the margins at the left and right ends of the first scan image I12 are greater than or equal to the reference, and as shown in FIG. The first search process is executed for the combination of the two ends determined as the processing target, whereby the first type arrangement position of the two document images AIL and AIR is determined.

  For example, when the amount of margin at the right end of the first scan image I12 is less than the reference, in the processing target determination process of FIG. 9, the right end RT1 of the document image in the first scan image I12 and the second scan image I22 At least one of a combination with the right end RT2 of the original image and a combination of the right end RT1 of the original image in the first scan image I12 and the left end LT2 of the original image in the second scan image I22 is two ends to be processed. Determined as a combination. Therefore, in this case, in the arrangement position determination process of FIG. 10, at least one of an end image along the right end RT1, an end image along the right end RT2, and an end image along the left end LT2 is used. The first search process (S235 in FIG. 10) is executed.

  Conversely, if the amount of margin at the right end of the first scan image I12 is greater than or equal to the reference, the combination of the two ends including the right end RT1 of the document image in the first scan image I12 is determined in the process target determination process of FIG. The combination of the two ends to be processed is not determined. Therefore, in this case, in the arrangement position determination process in FIG. 10, the first search process (S235 in FIG. 10) is executed without using the end image along the right end RT1.

  For example, when the amount of margin at the left end of the first scan image I12 is less than the reference, in the processing target determination process of FIG. 9, the left end LT1 of the document image in the first scan image I12 and the second scan image I22. At least one of the combination with the right end RT2 of the original image in the image and the combination of the left end LT1 of the original image in the first scan image I12 and the left end LT2 of the original image in the second scan image I22 Determined as a two-end combination. Therefore, in this case, in the arrangement position determination process in FIG. 10, at least one of the end image along the left end LT1, the end image along the right end RT2, and the end image along the left end LT2 is used. The first search process (S235 in FIG. 10) is executed.

  On the other hand, when the amount of margin at the left end of the first scan image I12 is greater than or equal to the reference, the combination of the two ends including the left end LT1 of the document image in the first scan image I12 is determined in the processing target determination process of FIG. The combination of the two ends to be processed is not determined. Therefore, in this case, in the arrangement position determination process in FIG. 10, the first search process (S235 in FIG. 10) is executed without using the end image along the left end LT1.

  As a result, the first search process is executed without using the edge image along the edge whose margin amount is greater than or equal to the reference among the document images in the first scan image I12. Execution of the first search process that does not contribute can be suppressed. As a result, the processing time of the first search process can be reduced. Therefore, it is possible to reduce the processing time when generating image data representing the arranged image.

  Furthermore, in the first embodiment, it is determined whether or not the margin amounts at the left and right ends of the second scan image I22 are greater than or equal to the reference (S45 and S40 in FIG. 2). Then, the combination of the two ends to be processed is determined as shown in FIG. 9 using whether or not the margins at the left and right ends of the second scan image I22 are greater than or equal to the reference, and as shown in FIG. The first search process is executed for the combination of the two ends determined as the processing target, whereby the first type arrangement position of the two document images AIL and AIR is determined.

  For example, when the amount of margin at the right end of the second scan image I22 is less than the reference, in the processing target determination process of FIG. 9, the right end RT2 of the document image in the second scan image I22 and the first scan image I12 At least one of the combination with the right end RT1 of the document image and the combination of the right end RT2 of the document image in the second scan image I22 and the left end LT1 of the document image in the first scan image I12 is two ends to be processed. Determined as a combination. Therefore, in this case, in the arrangement position determination process of FIG. 10, at least one of the end image along the right end RT2, the end image along the right end RT1, and the end image along the left end LT1 is used. The first search process (S235 in FIG. 10) is executed.

  Conversely, if the amount of margin at the right end of the second scan image I22 is greater than or equal to the reference, the combination of the two ends including the right end RT2 of the document image in the second scan image I22 is determined in the processing target determination process of FIG. The combination of the two ends to be processed is not determined. Therefore, in this case, in the arrangement position determination process in FIG. 10, the first search process (S235 in FIG. 10) is executed without using the end image along the right end RT2.

  For example, when the amount of margin at the left end of the second scan image I22 is less than the reference, in the processing target determination process of FIG. 9, the left end LT2 of the document image in the second scan image I22 and the first scan image I12. At least one of a combination with the right end RT1 of the original image in the image and a combination of the left end LT2 of the original image in the second scan image I22 and the left end LT1 of the original image in the first scan image I12 Determined as a two-end combination. Therefore, in this case, in the arrangement position determination process in FIG. 10, at least one of the end image along the left end LT2, the end image along the right end RT1, and the end image along the left end LT1 is used. The first search process (S235 in FIG. 10) is executed.

  On the contrary, if the margin amount at the left end of the second scan image I22 is greater than or equal to the reference, the combination of the two ends including the left end LT2 of the document image in the second scan image I22 is determined in the processing target determination process of FIG. The combination of the two ends to be processed is not determined. Therefore, in this case, in the arrangement position determination process of FIG. 10, the first search process (S235 of FIG. 10) is executed without using the end image along the left end LT2.

  As a result, the first search process is executed without using the edge image along the edge whose margin amount is greater than or equal to the reference among the document images in the second scan image I22. Execution of the first search process that does not contribute can be further suppressed. As a result, the processing time of the first search process can be further reduced. Therefore, it is possible to further reduce the processing time when generating image data representing the arranged image.

  More specifically, the margin amount at the right end of the first scan image I12 is less than the reference, the margin amount at the left end of the first scan image I12 is greater than or equal to the reference, and the margin amount at the right end of the second scan image I22. When the margin amount is less than the reference and the margin amount at the left end of the second scan image I22 is greater than or equal to the reference, the right end RT1 of the document image in the first scan image I12 and the second scan image I22 The combination with the right end RT2 of the document image is determined to be a combination of two ends to be processed, and the other two ends are not determined to be a combination of two ends to be processed. Therefore, in this case, without using the end image along the left end LT1 of the document image in the first scan image I12 and the end image along the left end LT2 of the document image in the second scan image I22, The first search process is executed using the end image along the right end RT1 of the document image in the first scan image I12 and the end image along the right end RT2 of the document image in the second scan image I22. The Therefore, since the first search process is executed using only the necessary edge image based on whether the margin amount is equal to or larger than the reference, the processing time for generating the image data representing the arranged image is further increased. Can be reduced

  As can be understood from the above description, the document image in the first scan image I12 is an example of the first target image, and the document image in the second scan image I22 is an example of the second target image. The right end is an example of an end in the first direction, and the left end is an example of an end in the second direction. The specific color pixel is an example of first to fourth pixels.

B. Second embodiment:
B-1. Scan pattern B
In the second embodiment, in addition to the method of scan pattern A shown in FIGS. 4 and 5, the scan pattern B described below is used to perform two readings to generate two scan data. there is a possibility.

  FIG. 13 is an explanatory diagram of the scan pattern B. In the scan pattern B, as shown in FIG. 13A, the document 10 is read by the scanner unit 250 in a state where the document 10 is folded in half along a broken line VL. The broken line VL is a line at the same position as the line CL in FIG. Accordingly, the size of the document 10 in a folded state is A4 size. The right area 10R is located on one side of the document 10 in a folded state, and the left area 10L is located on the other side.

  On the other hand, as shown in FIGS. 13B and 13C, an image including the left region 10L and not including the right region 10R (hereinafter, also referred to as a left original image BIL) is obtained while scanning pattern B is read twice. The left side scan data indicating the left side original image BIL is generated by being read. As shown in FIGS. 13D and 13E, on the other side of the two scanning patterns B, an image including the right region 10R and not including the left region 10L (hereinafter also referred to as a right original image BIR) is obtained. The right side scan data indicating the right side original image BIR is generated by being read. Which of the left original image BIL and the right original image BIR is read first depends on the user, and the multifunction device 200 and the server 400 cannot recognize which was read first.

  There are two types of modes shown in FIGS. 13B and 13C for reading the left original image BIL. In both modes 1 and 2, as shown in FIGS. 13B and 13C, the left region 10L faces the original plate 255 side, and the right region 10R faces the opposite side of the original plate 255. In Mode 1, as shown in FIG. 13B, the document 10 is arranged on the document table 255 with the broken line VL of the document 10 facing the front side of the document table 255 (the lower side in FIG. 13B). The In Mode 2, as shown in FIG. 13C, the document 10 is arranged on the document table 255 with the polygonal line VL of the document 10 facing the back side of the document table 255 (upper side in FIG. 13C). .

  There are two types of modes shown in FIGS. 13D and 13E for reading the right original image BIR. In both modes 1 and 2, as shown in FIGS. 13D and 13E, the right area 10R faces the original plate 255 side, and the left area 10L faces the opposite side of the original plate 255. In Mode 1, as shown in FIG. 13D, the document 10 is placed on the document table 255 with the broken line VL of the document 10 facing the front side of the document table 255 (the lower side of FIG. 13D). The In Mode 2, as shown in FIG. 13E, the document 10 is placed on the document table 255 with the polygonal line VL of the document 10 facing the back side of the document table 255 (the upper side of FIG. 13E). .

  Which mode is used for reading any document image BIL or BIR depends on the user, and the multifunction device 200 and the server 400 cannot recognize the used mode.

  FIG. 14 is a diagram illustrating an example of a scan image represented by scan data generated by the scan pattern B. The left-side scan images BLa and BLb in FIGS. 14A and 14B are examples of left-side scan images generated in the first and second reading modes 1 and 2 of the left original image BIL. The right-side scan images BRa and BRb in FIGS. 14C and 14D are examples of right-side scan images generated in the first and second modes of reading the right-side original image BIR. The length WB (FIG. 13A) in the short direction of the document 10 in a folded state is shorter than the length WT in the short direction of the document table 255 (FIG. 13B). For this reason, when the scan pattern B is used, in the left-side scan image and the right-side scan image, at least one of the left and right ends has a certain margin, and the other has a margin and a margin that cannot be. (FIGS. 14A to 14D).

  As can be understood from the above description, the scan image (left scan image BL or right scan image BR) generated by the scan pattern B is a double-sided marginal image (for example, FIG. ), (D) images BLb, BRb), and one-sided marginal images with no margin MG at one end of the left and right ends and margin MG at the other end (for example, FIG. 14 (A), ( C) image BLa, BRa).

  In the second embodiment, the process target determining process in S55 of FIG. 2 and the arrangement position determining process in S65 are different from those in the first embodiment. Other processes are the same as those in the first embodiment.

B-2. Second search process:
In the arrangement position determination process described later in the second embodiment, the second search process is used together with the first search process of FIG. 11 described in the first embodiment. The second search process is a search process suitable for the case where the scan images I12 and I22 are generated using the scan pattern B. The second search process is a second type of arrangement in which the two original images do not overlap as a relative arrangement position of the original image in the first scan image I12 and the original image in the second scan image I22. This is a process for determining the position. For convenience of explanation, the second search process will be described first.

  FIG. 15 is a flowchart of the second search process. FIG. 16 is an explanatory diagram of the second search process. Similarly to the first search process, an end image along one end of the first scan image I12 constituting a combination of two ends to be processed and one end of the second scan image I22 constituting a combination of the two ends to be processed. And an end image along the line. Here, the second search process will be described by taking two scan images I12b and I22b shown in FIG. 16A represented by two scan data generated using the scan pattern B as an example. The first scan image I12b in FIG. 16A includes a left original image BIL, and the second scan image I22b includes a right original image BIR. Here, as an example, the combination of the right end RT1b of the document image BIL in the first scan image I12b and the left end LT2b of the document image BIR in the second scan image I22b is a combination of the two ends to be processed. explain.

  In S400, the CPU 410 executes margin deletion and rotation on the scanned images I12b and I22b, as in S300 of FIG. Since the scan images I12b and I22b in FIG. 16A are images after the processing of S400, there is no margin at the right end of the first scan image I12b and there is no margin at the left end of the second scan image I22b.

  In S401, the CPU 410 executes a pixel restoration process for restoring the missing pixels in the original images BIL and BIR in the scanned images I12b and I22. FIG. 16A shows restoration areas AD1 and AD2 in which a plurality of pixels restored by the pixel restoration process are located. As described above, in the scan pattern B, the document 10 folded in half is read. For this reason, an image of a very thin linear region positioned on the polygonal line VL of the document 10 appears in both the document images BIL and BIR in the scan images I12b and I22b generated using the scan pattern B. It is missing. The restoration areas AD1 and AD2 in FIG. 16A correspond to very thin linear areas located on the polygonal line VL of the document 10.

  In the second search process, it is assumed that the processing target edge of the document image in the first scan images I12 and I22 is an edge corresponding to the document line VL. For this purpose, the first restoration area AD1 is provided adjacent to the outside of the processing target end of the document image in the scanned image I12, and the second restoration area AD2 is processed for the document image in the second scanned image I22. Adjacent to the outside of the subject's end. In the example of FIG. 16A, the first restoration area AD1 is adjacent to the outside of the right end RT1b of the left original image BIL in the first scan image I12b, and the second restoration area AD2 is in the second scan image I22b. It is adjacent to the outside of the left end LT2b of the right original image BIR. The restoration areas AD1 and AD2 are very thin linear areas extending from the upper end to the lower end over the entire length of adjacent document images in the vertical direction. The width in the direction perpendicular to the adjacent ends of the restoration regions AD1 and AD2, that is, the width ΔW in the horizontal direction in FIG. 16A is a value experimentally determined in advance. In this embodiment, the width ΔW is 3 pixels when the horizontal resolution of the scanned images I12b and I22b is 300 dpi.

  FIG. 17 is a flowchart of pixel restoration processing. In S500, the CPU 410 selects a target image from the scanned images I12b and I22b. In step S <b> 505, the CPU 410 sets the above-described restoration area along the processing target end of the document image in the target image. For example, when the first scan image I12b is the target image, the first restoration area AD1 is set along the right end RT1b of the left original image BIL in the first scan image I12b.

  In S510, the CPU 410 selects a target line in the restoration area. For example, in this embodiment, the horizontal width ΔW of the restoration area is 3 pixels, and thus includes a line of 3 pixels extending in the vertical direction. The CPU 410 selects one target line from the three pixel lines in order from the side close to the target image. For example, as shown on the right side of FIG. 17, the first restoration area AD1 includes three lines L1 to L3. Therefore, when the first scan image I12b is the target image, the line L1 is first selected as the target line, and then the line L2 is selected as the target line.

  In S515, the CPU 410 selects one target pixel from a plurality of pixels included in the target line. For example, the target pixel is sequentially selected from a plurality of pixels included in the target line from the upper side to the lower side.

  In S520, the CPU 410 determines the value of the target pixel (RGB value in this embodiment) using the values of a plurality of neighboring pixels of the target pixel. For example, when the scanned image I12b in FIG. 16A is a target image, the value of the target pixel is determined using the values of four neighboring pixels located on the left side of the target pixel. For example, when the pixel X of the line L1 shown on the right side of FIG. 16 is the target pixel, the four neighboring pixels are the three pixels A, B, and C of the line La adjacent to the left side of the line L1, and One pixel D of the line Lb adjacent to the left side of the line La. Pixel A is a first neighboring pixel closest to the pixel of interest among pixels located on the left side of the pixel of interest, pixels B and C are second neighboring pixels closest to the pixel of interest, and pixel D is The third neighboring pixel that is the third closest to the target pixel. A specific component value (for example, R component value) Vx of the pixel X is calculated by the following equation (a) using the specific component values Va to Vd of the pixels A to D.

Vx = Ka * Va + Kb * Vb + Kc * Vc + Kd * Vd (a)
Here, Ka to Kd are coefficients determined in advance according to the distance between each of the pixels A to D and the pixel X. The coefficients Ka to Kd are determined so that the total value of Ka to Kd is 1 and becomes larger as the distance from the pixel X is shorter. In this embodiment, Ka = 0.5, Kb = Kc = 0.2, and Kd = 0.1.

  The CPU 410 determines the RGB value of the pixel of interest by calculating the values of the three RGB components of the pixel of interest using equation (a).

  Although illustration is omitted, when the second scan image I22b in FIG. 16A is the target image, the value of the target pixel is the value of four neighboring pixels located on the right side of the target pixel. Determined.

  In S525, the CPU 410 determines whether or not all the pixels on the target line have been processed as the target pixel. When there is an unprocessed pixel (S525: NO), the CPU 410 returns to S515 and selects the unprocessed pixel as a target pixel. If all the pixels have been processed (S525: YES), the CPU 410 determines in S530 whether or not all the lines in the restoration area set for the target image have been processed as the target line. When there is an unprocessed line (S530: NO), the CPU 410 returns to S510 and selects an unprocessed line as a target line. If all lines have been processed (S530: YES), the CPU 410 advances the process to S535.

  In S535, the CPU 410 determines whether both scanned images I12 and I22 have been processed. If both scan images have not been processed (S535: NO), the CPU 410 returns to S500 and selects an unprocessed scan image as a target image. If the CPU 410 has processed both scanned images (S535: YES), the pixel restoration process is terminated.

  When the combination of the two ends to be processed is a combination that can correctly reproduce the document 10, a linear image composed of a plurality of first restoration pixels in the first restoration area AD1 and the second restoration area Both of the linear images constituted by the plurality of right-side restored pixels in AD 2 are images representing the linear images located on the polygonal line VL of the document 10. Therefore, in this case, it is considered that the image in the first restoration area AD1 and the image in the second restoration area AD2 include portions that are similar to each other.

  When the pixel restoration process ends, the CPU 410 determines the reference area SPb in S402 of FIG. In the present embodiment, the entire second restoration area AD2 is determined as the reference area SPb. Instead, a part of the second restoration area AD2, or an area constituted by the second restoration area AD2 and a part of the document image adjacent to the second restoration area AD2, may be determined as the reference area SPb. .

  In S405, the CPU 410 determines one candidate region of interest from among a plurality of candidate regions determined based on a predetermined reference range REb for determining a candidate region. For example, as illustrated in FIG. 16A, the reference range REb is a range indicated by a broken line in the upper right portion of the first scan image I12b.

  FIG. 18 is an enlarged view in the vicinity of the reference range REb in FIG. The reference range REb is a rectangular range of vertical M2 pixels × horizontal N2 pixels (M2 and N2 are integers of 2 or more). N2 is about 10, for example, and M2 is, for example, several tens to several hundreds. The right end of the reference range REb coincides with the right end of the first restoration area AD1 on the right side of the first scan image I12b. The center in the vertical direction of the reference range REb is located at the upper end of the first scan image I12b.

  The CPU 410 selects one target pixel position from (M2 × N2) pixel positions in the reference range REb. When the reference region SPb is arranged such that the upper left pixel Pc (FIG. 16A) of the reference region SPb is positioned at the target pixel position, the CPU 410 overlaps the first scan image I12b or the reference region SPb. An area in the restoration area AD1 is specified as a candidate area for attention. In this way, (M2 × N2) pixel positions in the reference range REb have a one-to-one correspondence with (M2 × N2) candidate regions. For example, the candidate area NT1 indicated by hatching in FIG. 18 is specified when the upper right pixel position Pt1 in the reference range REb is the target pixel position. The candidate area NT2 indicated by hatching in FIG. 18 is specified when the lower left pixel position Pt2 in the reference range REb is the target pixel position. As shown in FIG. 18, the search area SAb, which is an area where a candidate area can be set, is an area located on the right side of the position LP at the left end of the reference range REb. The search area SAb for the first arrangement position determination process includes a first restoration area AD1 and a partial area of the first scan image I12b.

  For example, the first attention candidate area is the candidate area NT1 in FIG. Then, by moving the pixel position in the reference range REb for specifying the candidate region of interest NT1 in increments of one pixel from the position for specifying the candidate region NT1 in the vertical and horizontal directions (M2 × N2) One candidate region of interest is sequentially identified from the candidate regions.

  The processing of S410 to S435 is the same as the processing of S310 to S335 of FIG. That is, in other words, when one or more similar regions CPb exist in the (M2 × N2) attention regions set in the search region SAb in the processing of S410 to S435, the reference region SPb The one similar region CPb having the highest similarity rate is determined as the similar region CPb in the combination of the two ends to be processed, and the second search process is ended. If no similar region CPb exists in the (M2 × N2) attention regions, the second search process ends without determining the similar region CPb.

B-3. Processing target decision processing:
Next, the processing target determination process of the second embodiment will be described. In the processing target determination process of the second embodiment, in addition to the two-end combination that is the processing target of the first search process, the two-end combination that is the processing target of the second search process is determined.

  FIG. 19 is a flowchart of a processing target determination process according to the second embodiment. In step S605, the CPU 410 determines whether the margin amounts at the left and right ends of the first scan image I12 are less than the reference value. When the margin amount at both ends of the first scan image I12 is less than the reference value (S605: YES), in S610, the CPU 410 determines the first scan image I12 as an image with no margins on both sides. If the margin amount at least one of the left and right ends of the first scan image I12 is greater than or equal to the reference value (S605: NO), the CPU 410 determines the first scan image I12 as a margin image in S615. . The margin image is a concept including the above-described one-side margin image and both-side margin image.

  In S620, the CPU 410 determines whether or not the margin amounts at the left and right ends of the second scan image I22 are less than the reference value. When the margin amount at both ends of the second scan image I22 is less than the reference value (S620: YES), the CPU 410 determines the second scan image I22 as a blank image with no margins at S625. If the margin amount at least one of the left and right ends of the second scan image I22 is greater than or equal to the reference value (S620: NO), in S630, the CPU 410 determines the second scan image I22 as a margin image. .

  In S635, the CPU 410 determines whether or not both of the scanned images I12 and I22 are both-side marginless images. If both the scanned images I12 and I22 are both side marginless images (S635: YES), the process proceeds to S645. If at least one of the scanned images I12 and I22 is a marginal image (S635: NO), in S640, the CPU 410 determines whether one of the scanned images I12 and I22 is a blank marginless image. . When one of the scanned images I12 and I22 is a double-sided marginless image (S640: YES), one of the scanned images I12 and I22 is a double-sided marginless image and the other is a blanked image. In this case, the CPU 410 advances the process to S655. When one of the scanned images I12 and I22 is not an image with no margins on both sides (S640: NO), both of the scanned images I12 and I22 are marginal images. In this case, the CPU 410 advances the process to S665.

  When both the scanned images I12 and I22 are both-side marginless images, first, in S645, the CPU 410 selects all the two end combinations, that is, the above-described two end combinations (1) to (4). Then, it is determined that it is a processing target of the first search process. In this case, it is considered that the scan pattern A was used, and there is a possibility that the original 10 can be correctly reproduced for all the combinations at the two ends (1) to (4).

  In step S650, the CPU 410 determines that all combinations of the two ends are not processing targets of the second search process. That is, in this case, the second search process is not executed in the arrangement position determination process described later. This is because, in scan pattern B, no marginal image on both sides is not generated as a scan image, so it is considered that scan pattern A is used when at least one scan image is a no marginal image on both sides. . As a result, it is possible to suppress the execution of the unnecessary second search process and reduce the processing time of the arrangement position determination process.

  When one of the scanned images I12 and I22 is a blank image with no margins and the other is a margined image, first, in S655, the CPU 410 executes S205 to S225 in FIG. The combination of the two ends to be processed and the combination of the two ends not to be processed in the first search process are determined. As a result, as in the first embodiment, only the two-end combination that can correctly reproduce the document 10 is the processing target of the first search process, so that the processing time of the arrangement position determination process is reduced. can do. In this case, it is considered that the scan pattern A is used when one of the scan images I12 and I22 is an image with no margins on both sides. As described above, in the scan pattern B, a blank image with no margins is not generated as a scan image. Therefore, in this case, the other marginal image of the scanned images I12 and I22 is considered to be a one-sided marginal image. This is because an image with margins on both sides can be generated with the scan pattern B but cannot be generated with the scan pattern A.

  In S660, as in S650, the CPU 410 determines that all combinations of the two ends are not processing targets of the second search process. As described above, in the scan pattern B, a blank image with no margins is not generated as a scan image. As a result, it is possible to suppress the execution of the unnecessary second search process and reduce the processing time of the arrangement position determination process.

  When both the scanned images I12 and I22 are marginal images, first, in S665, as in S655, the CPU 410 executes S205 to S225 in FIG. The combination of the two ends to be determined and the combination of the two ends not to be processed in the first search process are determined. As a result of this processing, when both of the scan images I12 and I22 are single-sided marginal images, the margin of the first scan image I12 is less than the reference and the margin of the second scan image I22 is less than the reference. A combination with a certain end is determined to be a target of the first search process. Then, it is determined that the other three two-end combinations are not processing targets of the first search process. When at least one of the scanned images I12 and I22 is a double-sided marginal image, it is determined that all the combinations of the two ends are not processing targets of the first search process. This is because when the scan pattern A is used, a double-sided marginal image is not generated as a scan image. As a result, when there is a possibility that the scan pattern A is used, only the two-end combination that may be able to reproduce the original 10 correctly is regarded as the processing target of the first search process. If there is no possibility of being used, it is determined that all combinations of two ends are not processing targets of the first search process. Therefore, the processing time of the arrangement position determination process can be reduced.

  In S670, the CPU 410 determines that all of the four two-end combinations are processing targets of the second search process. In this case, there is a possibility that the scan pattern B is used. When the scan pattern B is used, the document 10 can be correctly reproduced for all the combinations of the two ends (1) to (4). This is because it may be a combination.

  Thus, for each of the first search process and the second search process, when the combination of two ends to be processed and the combination of two ends not to be processed are determined, the process target determination process is Is terminated.

  According to the processing target determination process described above, each of the first search process and the second search process is based on whether the margin amounts at the left and right ends of the scanned images I12 and I22 are greater than or equal to the reference. It is possible to appropriately determine a combination of two ends to be processed and a combination of two ends that are not treated. As a result, it is possible to suppress the unnecessary first search process and second search process from being executed, and to reduce the processing time of the arrangement position determination process.

B-4: Arrangement position determination processing:
Next, the arrangement position determination process of the second embodiment will be described. FIG. 20 is a flowchart of the arrangement position determination process according to the second embodiment. In step S <b> 705, the CPU 410 determines whether or not there is one or more two-end combinations determined to be processing targets of the first search processing in the above-described placement position processing target determination processing.

  When there is one or more two-end combinations determined to be the processing target of the first search process (S705: YES), in S710, the CPU 410 determines from the one or more two-end combinations. One combination of two ends to be processed is selected. In S715, the first search process is executed for the two combinations of processing targets that have been selected. In S720, the CPU 410 determines whether or not all of the one or more two-end combinations described above that have been determined to be the target of the first search process have been processed. If there is an unprocessed two-end combination (S720: NO), the process returns to S710 to select the two-end combination. If all of the one or more two-end combinations described above have been processed (S720: YES), the CPU 410 advances the process to S725.

  If there is no two-end combination determined to be the processing target of the first search process (S705: NO), the CPU 410 skips S710 to S720 and advances the process to S725.

  In S725, the CPU 410 determines whether or not there is one or more two-end combinations determined to be processing targets of the second search process in the above-described placement position processing target determination processing.

  When there is one or more two-end combinations determined to be the processing target of the second search process (S725: YES), in S730, the CPU 410 determines from the one or more two-end combinations. One combination of two ends to be processed is selected. In S735, the second search process is executed for the two combinations of processing targets that have been selected. In S740, the CPU 410 determines whether or not all of the one or more combinations of the two ends determined to be processing targets of the second search process have been processed. If there is an unprocessed two-end combination (S740: NO), the process returns to S730 to select the two-end combination. If all of the one or more two-end combinations described above have been processed (S740: YES), the CPU 410 advances the process to S745.

  If there is no combination of the two ends determined to be the processing target of the second search process (S725: NO), the CPU 410 skips S730 to S740 and proceeds to S745.

  In S745, the CPU 410 determines whether there is a combination of two ends for which a similar region is determined in the first search process of S715 or the second search process of S735 among the two combinations of processing targets. Judging. Normally, when the scan pattern A is used, there is a high possibility that the similar region CPa is determined in the first search process for a combination of two ends that can correctly reproduce the document 10 by joining the two. . Further, when the scan pattern B is used, there is a high possibility that the similar region CPb is determined in the second search process for a combination of two ends that can correctly reproduce the original 10 by joining the two.

  When there are one or more two-end combinations for which similar regions have been determined (S745: YES), in S755, the CPU 410 determines that the similar region has the highest similarity among the one or more two-end combinations. The arrangement position is determined by the combination of the two ends determined.

  When the similar region with the highest similarity rate is determined in the first search process of S715, the reference used in the first search process for the combination of the two ends is the same as in the first example. The first-type arrangement positions of the two scanned images I11 and I12 (that is, the first-type arrangement positions of the two document images) so that the area SPa and the determined similar area CPa overlap. Is determined (see FIG. 7).

  On the other hand, when the similar region with the highest similarity is determined in the second search process of S735, based on the position of the similar region CPb with the highest similarity determined in the second search process. A second type arrangement position of the two scanned images I11 and I12 (that is, a second type arrangement position of the two document images) is determined. This will be described with reference to the example of FIG. In the example of FIG. 16B, the combination of the two ends where the similarity region CPb having the highest similarity is determined is the right end RT1b of the document image in the first scan image I12b and the document image in the second scan image I22b. And the left end LT2b. As shown in this example, the relative positions in the horizontal direction are set so that the two ends constituting the combination of the two ends where the similar region CPb having the highest similarity rate is determined face each other with a predetermined gap ΔIT therebetween. Is determined. Then, the pixel position Ps within the reference range REb corresponding to the similar region CPb having the highest similarity is specified. For example, when the similar region CPb having the highest similarity is the candidate region NT1 in FIG. 18, the pixel position Pt1 of the reference range REb is the pixel position Ps in the reference range REb corresponding to the similar region CPb. . Then, as shown in FIG. 16B, the vertical position of the pixel position Ps within the specified reference range REb is determined as the vertical position of the upper end of the second scan image I22. In FIG. 16B, in order to avoid complication of the drawing, the margins and additional regions in the scan images I12b and I22b are not shown. Since the reference range REb is set for the first scan image I12b, the relative vertical positions of the scan images I12b and I22b are thereby determined.

  If there is no combination of two ends for which similar regions have been determined (S745: NO), in S750, the CPU 410 compares the two scanned images I12 and I22 (two original images AIL and AIR) with each other. The placement position is determined as the default position. The default position is the same as the default position of the first embodiment, for example.

  According to the second embodiment described above, when the amount of margin at the right end of the first scan image I12 is greater than or equal to the reference, the processing target determination process in FIG. 19 determines the document image in the first scan image I12. The combination of the two ends including the right end is not determined as the processing target of the first search process. Therefore, in this case, the first search process is not executed using the edge image along the right edge of the document image in the first scan image I12. In this case, in the processing target determination process of FIG. 19, the combination of the two ends including the right end of the document image in the first scan image I12 can be determined as the processing target of the second search process. Therefore, in this case, as a result, the second search process is executed using the edge image along the right edge of the document image in the first scan image I12.

  As described above, when the amount of margin at the right end of the first scan image I12 is equal to or larger than the reference, the second search to be executed using the end image along the right end of the document image in the first scan image I12. The process is executed, and the first search process that does not need to be executed is not executed. As a result, the processing time required for the search process can be reduced. Moreover, the possibility that an incorrect arrangement position is determined can be reduced by using an incorrect type of search processing.

  Furthermore, when the first scan image I12 is a double-sided margin image, that is, the right margin amount of the first scan image I12 is greater than or equal to the reference, and the left margin amount of the first scan image I12 is greater than or equal to the reference. In some cases, it is unlikely that scan pattern A was used, and scan pattern B may be used. In this case, in S665 of the process target determination process of FIG. 19, all the combinations of the two ends are not determined as the process targets of the first search process (see S205 to S225 of FIG. 9). In this case, all the combinations of the two ends are determined as the processing targets of the second search process (S670 in FIG. 19). Thus, in this case, the CPU 410 executes the second search process without executing the first search process. Therefore, since only the second search process is executed without executing the unnecessary first search process, an appropriate arrangement position can be determined and the processing time can be reduced.

  Further, when the first scan image I12 is an image with no margins on both sides, that is, the margin amount at the right end of the first scan image I12 is less than the reference, and the margin amount at the left end of the first scan image I12 is less than the reference. In some cases, it is unlikely that the scan pattern B was used, and it is considered that the scan pattern A was used. In this case, in S645 or S655 of the processing target determination process of FIG. 19, at least one two-end combination is determined as the processing target of the first search process. In this case, all combinations of the two ends are not determined as the processing target of the second search process (S650 or S660 in FIG. 19). Thus, in this case, the CPU 410 executes the first search process and does not execute the second search process. Therefore, only the first search process is executed without executing the unnecessary second search process, so that an appropriate arrangement position can be determined and the processing time can be reduced.

  Further, when both of the two scanned images I12 and I22 are marginal images (both side marginal images or one side marginal images), that is, the margin amount at one of the left and right sides of the first scan image I12. 19 is equal to or greater than the reference, and the margin amount at least one of the left and right ends of the second scan image I22 is equal to or greater than the reference, S670 in FIG. 19 is executed, and all the combinations of the two ends are determined. , It is determined that it is a processing target of the second search process. Therefore, in this case, the CPU 410, regardless of the margin amount at the other left and right edges of the first scan image I12 and the margin amount at the other left and right edges of the second scan image I22, The second search process is executed for all combinations of two ends.

  When the scan pattern B is used, a margin is formed at at least one of the left and right ends of each scan image (FIG. 14). In this case, the marginal amount alone cannot separate the two-end combination that should be the target of the second search process and the two-end combination that should not be the target of the second search process. . In the present embodiment, in such a case, the second search process for all combinations of two ends is executed, so that an appropriate arrangement position can be determined.

  By the way, when the scan pattern A is used, the central portion CA of the document 10 in FIG. 3 is located in the document image in the first scan image I12 and in the second scan image I22 as shown in FIG. Appears both in the original image. For this reason, it is appropriate to determine the first type arrangement position where these two document images overlap. On the other hand, when the scan pattern B is used, the linear image positioned on the polygonal line VL of the document 10 is an original image in the first scan image I12 and an original image in the second scan image I22. Neither appears. Further, there is no area on the document 10 that appears in both of these two document images. For this reason, it is appropriate to determine the second type of arrangement position where these two document images do not overlap. According to the present embodiment, an appropriate arrangement position is determined by executing an appropriate search process corresponding to the used scan pattern. As a result, processed image data representing a processed image that can appropriately reproduce the document 10 can be generated.

C. Third embodiment:
In the third embodiment, the arrangement position determining process in S65 of FIG. 2 is different from the first embodiment. Further, in the third embodiment, a processing order determination process indicated by a broken line in FIG. 2 is performed between the processing target determination process in S55 of FIG. 2 and the arrangement position determination process in S65. Other processes in the third embodiment are the same as those in the first embodiment. Further, in the third embodiment, it is assumed that the scan pattern A is used as in the first embodiment, and it is not assumed that the scan pattern B is used.

C-1. Arrangement position determination processing:
FIG. 21 is a flowchart of the arrangement position determination process according to the third embodiment. In S260, as in S230 of FIG. 10, the CPU 410 selects one combination from one or more two-end combinations determined to be the processing target of the first search process in the processing target determination process. Select. In S265, as in S235 of FIG. 10, the CPU 410 executes the first search process for the selected combination of the two ends.

  In S270, the CPU 410 determines whether or not a similar area CPa has been determined for the selected combination of the two ends in the first search process in S265. When the similar region CPa is determined (S270: YES), in S275, the CPU 410 determines the first type arrangement position by the combination of the two ends to be processed, and the arrangement position determination process. Exit. That is, the first scan image I11 and the second scan image are overlapped so that the reference region SPa used in the first search process for the two end combinations of the current processing target overlaps the determined similar region CPa. A placement position relative to I22 is determined.

  If the similar region CPa has not been determined (S270: NO), in S280, the CPU 410 determines whether or not all the combinations of the two ends that have been determined to be subject to the first search process have been selected. to decide. When there is an unselected combination of two ends (S280: NO), the CPU 410 returns to S260 and selects the combination of the two ends. If all the combinations of the two ends to be processed have been selected (S280: YES), in S285, the CPU 410 determines the relative arrangement position of the two scan images I12 and I22 as a default position. Then, the arrangement position determination process ends.

  As can be understood from the above description, in the third embodiment, when there are a plurality of combinations of two ends determined to be processing targets of the first search process, the first search process for the first combination is performed. When the similar region CPa is determined, the first search process is not executed for the second combination whose processing order is later than the first combination. This is different from the first embodiment in which the first search process is always executed for all the combinations of the two ends determined as the process targets in the process target determination process.

  For this reason, in the third embodiment, when there are a plurality of combinations of two ends determined to be processing targets of the first search process, the first search process is performed for these combinations of two ends. The overall processing time can be reduced by determining the processing order so that the document 10 is processed first in the order that the possibility of correctly reproducing the document 10 is high. Therefore, in this embodiment, the processing order determination process in S60 of FIG. 2 is added.

C-2. Processing order determination processing:
FIG. 22 is a flowchart of the processing order determination process according to the third embodiment. In S800, the CPU 410 determines whether there are a plurality of combinations of two ends determined to be processing targets of the first search processing in the processing target determination processing. If the number of combinations at the two ends determined to be the processing target of the first search process is one (S800: NO), the CPU 410 ends the process because it is not necessary to determine the processing order.

  If there are a plurality of combinations of two ends determined to be the processing target of the first search process (S800: YES), in S805, the CPU 410 selects from the plurality of combinations of two ends. Select one combination. In S810, the CPU 410 makes a combination of the first end image TI1 along the edge of the original image in the first scan image I12 constituting the selected combination of the two ends and the second combination of the selected two ends. Histograms of the second end image TI2 along the edge of the document image in the scan image I22 are generated.

  As a specific example, in FIG. 6B, two ends constituting the selected combination of the two ends are the right end RT1 of the document image in the first scan image I12 and the document image in the second scan image I22. The first end image TI1 and the second end image TI2 in the case of the right end RT2 are illustrated. As shown in this example, the end images TI1 and TI2 extend over the entire length of the processing target end along the processing target end of the scan images I12 and I22 (both are right ends in FIG. 6B). It is a partial image extending. The width Wp in the horizontal direction of the end images TI1 and TI2 is, for example, the width Wca of the central portion CA (FIG. 3) of the document 10, that is, the width Wca of the region that can be expressed in both of the two scan images I12 and I22. Is a value determined in advance. The maximum value of the width Wca corresponds to, for example, the assumed width WS in the specific direction of the original 10 (in the example of FIG. 3, the width in the longitudinal direction of A3) and the specific direction of the original 10 when the original 10 is read. A value (2 × WT−WS) determined based on the width WT in the direction of the document table 255 (in the example of FIG. 4, the width in the short direction of the document table 255). For example, the width Wp is set to a value of 70% to 100% of the maximum value of the width Wca.

  The CPU 410 generates a histogram of each RGB component value of the first end image TI1 and a histogram of each RGB component of the second end image TI2. For example, the histogram of the R component is generated by classifying each pixel in the target end image into a plurality of classes according to the R component value of each pixel. In this embodiment, histogram data is generated with each of the 256 gradation values that the R component value can take as one class. The same applies to histograms of other component values.

  The histogram may not be generated for all the pixels in the end images TI1 and TI2, but may be generated for some of the pixels in the end images TI1 and TI2. For example, a histogram may be generated for some pixels selected to be uniformly distributed in the end images TI1 and TI2, for example, some pixels selected every several pixels. In addition, the histogram is not limited to the histogram of each component of RGB, and may be a histogram of all or part of luminance, saturation, and hue, for example.

  FIG. 23 is a diagram illustrating an example of an R component histogram of the end images TI1 and TI2. The broken line is a histogram of the first end image TI1, and the solid line is a histogram of the second end image TI2.

  In S815, the CPU 410 calculates an overlapping area Sov between the histogram of the first end image TI1 and the histogram of the second end image TI2. In FIG. 23, the overlapping area of the R component is shown by hatching. For example, for each of the 256 classes of R component histograms, the CPU 410 counts the number of pixels (frequency) of the histogram of the first end image TI1 and the number of pixels (frequency) of the histogram of the second end image TI2. And the smaller of the two values is specified. Then, the CPU 410 calculates the total value of the smaller values specified for each of the 256 classes as an overlapping area of R components. The CPU 410 similarly calculates the overlapping area for the G component and the B component, and calculates the total value of the overlapping areas of the three RGB components as the overlapping area Sov. The overlapping area Sov is a feature value indicating the similarity between the histograms of the two end images TI1 and TI2. A larger overlap area Sov indicates that the histograms of the two end images TI1 and TI2 are more similar. In the modification, the overlapping area of any one of the three components of RGB may be calculated as the overlapping area Sov. In particular, when the luminance is generated as a histogram, the overlapping area Sov may be calculated using only the luminance.

  In S820, it is determined whether all combinations of two ends determined to be processing targets have been selected. If there is an unselected combination of two ends (S820: NO), the CPU 410 returns to S805 and selects the combination of the two ends. When all the combinations of the two ends to be processed have been selected (S820: YES), the CPU 410 advances the process to S825. At the time when the process proceeds to S825, the overlap area Sov is calculated for all combinations of two ends determined to be processing targets.

  In S825, the CPU 410 determines the processing order of a plurality of combinations of two ends determined to be processing targets in descending order of the overlapping area Sov, and ends the processing order determination process.

  According to the third embodiment described above, when the first search process is executed for each of a plurality of two-end combinations, two of the plurality of two-end combinations are included in the combination. A histogram of two edge images TI1 and TI2 along the edge is generated (S810). Then, using the histogram, the order of the first search processing for a combination having a relatively high similarity between the histograms of the two end images TI1 and TI2 is similar to the histogram of the two end images TI1 and TI2. The processing order of a plurality of two-end combinations is determined so as to precede the order of the first search processing for the combination having a relatively low degree (S815, S825). As a result, the first search processing is executed according to the processing order determined so that the order of the first search processing for the combination having the high similarity of the histograms of the two end images TI1 and TI2 is first, When the arrangement position is determined for a specific combination, the first search process is not executed for a combination whose order is determined after the specific combination among a plurality of combinations. Processing time can be further reduced. This is because the two end images TI1 and TI2 constituting the combination that can reproduce the document 10 are considered to be similar to the two end images TI1 and TI2 constituting the other combinations. Since the processing time required for the histogram generation process is significantly shorter than that of the first search process, the process of the first search process is more than the increase of the process time by adding the process order determination process. The expected value that can reduce the time is larger.

D. Fourth embodiment:
The fourth embodiment differs from the third embodiment in the arrangement order determination process. Other processes in the fourth embodiment are the same as those in the third embodiment. FIG. 24 is a flowchart of the arrangement order determination process according to the fourth embodiment. The processes in S900 to S915 in FIG. 24 are the same as the processes in S800 to S815 in FIG.

  In S920, the CPU 410 determines whether or not the overlapping area Sov calculated for the selected combination of the two ends is equal to or greater than the threshold value THs. The threshold value THs is a value that is experimentally predetermined as a value of the overlapping area Sov indicating that the two end images TI1 and TI2 are somewhat similar. For example, the threshold value THs is a value that is 30 to 60% of the area of the histogram of the first end image TI1.

  When the overlapping area Sov is equal to or greater than the threshold value THs (S920: YES), the CPU 410 determines the selected combination of the two ends as a priority target in S930. When the overlapping area Sov is less than the threshold value THs (S920: NO), the CPU 410 determines the selected combination of the two ends as a non-priority target in S935.

  In S940, the CPU 410 calculates the dynamic range DR of the histograms of the two end images TI1 and TI2. For example, the number of pixels (frequency) in the histogram of the R component of the first end image TI1 is sequentially added from the class of the minimum value (0) toward the class of the component value larger than the minimum value. Then, the R component value of the class when the added value exceeds a predetermined ratio (5% in this embodiment) of the number of pixels of the first end image TI1 is determined as the lower limit value R1. Further, the number of pixels (frequency) in the histogram of the R component of the first end image TI1 is sequentially added from the class of the maximum value (255) toward the class of the component value smaller than the maximum value. Then, the R component value of the class when the added value exceeds the predetermined ratio (5% in this embodiment) is determined as the upper limit value R2. The dynamic range DR1_R of the R component value of the first end image TI1 is determined as (R2-R1). Similarly, the dynamic range DR2_R of the R component value of the second end image TI2 is determined, and the sum of the dynamic ranges of the R component values of the two end images TI1 and TI2 (DR1_R + DR2_R) is the dynamic value of the R component. Calculated as the range DR_R. Similarly, the dynamic ranges DR_G and DR_B of the G component and the B component are calculated, and the sum of the dynamic ranges of the three components is calculated as the dynamic range DR to be calculated (DR = DR_R + DR_G + DR_B).

  The dynamic range DR is a feature value indicating the complexity of the two end images TI1 and TI2. For example, when the two end images TI1 and TI2 represent a relatively simple image such as a solid region such as the background of the document 10, the dynamic range DR of the two end images TI1 and TI2 is: Relatively small. When the two end images TI1 and TI2 represent a relatively complicated image such as an object in the document 10, the dynamic range DR of the two end images TI1 and TI2 becomes relatively large.

  In S945, it is determined whether or not all combinations of two ends determined to be processing targets have been selected. If there is an unselected combination of two ends (S945: NO), the CPU 410 returns to S905 and selects the combination of the two ends. When all the combinations of the two ends to be processed have been selected (S945: YES), the CPU 410 advances the process to S950. At the time when the process proceeds to S950, the dynamic range DR of the two end images TI1 and TI2 is calculated for all combinations of two ends determined to be processed, and the combination of the two ends is determined. All are determined to be either priority objects or non-priority objects.

  In S950, the CPU 410 determines the processing order in the descending order of the dynamic range DR for one or more two-end combinations determined as priority targets.

  In S955, the CPU 410 determines the processing order of the one or more combinations of two ends determined as non-priority targets in descending order of the dynamic range DR. However, the processing order of one or more two-end combinations determined as non-priority objects is determined to be an order after the one or more two-end combinations determined as priority objects.

  According to the fourth embodiment described above, when the first search process is executed for each of a plurality of two-end combinations, two end images TI1 for each of the plurality of two-end combinations. , A feature value (specifically, dynamic range DR) indicating the complexity of TI2 is calculated (S940). Then, the processing order of the combination of two ends with relatively high complexity of the two end images TI1, TI2 is the processing order of the combination of two ends with relatively low complexity of the two end images TI1, TI2. The processing order of a plurality of two-end combinations is determined so as to come further.

  Since the two end images TI1 and TI2 constituting a combination that can correctly reproduce the document 10 are images showing the central portion CA of the document 10, there is a high possibility that the main object of the document 10 is included. For this reason, there is a high possibility that the two end images TI1 and TI2 constituting a combination capable of correctly reproducing the document 10 are relatively complicated images. On the other hand, the two end image TI <b> 1 and TI <b> 2 constituting another combination include an image showing an end region of the document 10. For example, the edge area of the document 10 may include a simple image such as a background, and thus may be a simple image as compared to the central portion CA. For this reason, as described above, the first search process is executed in accordance with the processing order determined so as to prioritize the combination of the two ends where the two end images TI1 and TI2 are complicated. The processing time of one search process can be further reduced. Furthermore, it is possible to reduce the possibility that the placement position is erroneously determined by a combination of two ends where the document 10 cannot be reproduced correctly.

D. Variations:
(1) When the processing order determination process of S60 of FIG. 2 is executed as in the third and fourth embodiments, the processing target determination process of S55 of FIG. 2 may be omitted. In this case, the processing order may be determined for all combinations of two ends in S60 of FIG. In this case, for example, when it is assumed that only the scan pattern B is used, the second search process may be performed instead of the first search process. That is, the arrangement position determined in the third and fourth embodiments may be an arrangement position where two target images overlap or may not be overlapped. Any arrangement position may be used as long as the end image along one of the ends and the end image along one of the other target images are associated with each other.

(2) In the second embodiment, when it is determined in S35 and S40 of FIG. 2 that the margin amount at both ends of the first scan image I12 is less than the reference, the scan pattern A is used, and the scan pattern It is considered that B is not used. In this case, the CPU 410 may omit the margin amount determination process of the second scan image I22 in S45 and S50 of FIG. In the processing target determination process of S55, the CPU 410 determines that all the two end combinations are processing targets for the first search process, and all the two end combinations are not processing targets for the second search process. It may be determined.

(3) In the second embodiment, when it is determined in S35 and S40 of FIG. 2 that the margin amount at both ends of the first scan image I12 is equal to or larger than the reference, the scan pattern B is used, and the scan pattern It is thought that A is not used. In this case, the CPU 410 may omit the margin amount determination process of the second scan image I22 in S45 and S50 of FIG. In the processing target determination process of S55, the CPU 410 determines that all the two end combinations are the processing targets of the second search process, and does not set all the two end combinations as the processing targets of the first search process. It may be determined.

(4) The overlapping area Sov of the two histograms calculated in the third embodiment is an example of a feature value indicating the similarity between the two histograms, and is not limited thereto. The feature value indicating the similarity between the two histograms may be another feature such as the degree of coincidence of a plurality of peak positions of the two histograms. The dynamic range of the two histograms calculated in the third embodiment is an example of a feature value indicating the complexity of the two end images TI1 and TI2, and is not limited thereto. The feature values indicating the complexity of the two end images TI1 and TI2 are, for example, other feature values such as the variance values of the two histograms and the edge amounts of the two end images TI1 and TI2. Also good.

(5) In the first embodiment, the processes of S45 and S50 in FIG. 2 may be omitted. That is, the margin amount determination process at both ends of the first scan image I12 may be executed, and the margin amount estimation process of the second scan image I22 may be omitted. In this case, for example, when only one margin amount of the left and right ends of the first scan image I12 is less than the reference value, only the margin amount of the left and right ends of the first scan image I12 is obtained. It is only necessary to determine two combinations including an end that is less than the reference value as processing targets of the first search process and determine that the remaining two combinations are not processing targets of the first search process. . Further, when the margin amounts at the left and right ends of the first scan image I12 are less than the reference value, all combinations may be determined to be processing targets of the first search process.

(6) In the scan pattern B, scan data representing images each including two document images BIL and BIR is generated by reading both sides of the document 10 folded in half by the broken line VL (FIG. 2). S15). Instead of this, scan data representing images each including two document images BIL and BIR is obtained by reading the document 10 which is cut by the broken line VL and physically separated into two parts in two steps. May be generated.

(7) In the margin amount determination processing of the above embodiment, basically, an area between the edge of the scanned image and the edge of the document image is a blank, but the edge (background) of the document is white. If the color of the platen is white, the edge of the original image cannot be determined, and there is a possibility that a margin is left between the edge of the scanned image and the edge of the object of the original image. There is. Even in this case, the arranged image is generated appropriately without any problem. Specifically, since it is assumed that there is an object at the central portion (end to be connected) of the original, the margin is correctly determined at the end corresponding to the central portion of the original of the scanned image. Therefore, in the first and third embodiments, the margin amount is less than the reference value at the end corresponding to the central portion of the document among the both ends of the scanned image, and the margin amount is greater than the reference value at the end corresponding to the end portion of the document. It becomes. For this reason, as a result, the arranged image is generated so that the end corresponding to the central portion of the document is located in the central portion of the arranged image. In the second and fourth embodiments, the combination of two ends including the end corresponding to the center portion of the document and the combination of two ends including the end corresponding to the end portion of the document are always subjected to the search process. Will be. Therefore, in any of the first to fourth embodiments, the arranged image is generated appropriately without any problem.

(8) In the above embodiment, arranged image data representing an arranged image in which two document images in the two scanned images I12 and I22 are arranged in the horizontal direction is generated. Instead of this, arranged image data representing an arranged image in which one original image and another original image are arranged in the vertical direction may be generated. In this case, for example, it is determined whether or not the margin amount at the lower end of the scan image including one original image and the margin amount at the upper end of the scan image including another original image are equal to or greater than a reference, respectively. . In the processing target determination process in S55 of FIG. 2, it is only necessary to determine whether or not the combination of the lower end and the upper end of the two document images is to be a search processing target based on the determination result. . Further, in the arrangement order determination processing in S60 of FIG. 2, the processing order may be determined for the combination of the lower end and the upper end of the two document images.

(9) The processing executed by the CPU 410 of the server 400 in the above embodiment (for example, the processing of S25 to S70 in FIG. 2) may be executed by the CPU 210 of the multifunction device 200, for example. In this case, the server 400 is unnecessary, and the multi-function device 200 may execute the processing of FIG. 2 alone. Further, the processing executed by the CPU 410 of the server 400 may be executed by the CPU (not shown) of the personal computer 500 (FIG. 1) connected to the multifunction device 200. For example, the CPU of the personal computer 500 may execute these processes by executing a scanner driver program installed in the personal computer 500. Further, the server 400 may be configured by one computer as in the present embodiment, or may be configured by a computer system including a plurality of computers that can communicate with each other.

(10) In the above embodiment, the server 400 acquires the scan data in the image file format in S25 of FIG. 2, and outputs (transmits) the arranged image data in the image file format in S75 of FIG. )doing. Instead, for example, when the CPU 210 of the multi-function peripheral 200 executes the processes of S25 to S45 as in the above modification, the CPU 210 converts the scan data generated using the scanner unit 250 into an image file. You may acquire as it is, without converting. Further, the CPU 210 may output the arranged image data by printing the arranged image data on the paper as it is without converting the arranged image data into an image file.

(11) In the above embodiment, arranged image data representing an arranged image in which two document images are arranged is generated using two image data. However, the present invention is not limited to this, and one arranged image may be generated using an arbitrary number of image data. For example, arranged image data representing an arranged image in which four document images are arranged may be generated using four image data.

(12) In the above embodiment, the two pieces of image data used for generating the arranged image data are generated by the scanner unit 250 of the multifunction device 200. The present invention is not limited to this, and various image data representing an optically read image can be employed. For example, two pieces of image data may be generated by optically reading both sides of the document 10 folded in half by photographing with a digital camera. Alternatively, two image data are generated by optically reading the left area 10L and the partial area CAR of the document 10 and the right area 10R and the partial area CAL of the document 10 by photographing with the digital camera. May be. Further, these image data are not limited to image data generated by a reading device (scanner or digital), but may be image data generated using an application program such as drawing creation or document creation.

(13) In the above embodiment, a part of the configuration realized by hardware may be replaced with software, and conversely, a part of the configuration realized by software may be replaced with hardware. Also good.

  As mentioned above, although this invention was demonstrated based on the Example and the modification, Embodiment mentioned above is for making an understanding of this invention easy, and does not limit this invention. The present invention can be changed and improved without departing from the spirit and scope of the claims, and equivalents thereof are included in the present invention.

  200 ... multifunction device, 210 ... CPU, 220 ... volatile storage device, 221 ... buffer area, 230 ... nonvolatile storage device, 231 ... control program, 240 ... printer , 250 ... scanner unit, 255 ... manuscript table, 260 ... operation unit, 270 ... display unit, 280 ... communication unit, 400 ... server, 410 ... CPU, 420 ... volatile storage device, 421 ... buffer area, 430 ... nonvolatile storage device, 431 ... computer program, 433 ... UI data group, 480 ... communication unit, 500 ... Personal computer, 1000 ... Image processing system

Claims (12)

An image acquisition unit that acquires first image data representing a first image including a first target image and second image data representing a second image including a second target image;
Determining whether or not the number of first pixels included between an end in the first direction of the first target image and an end in the first direction of the first image is equal to or greater than a reference; The determination part of
The number of continuous second pixels included between an end in the second direction opposite to the first direction of the first target image and an end in the second direction of the first image is greater than or equal to a reference. A second determination unit for determining whether there is,
An end image along one end of the first target image and the second direction of the first target image, and one of the first direction and the second direction of the second target image. Relative arrangement positions of the first target image and the second target image by executing a first search process for searching for a first type arrangement position where the end image along the edge overlaps. A first position determining unit for determining
The first target image and the second target image are arranged at the determined arrangement position, and the first target image and the second target image represent a arranged image indicating one object. An image generation unit for generating arranged image data;
The first position determination unit includes:
When the number of the first pixels is less than the reference, an end image along the end in the first direction of the first target image and an end in the first direction of the second target image Performing the first search process using at least one of an end image along the end image along the end in the second direction of the second target image,
When the number of the first pixels is equal to or greater than a reference, the first search process is executed without using an end image along the end in the first direction of the first target image,
When the number of the second pixels is less than the reference, an end image along the end in the second direction of the first target image and an end in the first direction of the second target image Performing the first search process using at least one of an end image along the end image along the end in the second direction of the second target image,
When the number of the second pixels is equal to or greater than a reference, the first search process is executed without using the end image along the end in the second direction of the first target image. apparatus. The image processing apparatus according to claim 1, further comprising:
Determining whether or not the number of third pixels included between an end in the first direction of the second target image and an end in the first direction of the second image is greater than or equal to a reference; The determination part of
Determining whether or not the number of fourth pixels included between the second direction end of the second target image and the second direction end of the second image is greater than or equal to a reference; 4 determination units;
With
The first position determination unit includes:
When the number of the third pixels is less than the reference, an end image along the end in the first direction of the second target image and an end in the first direction of the first target image Performing the first search process using at least one of an end image along the end image along the end in the second direction of the first target image,
When the number of the third pixels is equal to or greater than a reference, the first search process is performed without using an end image along the end in the first direction of the second target image,
When the number of the fourth pixels is less than the reference, an end image along the end in the second direction of the second target image and an end in the first direction of the first target image Performing the first search process using at least one of an end image along the end image along the end in the second direction of the first target image,
When the number of the fourth pixels is equal to or greater than a reference, the first search process is executed without using an end image along the end in the second direction of the second target image. apparatus. The image processing apparatus according to claim 2,
The first position determination unit includes:
The number of the first pixels is less than the reference, the number of the second pixels is more than the reference, the third pixel is less than the reference, and the number of the fourth pixels Is equal to or greater than the reference, the end image along the end in the second direction of the first target image and the end image along the end in the second direction of the second target image are not used. In addition, the first search process is performed using an end image along the end in the first direction of the first target image and an end image along the end in the first direction of the second target image. An image processing apparatus that executes The image processing apparatus according to any one of claims 1 to 3, further comprising:
An end image along one end of the first target image and the second direction of the first target image, and one of the first direction and the second direction of the second target image. Relative arrangement of the first target image and the second target image by executing a second search process for searching for a second type of arrangement position that does not overlap the end image along the end. A second position determining unit for determining a position;
When the number of the first pixels is greater than or equal to a reference, the first position determination unit uses the end image along the end in the first direction of the first target image, and The search process of
When the number of the first pixels is greater than or equal to a reference, the second position determination unit uses the end image along the end in the first direction of the first target image, and An image processing apparatus that executes the search process. The image processing apparatus according to claim 4,
The first position determination unit does not execute the first search process when the number of the first pixels is equal to or larger than the reference and the number of the second pixels is equal to or larger than the reference. ,
The second position determination unit executes the second search process when the number of the first pixels is equal to or larger than the reference and the number of the second pixels is equal to or larger than the reference. Image processing device. The image processing apparatus according to claim 4 or 5, wherein
The first position determination unit executes the first search process when the number of the first pixels is less than a reference and the number of the second pixels is less than a reference.
The second position determination unit does not execute the second search process when the number of the first pixels is less than the reference and the number of the second pixels is less than the reference. Image processing device. The image processing apparatus according to claim 2 or 3, further comprising:
An end image along one end of the first target image and the second direction of the first target image, and one of the first direction and the second direction of the second target image. Relative arrangement of the first target image and the second target image by executing a second search process for searching for a second type of arrangement position that does not overlap the end image along the end. A second position determining unit for determining a position;
In the second position determination unit, the number of one of the first pixel and the second pixel is equal to or more than a reference, and the second position determination unit includes the third pixel and the fourth pixel. When the number of one is equal to or more than a reference, the number of the other of the first pixel and the second pixel and the other of the third pixel and the fourth pixel Regardless of the number, any one of the first direction and the second direction of the first target image, and the first direction and the second direction of the second target image. An image processing apparatus that executes the second search process for all combinations with any one of the ends. The image processing apparatus according to claim 1, further comprising:
Any one of the first direction and the second direction of the first target image, one of the first direction and the second direction of the second target image, and When the first search process is executed for each of the plurality of combinations, a histogram of two edge images along two edges constituting the combination is generated for each of the plurality of combinations. A histogram generator;
Using the histogram, the order of the first search processing for a combination having a relatively high similarity between the histograms of the two end images is relatively low. An order determining unit that determines the order of the plurality of combinations so as to precede the order of the first search processing for combinations;
With
The first position determination unit includes:
Performing the first search process for the plurality of combinations according to the determined order;
Among the plurality of combinations, when the first type arrangement position is determined for a specific combination, among the plurality of combinations, the combination whose determined order is later than the specific combination, An image processing apparatus that does not execute the first search process. An image acquisition unit that acquires first image data representing a first image including a first target image and second image data representing a second image including a second target image;
For each of a plurality of combinations of any one of the plurality of ends of the first target image and any one of the plurality of ends of the second target image. A histogram generator for generating a histogram of two edge images along two edges constituting
Using the histogram, the search processing order for a combination with a relatively high degree of similarity between the two edge images is the search for a combination with a relatively low degree of similarity between the two edge images. An order determining unit that determines the order of the plurality of combinations so as to precede the processing order, wherein the search processing includes an end image along one end of the first target image, and the end image The order determination unit, which is a process of searching for a position that associates an end image along one of the ends of the second target image;
A position determination unit that determines a relative arrangement position of the first target image and the second target image by executing the search process;
The first target image and the second target image are arranged at the determined arrangement position, and the first target image and the second target image represent a arranged image indicating one object. An image generation unit for generating arranged image data;
With
The position determination unit
Performing the search process for the plurality of combinations according to the determined order;
When the arrangement position is determined for a specific combination among the plurality of combinations, the search process is performed for a combination after the determined order after the specific combination among the plurality of combinations. An image processing device that does not execute. The image processing apparatus according to claim 9, further comprising:
For each of the plurality of combinations, a calculation unit that calculates a feature value indicating the complexity of two end images along two ends constituting the combination,
The order determination unit uses the feature value to determine that the combination of the two end images has a relatively high complexity before the combination of the two end images has a relatively low complexity. An image processing apparatus that determines the order of the plurality of combinations. An image acquisition function for acquiring first image data representing a first image including a first target image and second image data representing a second image including a second target image;
Determining whether or not the number of first pixels included between an end in the first direction of the first target image and an end in the first direction of the first image is equal to or greater than a reference; Judgment function,
The number of continuous second pixels included between an end in the second direction opposite to the first direction of the first target image and an end in the second direction of the first image is greater than or equal to a reference. A second determination function for determining whether there is,
An end image along one end of the first target image and the second direction of the first target image, and one of the first direction and the second direction of the second target image. Relative arrangement positions of the first target image and the second target image by executing a first search process for searching for a first type arrangement position where the end image along the edge overlaps. A first position determining function for determining
The first target image and the second target image are arranged at the determined arrangement position, and the first target image and the second target image represent a arranged image indicating one object. An image generation function for generating arranged image data;
Is realized on a computer,
The first position determination function includes:
When the number of the first pixels is less than the reference, an end image along the end in the first direction of the first target image and an end in the first direction of the second target image Performing the first search process using at least one of an end image along the end image along the end in the second direction of the second target image,
When the number of the first pixels is equal to or greater than a reference, the first search process is executed without using an end image along the end in the first direction of the first target image,
When the number of the second pixels is less than the reference, an end image along the end in the second direction of the first target image and an end in the first direction of the second target image Performing the first search process using at least one of an end image along the end image along the end in the second direction of the second target image,
A computer program that executes the first search process without using an end image along an end in the second direction of the first target image when the number of the second pixels is equal to or larger than a reference. . An image acquisition function for acquiring first image data representing a first image including a first target image and second image data representing a second image including a second target image;
For each of a plurality of combinations of any one of the plurality of ends of the first target image and any one of the plurality of ends of the second target image. A histogram generation function for generating histograms of two edge images along two edges constituting
Using the histogram, the search processing order for a combination with a relatively high degree of similarity between the two edge images is the search for a combination with a relatively low degree of similarity between the two edge images. An order determination function for determining the order of the plurality of combinations so as to precede the processing order, wherein the search processing includes an end image along one end of the first target image and the end image The order determination function, which is a process of searching for a position that associates an end image along one of the ends of the second target image;
A position determination function for determining a relative arrangement position of the first target image and the second target image by executing the search process;
The first target image and the second target image are arranged at the determined arrangement position, and the first target image and the second target image represent a arranged image indicating one object. An image generation function for generating arranged image data;
Is realized on a computer,
The position determination function is:
Performing the search process for the plurality of combinations according to the determined order;
When the arrangement position is determined for a specific combination among the plurality of combinations, the search process is performed for a combination after the determined order after the specific combination among the plurality of combinations. A computer program that does not run.

Images