JP6149782B2 - Image processing apparatus and computer program - Google Patents

Description

  The present invention relates to a technique for generating composite image data representing an image obtained by combining a first image and a second image.

  A technique for generating composite image data representing an image obtained by combining a first image and a second image is known. For example, in the technique disclosed in Patent Document 1, an original having a size that cannot be read at one time is read in two using a scanner, and scan data representing the first image and the second image are read. Is obtained. Then, output image data representing an image obtained by combining the first image and the second image is generated using the two scan data. The position where the first image and the second image are combined is determined using pattern matching.

JP-A-4-290066 JP-A-4-19589 JP 11-196261 A JP 2003-23530 A JP-A-1-39153

  However, in the above technique, the first image and the second image are synthesized by the same image processing regardless of what the first image is. As a result, it cannot be said that the image processing is sufficiently optimized, and there is a possibility that the accuracy of determining the combination position of the first image and the second image is lowered.

  According to the present invention, when generating composite image data representing a composite image obtained by combining the first image and the second image, the accuracy of determining the composite position of the first image and the second image is reduced. It aims at suppressing.

  SUMMARY An advantage of some aspects of the invention is to solve at least a part of the problems described above, and the invention can be implemented as the following application examples.

Application Example 1 An image processing apparatus, an acquisition unit that acquires first image data representing a first image and second image data representing a second image, and the first image A plurality of regions between a reference region determining unit that determines a reference region, which is a partial region, a reference region in the first image, and a plurality of candidate regions in the second image. A calculation unit that calculates the similarity of the corresponding region, a corresponding region determination unit that determines one candidate region of the plurality of candidate regions as a corresponding region corresponding to the reference region based on the plurality of similarities, Using the first image data and the second image data, a synthesized image obtained by synthesizing the first image and the second image so that the reference area and the corresponding area overlap each other is obtained. A generating unit that generates composite image data to be expressed, and the calculating unit includes the reference region Based on the variation of the values of the plurality of pixels, M (M is 1 or more) used for calculating the similarity among the N (N is an integer of 3 or more) pixels in the reference region. N integers less than N) and using M pixels in the reference region and M pixels in each candidate region corresponding to the M pixels in the reference region, An image processing apparatus that calculates the similarity.

  According to the above configuration, M to be used for calculating the degree of similarity is determined based on variations in values of a plurality of pixels in the reference region. Then, the similarity is calculated using the M pixels in the reference area and the M pixels in each candidate area. As a result, the similarity is calculated using an appropriate number of pixels, so that the corresponding region can be determined appropriately. Therefore, it is possible to suppress a decrease in accuracy in determining the synthesis position of the first image and the second image in the synthesized image.

Application Example 2 An image processing apparatus, an acquisition unit that acquires first image data representing a first image and second image data representing a second image, and the first image A reference area determining unit that determines a reference area that is a partial area of the first image; the reference area in the first image; and the second image in the second image corresponding to the reference area in the first image. A plurality of candidate regions, a calculation unit that calculates a plurality of similarities between the plurality of candidate regions, and one candidate region of the plurality of candidate regions as the reference region based on the plurality of similarities Using the corresponding area determining unit that determines the corresponding corresponding area, the first image data, and the second image data, the first image and the corresponding area are overlapped with each other. Generation for generating composite image data representing a composite image synthesized with the second image The reference area in the first image is different from the first partial reference area, and the second partial reference area is different from the first partial reference area. Each of the candidate regions in the second image corresponding to the first partial reference region and the second partial reference region in the first image, respectively. One partial candidate area and a second partial candidate area, wherein the calculation unit is configured to determine whether the first partial reference area and the second partial area are based on variations of a plurality of pixels in the reference area. A first portion between the first partial reference region and the first partial candidate region corresponding to the first partial reference region using a first weight value The second partial reference area and the second part are calculated using the second weight value by calculating the similarity. The reference region in the first image and the reference region in the first image are calculated by calculating a second partial similarity between the second partial candidate region corresponding to the reference region. An image processing device that calculates the degree of similarity between each of the candidate regions in the second image corresponding to.

  According to the above configuration, the first partial similarity between the first partial reference region and the first partial candidate region is calculated using the first weight value, and the second weight value is used. The second partial similarity between the second partial reference region and the second partial candidate region is calculated. Thereby, the similarity between the reference region and each candidate region corresponding to the reference region is calculated. As a result, the similarity is calculated by appropriately using the two partial similarities for the two partial reference regions having different variations, so that the corresponding region can be appropriately determined. Therefore, it is possible to improve the accuracy of determining the synthesis position of the first image and the second image in the synthesized image.

Application Example 3 An image processing apparatus, an acquisition unit that acquires first image data representing a first image and second image data representing a second image, and the first image A reference region determining unit that determines a reference region that is a partial region of the reference region; a reduction rate determining unit that determines a reduction rate based on variations in values of a plurality of pixels in the reference region; A reduced reference image obtained by reducing the image in the reference area in the image at the reduction ratio, and a plurality of reduced candidate images obtained by reducing the images in the plurality of candidate areas in the second image at the reduction ratio. And a calculation unit for calculating a plurality of similarities between the plurality of candidate regions, and determining one candidate region of the plurality of candidate regions as a corresponding region corresponding to the reference region A corresponding area determination unit, the first image data, and the second image data. And a generation unit that generates combined image data representing a combined image in which the first image and the second image are combined so that the reference region and the corresponding region overlap. apparatus.

  According to the above configuration, the similarity between the reduced reference image and the reduced candidate image reduced at the reduction rate determined based on the variation in the values of the plurality of pixels in the reference region is calculated. As a result, the similarity is calculated using an image of an appropriate size, so that the corresponding area can be determined appropriately. Therefore, it is possible to suppress a decrease in accuracy in determining the synthesis position of the first image and the second image in the synthesized image.

  The present invention can be realized in various forms. For example, a method for controlling an image processing apparatus, a computer program for realizing these apparatuses or methods, a recording medium on which the computer program is recorded, Or the like.

It is a block diagram which shows the structure of the image processing system in 1st Example. 5 is a flowchart showing the operation of the image processing system 1000. It is a figure which shows an example of the original document 10 used in a present Example. It is a figure which shows an example of UI screen. It is a figure which shows an example of a scanned image. It is a flowchart of a reference pixel determination process. It is explanatory drawing of a reference pixel determination process. It is a figure explaining a variation pixel and a non-variation pixel. It is a flowchart of a mask data generation process. It is explanatory drawing of a mask data generation process. It is a flowchart of a corresponding area determination process. It is a figure explaining the some candidate area | region which can be arrange | positioned in search area | region SA. 3 is a diagram illustrating an example of a composite image 30. FIG. It is a flowchart of the reference pixel determination process of 2nd Example. It is explanatory drawing of the reference pixel determination process of 2nd Example. It is a flowchart of the corresponding | compatible area | region determination process of 3rd Example. It is a flowchart of the reference pixel determination process of 4th Example. It is a flowchart of the corresponding | compatible area | region determination process of 4th Example.

A. First embodiment:
A-1: Configuration of Image Processing System 1000 FIG. 1 is a block diagram showing the configuration of the image processing system in the first embodiment. The image processing system 1000 includes a server 400 as an image processing apparatus and a multifunction device 200. The server 400 is connected to the Internet 70, and the multi-function device 200 is connected to the Internet 70 via a LAN (Local Area Network) 80. As a result, the server 400 and the multifunction device 200 can communicate with each other via the LAN 80 and the Internet 70. Further, the personal computer 500 of the user of the multifunction device 200 may be connected to the LAN 80.

  The server 400 includes a CPU 410, a volatile storage device 420 such as a DRAM, a nonvolatile storage device 430 such as a hard disk drive or a flash memory, and a communication unit 480 including an interface for connecting to a network such as the Internet 70. I have. The volatile storage device 420 is provided with a buffer area 421 for temporarily storing various intermediate data generated when the CPU 410 performs processing. The nonvolatile storage device 430 stores a computer program 431 and a UI data group 433.

  The computer program 431 and the UI data group 433 are installed in the server 400 by being uploaded to the server 400 via the Internet 70 by an administrator of the server 400, for example. Alternatively, the computer program 431 and the UI data group 433 may be provided in a form stored in, for example, a DVD-ROM, and may be installed in the server 400 by the administrator of the server 400. The CPU 410 implements image processing to be described later by executing the computer program 431.

  The MFP 200 includes a CPU 210, a volatile storage device 220 such as a DRAM, a nonvolatile storage device 230 such as a flash memory and a hard disk drive, a printer unit 240, a scanner unit 250, and an operation unit 260 such as a touch panel and buttons. A display unit 270 such as a liquid crystal display, and a communication unit 280 that communicates with an external device. For example, the communication unit 280 includes an interface for connecting to a network such as the LAN 80 and an interface for connecting to an external storage device (for example, a USB memory).

  The volatile storage device 220 is provided with a buffer area 221 for temporarily storing various data generated when the CPU 210 performs processing. A control program 231 is stored in the non-volatile storage device 230.

  The printer unit 240 executes printing using a printing method such as an inkjet method or a laser method. The scanner unit 250 generates scan data by optically reading a document using a photoelectric conversion element (for example, CCD, CMOS). The scanner unit 250 includes a so-called flat bed type document table. In this embodiment, the maximum size of a document that can be read at one time is larger than A4 and smaller than A3. Specifically, the scanner unit 250 can read a document having a length in the longitudinal direction that is the same as that of A4 and a length in the short direction that is slightly longer (for example, several centimeters) than A4. . Therefore, as will be described later, the A3 size document can be read in two steps so that the vicinity of the center in the longitudinal direction of the A3 size document overlaps.

  The CPU 210 executes control of the multifunction device 200 by executing the control program 231. For example, the CPU 210 controls the printer unit 240 and the scanner unit 250 to execute a copy process, a print process, a scan process, and the like. Furthermore, the CPU 210 can access the server 400 and execute a service use process that uses a service provided by the server 400.

A-2: Operation of Image Processing System 1000 FIG. 2 is a flowchart showing the operation of the image processing system 1000. The processing of this flowchart is started when the multifunction device 200 receives an instruction to use an image generation service provided by the server 400 from a user. As will be described in detail later, this image generation service is a service that combines a plurality of scan images represented by a plurality of scan data. The details of the plurality of scan data are described later. For example, the plurality of scan data are generated by reading a document having a size larger than the size that can be read at one time in a plurality of times.

  When the process is started, in S5, the CPU 210 of the multifunction device 200 transmits a service start request to the server 400. When receiving the service start request, the CPU 410 of the server 400 selects UI data necessary for providing the image generation service from the UI data group 433 (FIG. 1), and transmits the UI data to the multifunction device 200 (S10). ). Specifically, the UI data includes screen data representing a user interface screen (hereinafter referred to as UI screen) and control data. This control data includes, for example, various data necessary for the MFP 200 to perform predetermined processing (specifically, scan processing in S15 described later) using the UI screen. For example, the control data is used to perform processing (for example, transmission of scan data to the server 400) to be executed by the multifunction device 200 based on a user instruction received via a UI screen (for example, FIG. 4). Necessary information (for example, transmission address of scan data) is included.

  In S15, the CPU 210 executes a scan process for generating a plurality of scan data based on the received UI data. In the scan process, the CPU 210 generates two pieces of scan data by reading the document prepared by the user in two steps. The scan data according to the present embodiment is RGB image data including RGB component values (for example, values of 256 gradations from 0 to 255) for each pixel.

  FIG. 3 is a diagram illustrating an example of a document used in this embodiment. The size of the original 10 in FIG. 3 is about twice the size (A3 size in this embodiment) that is readable by the scanner unit 250 at one time (in this embodiment, a size slightly larger than the A4 size). .

  FIG. 4 is a diagram illustrating an example of a UI screen. First, the CPU 210 displays the UI screen UG1 of FIG. For example, the UI screen UG1 includes a message MS1 for prompting appropriate placement of the document 10 on the document table, a scan button SB, and a cancel button CB. The user places the document 10 on the document table and presses the scan button SB so that the left half 10L (FIG. 3) of the left side of the document 10 can be read according to the UI screen UG1. In response to the pressing of the scan button SB, the CPU 210 controls the scanner unit 250 to read the original, thereby generating left scan data.

  FIG. 5 is a diagram illustrating an example of a scanned image. FIG. 5A shows a left scan image 20L represented by the left scan data. The left-side scanned image 20L includes a left-side original image HIL that shows an approximately half area 10L (FIG. 3) on the left side of the original 10, and a margin WBL.

  Next, the CPU 210 displays a predetermined UI screen (not shown) on the display unit 270. Similar to the UI screen UG1, this UI screen includes a message for prompting appropriate placement of the document 10 on the document table, a scan button, and a cancel button. In accordance with the UI screen, the user places the document 10 on the document table and presses the scan button so that the half area 10R (FIG. 3) on the right side of the document 10 can be read. The CPU 210 generates right-side scan data by controlling the scanner unit 250 and reading a document in response to pressing of the scan button.

  FIG. 5B shows a right scan image 20R represented by the right scan data. The right-side scanned image 20R includes a right-side original image HIR that shows an approximately half area 10R (FIG. 3) on the right side of the original 10, and a margin WBR.

  Here, the image representing the center CA in the horizontal direction of the document 10 in FIG. 3 is included in both the region along the right side of the left scan image 20L and the region along the left side of the right scan image 20R. Yes. That is, as shown by hatching in FIG. 5, the left-side scan image 20L includes an image CIL that represents the center portion CA of the document 10, and the right-side scan image 20R includes an image CIR that represents the center portion CA of the document 10. . This is because, for example, the document 10 is placed on the document table so that the center CA of the document 10 can be read both when generating right-side scan data and when generating left-side scan data by using a UI screen or a manual. This is realized by instructing the user. Note that the image CIL in the left-side scan image 20L and the image CIR in the right-side scan image 20R both represent the partial CIA of the document 10, but there are some areas and colors depending on the user's operation. Differences can occur.

  In S20 of FIG. 2, the CPU 210 transmits right scan data representing the right scan image 20R and left scan data representing the left scan image 20L to the server 400. As a result, in S25, the CPU 410 of the server 400 acquires the right side scan data and the left side scan data, and stores them in the buffer area 421.

  In S30, the CPU 410 executes reference pixel determination processing using the right side scan data. FIG. 6 is a flowchart of the reference pixel determination process.

  In S105, the CPU 410 determines a reference area SP that is a partial area in the right-side scan image 20R. The reference area SP is determined as a rectangular area having a predetermined size arranged at a predetermined position in the right scan image 20R. Further, the reference area SP is arranged in an image CIR representing the center CA of the document 10 in the right-side scan image 20R. For example, the reference region SP is arranged along the side (in the present embodiment, the left side) along the image CIR representing the central portion CA among the four sides of the right scanned image 20R. More specifically, the reference region SP is arranged in the vicinity of the upper left corner of the right scan image 20R and at a position separated from the left end and the upper end by a predetermined margin (for example, a margin of 5 to 20 pixels). . The horizontal size of the reference region SP is, for example, 5 to 20 pixels, and the vertical size of the reference region SP is 1/4 to 1/2 of the number of pixels in the vertical direction of the right scan image 20R. . The reference area SP is arranged with a predetermined margin, because the edge of the right-side scan image 20R has noise compared to the inner part because the shadow of the edge of the document is reflected. This is because it is a large area, so that the noisy area is not included in the reference area SP.

  FIG. 7 is an explanatory diagram of the reference pixel determination process. FIG. 7A shows an image in the reference area SP determined in S105.

  In S110, the CPU 410 classifies the plurality of pixels in the reference region SP into two types of pixels, that is, a variation pixel and another pixel (also referred to as a non-variation pixel).

  FIG. 8 is a diagram for explaining a variation pixel and a non-variation pixel. The CPU 410 selects a plurality of pixels in the reference area SP one by one as the target pixel TP, and determines whether the target pixel TP is a variation pixel or a non-variation pixel. Specifically, as illustrated in FIG. 8, the CPU 410 performs the determination using pixels in the region FL corresponding to 3 horizontal pixels × 3 pixels centering on the target pixel TP. First, as shown in the equation of FIG. 8, the CPU 410 determines the difference ΔVn between the value of the target pixel TP (R0, B0, B0) and the values of the surrounding eight pixels (Rn, Gn, Bn). Is calculated. n is a number from 1 to 8 that identifies eight pixels around the pixel of interest, and is a number assigned to each pixel in the region FL of FIG. ΔVn is represented by the sum of the absolute values of the differences between the three types of component values, as shown in FIG. That is, ΔVn is represented by the sum of the absolute value of (Rn−R0), the absolute value of (Gn−G0), and the absolute value of (Bn−B0). The CPU 410 calculates the total value of the calculated eight differences ΔVn as the variation value V of the target pixel TP, as shown by an equation in FIG.

  When the variation value V of the target pixel TP is equal to or greater than the predetermined threshold Vth, the target pixel TP is determined to be a variation pixel. When the variation value V of the target pixel TP is less than the predetermined threshold Vth, it is determined that the target pixel TP is a non-variable pixel. Thereby, all the pixels in the reference region SP are classified into either a variation pixel or a non-variation pixel. As can be understood from the above description, it can be said that the variation pixel is a pixel whose value difference with a plurality of surrounding pixels is greater than or equal to the reference. It can be said that the non-uniform pixel is a pixel whose value difference with a plurality of surrounding pixels is less than the reference.

  FIG. 7B shows a binary image EP obtained by classifying a plurality of pixels in the reference region SP into a variation pixel and a non-variation pixel. The binary image EP is obtained, for example, by setting the value of a pixel determined as a variation pixel to “1” and the value of a pixel determined as a non-variation pixel to “0”. The single hatched first region SV in the binary image EP indicates a region constituted by a plurality of non-variable pixels, that is, a region in which variations in values of a plurality of internal pixels are relatively small. The cross-hatched second region BV in the binary image EP indicates a region constituted by a plurality of variation pixels, that is, a region in which variations in values of a plurality of internal pixels are relatively large. For example, the second area BV includes an area that includes many object edges in the reference area SP. In addition, the first area SV includes an area indicating a background and an area where the color change inside the object is poor.

  Accordingly, the ratio of the second area BV occupying the reference area SP, that is, the ratio of the number Nv of variation pixels to the total number Nt of pixels in the reference area SP (Nv / Nt) is determined for each right-side scan image 20R to be processed. Different. Specifically, (Nv / Nt) is relatively large when a scan image having a relatively large number of characteristic parts such as an edge of an object is a processing target at a position where the reference region SP is arranged. In addition, when a scan image having a relatively large amount of a region showing a background or a portion with little color change inside an object is a processing target at a position where the reference region SP is arranged, (Nv / Nt) is compared. Become smaller.

In S115, the CPU 410 calculates a ratio (Nv / Nt) of the number Nv of variation pixels to the total number Nt of pixels in the reference region SP. In S120, the CPU 410 calculates a thinning rate TR (unit:%). The thinning rate TR is calculated using the following equation (1).
TR = {0.2 + 0.7 × (Nv / Nt)} × 100 (1)

  In S125, the CPU 410 executes a mask data generation process for generating mask data based on the thinning rate TR. FIG. 9 is a flowchart of the mask data generation process. FIG. 10 is an explanatory diagram of the mask data generation process. The mask data is binary image data having a value indicating a reference pixel (“ON”) and a value indicating a non-reference pixel (“OFF”) for each pixel. That is, it can be said that the mask data generation process is a process of determining each of the plurality of pixels in the reference region SP as either a reference pixel or a non-reference pixel based on the thinning rate TR. .

  In S200 of FIG. 9, the CPU 410 prepares initial data of mask data in the buffer area 421. The initial data is binary image data representing an area having the same size (number of pixels in the vertical and horizontal directions) as the reference area SP, and the values of all the pixels are set to the initial value, that is, “OFF”. Yes. FIG. 10A shows an initial mask image MPa represented by initial data.

  In S205, the CPU 410 initializes the error value EV (sets it to zero). The error value EV is a parameter used for generating mask data.

  In S210, one pixel of interest is selected from a plurality of pixels PX in the initial mask image MPa. As shown by the arrows in FIG. 10A, for example, a plurality of pixels PX in the mask image MPa are selected one pixel at a time sequentially from the upper left end in the right direction, and one row in the horizontal direction. Are selected one by one in order from the upper left end in the right direction.

  In S215, the thinning-out rate TR is added to the current error value EV (EV ← EV + TR). In S220, CPU 410 determines whether or not error value EV after addition is 100 or more. When the error value EV is 100 or more (S220: YES), the CPU 410 determines the value of the target pixel as “OFF” in S225. That is, the CPU 410 maintains the value of the target pixel as the initial value “OFF”. In subsequent S230, the CPU 410 initializes (sets to zero) the error value EV.

  When the error value EV is less than 100 (S220: NO), the CPU 410 determines that the value of the target pixel is “ON” in S235. That is, the CPU 410 changes the value of the target pixel from the initial value “OFF” to “ON”.

  In S240, the CPU 410 determines whether or not all the pixels in the mask image MPa have been processed as the target pixel. When there is an unprocessed pixel (S240: NO), the CPU 410 returns to S210 and selects an unprocessed pixel as a target pixel. If all the pixels have been processed (S240: YES), the CPU 410 ends the mask data generation process. When the mask data generation process is completed, the mask data is generated.

  FIGS. 10B and 10C show an example of a mask image represented by the mask data generated by the mask data generation process. The hatched pixels in the mask images MPb and MPc in FIGS. 10B and 10C are “ON” pixels, and the non-hatched pixels are “OFF” pixels. The mask image MPb in FIG. 10B is a mask image when the thinning rate TR is 25%. In this mask image MPb, 25% of the pixels are “OFF” pixels and 75% of the pixels are “ON” pixels. The mask image MPc in FIG. 10C is a mask image when the thinning rate TR is 50%. In this mask image MPc, 50% of the pixels are “OFF” pixels and 50% of the pixels are “ON” pixels. FIG. 7C also illustrates the mask image MP generated by the mask data generation process of FIG. 9 corresponding to the reference region SP shown in FIG. The mask image MP in FIG. 9 is an example when the thinning rate TR is 50%.

  As can be understood from the above description, when the ratio of the reference pixels in the total number of pixels in the reference region SP is the reference rate RR, the thinning rate TR is 25%, and the reference rate RR is 75%. This means that the thinning rate TR is 50%, which means that the reference rate RR is 50%. That is, RR = (100−TR). It can be said that the reference rate RR is a value indicating the degree to which the values of a plurality of pixels in the reference region SP are referred to when calculating the similarity in the corresponding region determination process described later.

  As shown in the above equation (1), the thinning rate TR is a ratio (Nv / Nt) of the number Nv of the variation pixels to the total number Nt of the pixels in the reference region SP, that is, the variation pixels occupying the reference region SP. A value of 20% to 90% is determined according to the ratio (Nv / Nt). As (Nv / Nt) increases, the thinning rate TR increases. As (Nv / Nt) decreases, the thinning rate TR decreases.

  Therefore, the reference rate RR is determined to be a value of 10% to 80% according to the ratio (Nv / Nt) of the variation pixels occupying in the standard region SP. The larger the (Nv / Nt), the smaller the reference rate RR, and the smaller the (Nv / Nt), the larger the reference rate RR.

  When the mask data generation process ends, the reference pixel determination process in FIG. 6 ends. When the reference pixel determination process is completed, the CPU 410 executes the corresponding area determination process in S35 of FIG. The corresponding area determination process is a process for determining the corresponding area CP in the left scan image 20L corresponding to the reference area SP in the right scan image 20R. The corresponding area CP corresponding to the reference area SP can be defined as follows. A part of the document 10 shown in the reference area SP in the right scanned image 20R is defined as a specific portion SPT (FIG. 3). The corresponding area CP corresponding to the reference area SP is an area representing the specific portion SPT of the document 10 in the left scan image 20L.

  FIG. 11 is a flowchart of the corresponding area determination process. In S300, the CPU 410 determines a search area SA in the left scan image 20L. A search area SA is shown in the left-side scan image 20L of FIG. The search area SA is a predetermined area. The length in the vertical direction of the search area SA is equal to the total length in the vertical direction of the left scan image 20L. The length in the left-right direction of the search area SA is, for example, 20% to 50% of the length in the left-right direction of the left scan image 20L.

  In S305, the CPU 410 selects one target candidate area from among a plurality of candidate areas that can be arranged in the search area SA. FIG. 12 is a diagram illustrating a plurality of candidate areas that can be arranged in the search area SA.

  The size of the candidate area is the same as the size of the reference area SP (FIG. 5) in the right-side scan image 20R. The coordinates of the upper right point Pa of the candidate area NPa in which the upper side and the right side of the search area SA coincide with the upper side and the right side are (1, 1). Let (1, m) be the coordinates of the upper right point Pb of the candidate area NPb where the lower and right sides of the search area SA coincide with the lower and right sides. The coordinates of the upper right point Pc of the candidate area NPc where the upper side and the left side of the search area SA coincide with the upper side and the left side are (k, 1). The coordinates of the upper right point Pd of the candidate area NPd in which the lower side and the left side of the search area SA coincide with the lower side and the left side are defined as (k, m).

  The upper right point of the candidate area can be located at an arbitrary coordinate among (k × m) coordinates represented by (p, q). Here, p is an arbitrary integer from 1 to k, and q is an arbitrary integer from 1 to m. Therefore, (k × m) candidate areas can be set in the search area SA.

  As indicated by an arrow in FIG. 12, the candidate area NPa at the upper right end is selected as the first attention candidate area from these (k × m) candidate areas. Then, candidate areas whose positions are shifted downward by one pixel are sequentially selected. When the lower right candidate area NPb is selected, a candidate area shifted by one pixel to the left from the position of the upper right candidate area NPa is selected. After that, candidate areas whose positions are shifted downward by one pixel again are sequentially selected. The candidate region selected last is the candidate region NPd at the lower left end.

  When one attention candidate area is selected, in S310, the CPU 410 sets one pixel as the attention pixel among all the pixels in the reference area SP (FIG. 7A) in the right scan image 20R. select.

  In S320, the CPU 410 determines whether or not the target pixel is a reference pixel based on the value of a pixel (hereinafter also referred to as a corresponding mask pixel) in the mask image MP (FIG. 7C) corresponding to the target pixel. Judging. Specifically, when the value of the corresponding mask pixel is “ON”, it is determined that the target pixel is a reference pixel. When the value of the corresponding mask pixel is “OFF”, it is determined that the target pixel is not a reference pixel.

  When the target pixel is a reference pixel (S320: YES), in S325, the CPU 410 determines the value of the target pixel in the reference region SP, the value of the pixel in the target candidate region corresponding to the target pixel, The difference ΔVP is calculated. The values of the two pixels for which the difference ΔVP is to be calculated are (R1, G1, B1) and (R2, G2, B2). The difference ΔVP is represented by the sum of absolute values of differences between the three types of component values. That is, the difference ΔVP is represented by the sum of the absolute value of (R1−R2), the absolute value of (G1−G2), and the absolute value of (B1−B2). Instead, the Euclidean distance between (R1, G1, B1) and (R2, G2, B2) may be used for the difference ΔVP.

  In subsequent S330, CPU 410 determines whether or not calculated difference ΔVP is equal to or smaller than a predetermined reference value TH1. When the difference ΔVP is equal to or smaller than the predetermined reference value TH1 (S330: YES), in S335, the CPU 410 counts up the number of similar pixels SC. When the difference ΔVP is equal to or smaller than the predetermined reference value TH1, it is possible to determine that the target pixel in the reference region SP and the pixel in the target candidate region corresponding to the target pixel are similar pixels. is there.

  When the difference ΔVP is larger than the predetermined reference value TH1 (S330: NO), the CPU 410 skips S335 and advances the process to S340.

  When the target pixel is a non-reference pixel in S320 described above (S320: NO), in S325, the CPU 410 skips S325 to S335 and advances the process to S340.

  In S340, the CPU 410 determines whether or not all the pixels in the reference area SP have been processed as the target pixel. When there is an unprocessed pixel (S340: NO), the CPU 410 returns to S310 and selects an unprocessed pixel as a target pixel. If all the pixels have been processed (S340: YES), the CPU 410 advances the process to S342.

  In S342, the CPU 410 calculates the similarity (SC / Nrt) between the reference area SP and the candidate area of interest. The similarity (SC / Nrt) is a ratio of the number of similar pixels SC to the total number Nrt of reference pixels in the standard region SP. As the similarity (SC / Nrt) is larger, the reference region SP and the candidate region of interest are more similar.

  In S345, CPU 410 determines whether the similarity (SC / Nrt) is equal to or higher than threshold value TH2.

  When the similarity (SC / Nrt) is equal to or greater than the threshold value TH2 (S345: YES), in S350, the CPU 410 determines the current candidate region of interest as the corresponding region CP corresponding to the reference region SP, and The corresponding area determination process ends.

  If the similarity (SC / Nrt) is less than the threshold value TH2 (S345: NO), in S355, has the CPU 410 processed all candidate areas (FIG. 12) in the search area SA as attention candidate areas? Judge whether or not. If there is an unprocessed candidate area (S355: NO), the CPU 410 initializes the number of similar pixels SC in S360, and returns to S305 to select the unprocessed candidate area as the attention candidate area. . When all the candidate areas have been processed (S355: YES), the CPU 410 ends the corresponding area determination process without determining the corresponding area CP corresponding to the reference area SP.

  FIG. 5A shows the determined corresponding area CP in the search area SA of the left scan image 20L.

  When the corresponding area determination process is completed, the CPU 410 executes a composition process in S40 of FIG. In the synthesis process, using the right scan data and the left scan data, composite image data representing a composite image synthesized with the right original image HIR in the right scan image 20R and the left original image HIL in the left scan image 20L is obtained. Generated.

  FIG. 13 is a diagram illustrating an example of the composite image 30. As shown in FIG. 13, in the composite image 30, the right original image HIR and the left original image HIL are such that the reference area SP in the right original image HIR and the corresponding area CP in the left original image HIL overlap. Is synthesized. In the composite image 30, for example, the pixel value in the right original image HIR (right scan image 20R) is preferentially adopted as the pixel value in the region where the right original image HIR and the left original image HIL overlap each other. The As a result, the right original image HIR and the left original image HIL are combined to generate a combined image 30 representing the original 10 in FIG.

  If the corresponding region CP corresponding to the reference region SP cannot be determined in the corresponding region determination process in FIG. 11 (S355 in FIG. 11: YES), for example, mechanically, the right end of the left scanned image 20L Composite image data representing a composite image obtained by combining the two scan images is generated so that the side and the left end side of the right scan image 20R are in contact with each other (not shown).

  In S <b> 45 of FIG. 2, the CPU 410 transmits the generated composite image data to the multifunction device 200. When the composite image data is received, the CPU 210 of the multifunction device 200 stores the received composite image data in the nonvolatile storage device 230 and notifies the user that the composite image data has been received. The composite image data is used for the user. For example, the user can cause the MFP 200 to print the composite image 30 using these composite image data.

  According to the above embodiment, the value indicating the variation in the values of the plurality of pixels in the reference region SP in the right scan image 20R, that is, the number Nv of the variation pixels with respect to the total number Nt of pixels in the reference region SP described above. The ratio (Nv / Nt) is calculated (S115 in FIG. 6). Then, the thinning rate TR is calculated based on the value (Nv / Nt) indicating the variation (S120 in FIG. 6). As described above, the thinning rate TR has a relationship of RR = (100−TR) with the reference rate RR indicating the ratio of the reference pixels to the total number of pixels in the reference region SP. That is, determining the thinning rate TR in S120 of FIG. 6 is synonymous with determining the reference rate RR.

  Then, based on the thinning rate TR (in other words, based on the reference rate RR), the similarity (SC / Nrt) between the reference region SP and each of the plurality of candidate regions is calculated. More specifically, mask image data is generated based on the thinning rate TR (S125 in FIG. 6), and the similarity (SC / Nrt) is calculated with reference to the mask image data (S310 in FIG. 11). ~ S342). Then, based on the similarity (SC / Nrt), one candidate region among the plurality of candidate regions is determined as the corresponding region CP (S350 in FIG. 11). Therefore, in the synthesized image 30 (FIG. 13), it is possible to suppress a decrease in accuracy (hereinafter simply referred to as synthesis accuracy) for determining a position where the right original image HIR and the left original image HIL are synthesized.

  In other words, the CPU 410 determines N pixels (N is an integer of 3 or more) in the reference area SP (specifically, the reference area SP based on the variation in the values of the plurality of pixels in the reference area SP. M pixels (specifically, reference images) used for calculating the similarity are determined (specifically, reference images) (S125 in FIG. 6). Then, the CPU 410 uses the M pixels in the reference area SP and the M pixels in each candidate area corresponding to the M pixels in the reference area SP, and uses the similarity (SC / Nrt). Is calculated (S310 to S342 in FIG. 11). As a result, since the similarity (SC / Nrt) is calculated using an appropriate number of pixels, the corresponding region CP can be appropriately determined. Therefore, it is possible to suppress a decrease in accuracy in determining the composite position of the right scan image 20R and the left scan image 20L in the composite image 30.

  Specifically, in the corresponding area determination process in FIG. 11, if all the pixels in the reference area SP are set as reference pixels, the calculation load may be excessively increased. However, when the reference rate RR is simply lowered, the calculation load can be reduced, but the synthesis accuracy may be lowered. On the other hand, when the variation of a plurality of pixels in the reference region SP is relatively large, it is considered that a relatively large number of characteristic elements such as edges are included in the reference region SP. In this case, there is a high possibility that sufficient synthesis accuracy can be achieved even if the reference rate RR is lowered. On the other hand, when the variation of the plurality of pixels in the reference area SP is relatively small, the reference area SP has relatively no features such as an area indicating the background and an area in which the color change inside the object is scarce. It is considered that a relatively large number of images are included. In this case, it is difficult to ensure the synthesis accuracy. Therefore, if the reference rate RR is lowered, sufficient synthesis accuracy may not be achieved. For this reason, in this case, it is preferable to increase the reference rate RR. In this embodiment, the CPU 410 determines a higher reference rate RR as the value (Nv / Nt) indicating a plurality of variations in the reference area SP is smaller. In other words, the value (Nv / Nt) is higher. The smaller the value is, the lower the thinning-out rate TR is determined (S120 in FIG. 6). Then, the CPU 410 calculates the similarity (SC / Nrt) using the values of a large number of pixels in the reference area SP as the reference rate RR is higher (FIG. 11). In other words, the smaller the variation in the values of the plurality of pixels in the reference area SP, the larger the number M of pixels in the reference area SP used for calculating the similarity (SC / Nrt). As a result, it is possible to improve the processing speed while suppressing a decrease in synthesis accuracy.

  Furthermore, in the above embodiment, the CPU 410 generates mask data that defines reference pixels and non-reference pixels based on the reference rate RR, in other words, based on the thinning rate TR (FIG. 9, FIG. 9). 10). That is, based on the reference rate RR, a plurality of reference pixels are determined from a plurality of pixels in the standard region SP. Then, the similarity (SC / Nrt) is calculated using the determined values of the plurality of reference pixels (FIG. 11). The number of reference pixels to be determined increases as the reference rate RR increases, in other words, as the decimation rate TR decreases (FIG. 10). Therefore, the similarity (SC / Nrt) can be appropriately calculated using the values of an appropriate number of reference pixels among a plurality of pixels in the standard region SP. As a result, it is possible to improve the processing speed while suppressing a decrease in synthesis accuracy.

  The similarity is calculated based on the difference ΔVP between the value of the reference pixel in the standard area SP and the value of the pixel in the candidate area corresponding to the reference pixel (S325 in FIG. 11). Therefore, if the number of pixels used for calculating the similarity among a plurality of pixels in the reference region SP increases, the number of pixels used for calculating the similarity among the plurality of pixels in the candidate region. Will also increase. That is, as the reference rate RR is higher, the CPU 410 increases the number of pixels in the reference area SP used when calculating the similarity and the number of pixels in the candidate area used when calculating the similarity. To do. Therefore, since the similarity is calculated using an appropriate number of pixels according to the reference rate RR, it is possible to appropriately suppress a decrease in synthesis accuracy.

B. Second Embodiment The reference pixel determination process of the second embodiment includes a process different from the reference pixel determination process of the first embodiment. The other points of the second embodiment are the same as those of the first embodiment. FIG. 14 is a flowchart of reference pixel determination processing according to the second embodiment. FIG. 15 is an explanatory diagram of reference pixel determination processing according to the second embodiment.

  In the reference pixel determination process of the second embodiment shown in FIG. 14, the processes of S105 and S110 are executed in the same manner as the reference pixel determination process of the first embodiment shown in FIG. As a result, as in the first embodiment, the plurality of pixels in the reference region SP are classified into a variation pixel and a non-variation pixel. FIG. 15B shows a binary image EP obtained by classifying into a variation pixel and a non-variation pixel. With the binary image EP shown in FIG. 15B, the reference region SP includes a first region SV composed of a plurality of non-variable pixels and a second region BV composed of a plurality of variation pixels. , Separated. In other words, the reference region SP includes a first region SV in which the variation in the values of the plurality of internal pixels is relatively small, and a second region BV in which the variation in the values of the plurality of internal pixels is relatively large. , Separated.

  In the reference pixel determination process of the second embodiment of FIG. 14, the process of S120B is executed instead of S115 to S125 of FIG. In S120B, the CPU 410 generates mask data in which a plurality of pixels in the first region SV composed of a plurality of non-variable pixels in the reference region SP are thinned out. Specifically, the CPU 410 sets the reference rate RR1 of the first region SV to 50%, and sets the reference rate RR2 in the second region BV to 100%. Then, the CPU 410 generates mask data according to the reference rates RR1 and RR2. In the first area SV in the mask image MP represented by the generated mask data, “ON” pixels indicating reference pixels and “OFF” pixels indicating non-reference pixels are 50% each. In the second region BV, only “ON” pixels indicating reference pixels are arranged (FIG. 15C).

  According to the second embodiment described above, the reference rate of the first region SV of the standard region SP and the reference rate of the second region are determined at different levels. That is, different reference rates are determined for the first region SV and the second region BV, which have different variations.

  In other words, the CPU 410 identifies the first area SV and the second area BV based on the variation of the plurality of pixels in the reference area SP (S110 in FIG. 14, FIG. 15 ( B)). Then, as a part of the M pixels used for the calculation of the similarity (SC / Nrt), M1 (M1) of N1 (N1 is an integer of 2 or more) pixels in the first area SV. Is an integer of 1 or more and less than N1, and M2 pixels (M2 is an integer of 1 or more and less than N2) out of N2 pixels (N2 is an integer of 2 or more) in the second region BV. ) Is determined (S120B in FIG. 14). The ratio (M1 / N1) of M1 pixels to N1 pixels in the first region SV, that is, the reference rate RR1 (specifically, 50%) of the first region SV, and the second The ratio (M2 / N2) of M2 pixels to N2 pixels in the region BV of the second region BV, that is, the reference rate RR2 (specifically, 100%) of the second region BV is different from each other. As a result, the synthesis accuracy of the right original image HIR and the left original image HIL in the composite image represented by the composite image data can be improved.

  More specifically, the reference rate RR2 (that is, the ratio (M2 / N2)) of the second region BV having a relatively large variation is the reference rate RR1 (that is, the rate) of the first region SV having a relatively small variation. Larger than (M1 / N1)). As a result, the synthesis accuracy in the synthesized image can be improved.

  When the corresponding region CP corresponding to the reference region SP is determined based on the value of the pixel in the region where the pixel variation is relatively small, the reference region SP is determined based on the value of the pixel in the region where the variation is relatively large. Compared with the case where the corresponding region CP corresponding to is determined, an erroneous region is easily determined as the corresponding region CP. In other words, it is more accurate to search for similar regions with reference to regions with many edges than to search for similar regions based on regions with few edges. You can explore. According to the second embodiment, since the reference rate of the second area BV in which the pixel variation is larger than the first area SV is higher than that of the first area, the corresponding area CP corresponding to the reference area SP is more accurately determined. Can be determined. Therefore, the synthesis accuracy in the synthesized image can be improved.

In the second embodiment, the pixel value of the first ratio (specifically, 50%) of the plurality of pixels in the first region SV and the plurality of pixels in the second region BV. The degree of similarity (SC / Nrt) is calculated using the second ratio (specifically, 100%) of pixel values. And
The second ratio is set higher than the first ratio. Thereby, the reference rate of the first region SV becomes higher than the reference rate of the second region BV. As a result, the synthesis accuracy in the synthesized image 30 can be improved.

C. Third Example In the third example, the process of S120B of the reference pixel determination process of the second example of FIG. 14 is omitted. Therefore, the reference pixel determination process of the third embodiment is finished when a plurality of pixels in the reference region SP are classified into a variation pixel and a non-variation pixel, as shown in FIG. The

  The corresponding area determining process of the third embodiment includes a process different from the corresponding area determining process (FIG. 11) of the first and second embodiments. The other points of the third embodiment are the same as those of the second embodiment. FIG. 16 is a flowchart of the corresponding area determining process according to the third embodiment. In the flowchart of FIG. 16, steps different from the corresponding area determination process of FIG. 11 are denoted by “C” at the end of the reference numerals, and the same steps as the corresponding area determination process of FIG. A reference is attached.

  In the corresponding area determination process of the third embodiment of FIG. 16, two types of parameters, the first similar pixel number SC1 and the second similar pixel number SC2, are used. In the corresponding area determination process of the third example of FIG. 16, in S320C, the CPU 410 refers to the binary image EP of FIG. 15B to determine whether or not the target pixel is a variation pixel.

  If the target pixel is a variation pixel (S320C: YES), that is, if the target pixel belongs to the second region BV composed of the variation pixel, the CPU 410 performs S325 and S330 in FIG. Similarly, the difference ΔVP between the value of the target pixel in the reference region SP and the value of the pixel in the target candidate region corresponding to the target pixel is calculated (S325), and the difference ΔVP is the predetermined reference value TH1. It is determined whether or not the following is true (S330). If the difference ΔVP is equal to or smaller than the predetermined reference value TH1 (S330: YES), in S335C, the CPU 410 counts up the second number of similar pixels SC2. On the other hand, when the difference ΔVP is larger than the predetermined reference value TH1 (S330: NO), the CPU 410 skips S335C and advances the process to S340.

  When the target pixel is a non-variable pixel (S320C: NO), that is, when the target pixel belongs to the first region SV composed of non-variable pixels, the CPU 410 performs S325 and S330 in FIG. Similarly, the difference ΔVP between the value of the target pixel in the reference region SP and the value of the pixel in the target candidate region corresponding to the target pixel is calculated (S336C), and the difference ΔVP is a predetermined reference value TH1. It is determined whether or not the following is true (S337C). When the difference ΔVP is equal to or smaller than the predetermined reference value TH1 (S337C: YES), in S338C, the CPU 410 counts up the first similar pixel number SC1. On the other hand, when the difference ΔVP is larger than the predetermined reference value TH1 (S337C: NO), the CPU 410 skips S338C and advances the process to S340.

  That is, in the corresponding region determination process of the third embodiment, when the target pixel is a pixel belonging to the first region SV in FIG. 15B (that is, a non-variable pixel), the target pixel and the target pixel are determined. The first similar pixel number SC1 is counted up on condition that the corresponding pixel in the candidate region of interest is similar. When the target pixel is a pixel belonging to the second region BV in FIG. 15B (that is, a variation pixel), the target pixel and the pixel in the target candidate region corresponding to the target pixel are similar. The second similar pixel number SC2 is counted up.

  In S340, the CPU 410 determines whether or not all the pixels in the reference area SP have been processed as the target pixel. When there is an unprocessed pixel (S340: NO), the CPU 410 returns to S310 and selects an unprocessed pixel as a target pixel. When all the pixels have been processed (S340: YES), the CPU 410 advances the process to S342C.

In S342C, the CPU 410 calculates the total similarity SCtotal using the first similar pixel number SC1 and the second similar pixel number SC2. The similarity SCtotal is calculated using the following equation (2).
SCtotal = WT1 * SC1 + WT2 * SC2 (2)

  Here, WT1 is a weight for the pixels in the first region SV, that is, non-variable pixels. WT2 is a weight for the pixels in the second region BV, that is, the variation pixels. The weighting WT2 for the pixels in the second area BV is larger than the weighting WT1 for the pixels in the first area SV. In this embodiment, WT1 = 0.5 and WT2 = 1.0. As the total similarity SCtotal is larger, the reference region SP and the candidate region of interest are more similar.

  The first term (WT1 × SC1) of the above formula (2) includes the first region SV in the reference region SP and the partial region (first partial candidate region) in the candidate region of interest corresponding to the first region SV. And a similarity between them (also referred to as a first partial similarity). The second term (WT2 × SC2) of the above formula (2) includes the second region BV in the reference region SP and a partial region (second partial candidate region) in the target candidate region corresponding to the second region BV. And the similarity between them (also referred to as the second partial similarity).

  In S345C, CPU 410 determines whether or not total similarity SCtotal is equal to or greater than threshold value TH3.

  If the total similarity SCtotal is equal to or greater than the threshold value TH3 (S345C: YES), in S350, the CPU 410 determines the current candidate region of interest as the corresponding region CP corresponding to the reference region SP, and the corresponding region. The decision process is terminated.

  When the total similarity SCtotal is less than the threshold value TH3 (S345: NO), in S355, the CPU 410 has processed all candidate areas (FIG. 12) in the search area SA as attention candidate areas. Judging. When there is an unprocessed candidate area (S355: NO), the CPU 410 initializes two similar pixel numbers SC1 and SC2 in S360C, and returns to S305 to focus on the unprocessed candidate area. Select as a candidate area. When all the candidate areas have been processed (S355: YES), the CPU 410 ends the corresponding area determination process without determining the corresponding area CP corresponding to the reference area SP.

  According to the third embodiment described above, the CPU 410 calculates the first partial similarity (WT1 × SC1) using the first weight value WT1, and uses the second weight value, By calculating the second partial similarity (WT2 × SC2), the similarity SCtotal between the reference region SP and each candidate region in the left scan image 20L corresponding to the reference region SP is calculated. As a result, since the similarity is calculated by appropriately using the two partial similarities for the two areas SV and BV having different variations, the corresponding area CP can be appropriately determined. Accordingly, it is possible to improve the accuracy of determining the composite position of the right scan image 20R and the left scan image 20L in the composite image 30.

  Then, when calculating the similarity SCtotal, the CPU 410 makes the weighting for the plurality of pixels WT2 in the second region BV larger than the weighting WT1 for the plurality of pixels in the first region SV. Thereby, when calculating the similarity SCtotal, the degree of referring to the value of the pixel in the second area BV is made higher than the degree of referring to the value of the pixel in the first area SV. As a result, the effect of the value of the pixel in the second region BV (that is, the value of the variation pixel) on the similarity SCtotal is made relatively large, so that the composition accuracy in the composite image is improved as in the second embodiment. it can.

D. Fourth embodiment:
The reference pixel determining process of the fourth embodiment includes a process different from the reference pixel determining process (FIG. 6) of the first embodiment. The corresponding area determining process of the fourth embodiment includes a process different from the corresponding area determining process (FIG. 11) of the first embodiment. The other points of the fourth embodiment are the same as those of the first embodiment.

  FIG. 17 is a flowchart of reference pixel determination processing according to the fourth embodiment. In the reference pixel determination process of the fourth embodiment, the processes of S120D and S125D are performed instead of S120 and S125 of FIG. In S120D, CPU 410 determines reduction ratio SR. The same value as the reference rate RR of the first embodiment is used for the reduction rate SR (unit:%). That is, the reduction ratio SR is determined based on the variation in the values of the plurality of pixels in the reference region SP. Specifically, like the reference rate RR in the first embodiment, the reduction rate SR increases as the variation in the values of the plurality of pixels in the reference region SP decreases.

  In S125D, the CPU 410 generates a reduced reference image obtained by reducing the image in the reference region SP of the right-side scan image 20R with the reduction rate SR determined in S120. Specifically, the CPU 410 reduces the number of pixels of the image in the reference area SP as appropriate, for example, by reducing the number of pixels corresponding to the reduction rate SR using the nearest neighbor method. A reference image is generated. For example, the number of pixels of the reduced reference image is the number obtained by multiplying the number of pixels of the image in the reference region SP before reduction by the reduction ratio SR. Therefore, the number of pixels of the reduced reference image increases as the reduction ratio SR increases. That is, the number of pixels of the reduced reference image increases as the variation in the values of the plurality of pixels in the reference region SP decreases.

  FIG. 18 is a flowchart of the corresponding area determining process according to the fourth embodiment. The processes in S400 and S405 in FIG. 18 are the same as the processes in S300 and S305 in FIG. In S407, the CPU 410 generates a reduced candidate image obtained by reducing the image in the attention candidate area with the reduction rate SR determined in S120. The generation method of the reduction candidate image is the same as the generation method of the reduction reference image (for example, the nearest neighbor method). The size (number of pixels in the vertical and horizontal directions) of the reduction candidate image is the same as the size of the reduction reference image generated in S125D of FIG. That is, each pixel in the reduced reference image generated in S125D in FIG. 6 has a one-to-one correspondence with each pixel in the reduced candidate image generated in S407.

  In S410, the CPU 410 selects one pixel among all the pixels of the reduced reference image as a target pixel.

  In S425, the CPU 410 calculates a difference ΔVP between the value of the target pixel in the reduced reference image and the value of the pixel in the reduced candidate image corresponding to the target pixel. The calculation method of the difference ΔVP is the same as the calculation of the difference ΔVP in S325 of FIG. In subsequent S430, CPU 410 determines whether or not calculated difference ΔVP is equal to or smaller than a predetermined reference value TH1. When the difference ΔVP is equal to or smaller than the predetermined reference value TH1 (S430: YES), in S435, the CPU 410 counts up the number of similar pixels SCm. When the difference ΔVP is larger than the predetermined reference value TH1 (S430: NO), the CPU 410 skips S435 and advances the process to S440.

  In S440, the CPU 410 determines whether or not all the pixels in the reduced reference image have been processed as the target pixel. When there is an unprocessed pixel (S440: NO), the CPU 410 returns to S410 and selects an unprocessed pixel as a target pixel. When all the pixels have been processed (S440: YES), the CPU 410 advances the process to S442.

  In S442, the CPU 410 calculates the similarity (SCm / Nmt) between the reduced reference image and the reduced candidate image of the candidate region of interest. The similarity (SCm / Nmt) is the ratio of the number of similar pixels SCm to the number of pixels Nmt in the reduced reference image. As the similarity (SCm / Nmt) is larger, the reduced reference image and the reduced candidate image in the candidate region of interest are more similar. Further, as the similarity (SCm / Nmt) is larger, the reference area SP and the attention candidate area are more similar.

  In S445, CPU 410 determines whether the similarity (SCm / Nmt) is equal to or higher than threshold value THm. When the similarity (SCm / Nmt) is equal to or greater than the threshold THm (S445: YES), in S450, the CPU 410 determines the current attention candidate region as the corresponding region CP corresponding to the reference region SP, and The corresponding area determination process ends.

  If the similarity (SCm / Nmt) is less than the threshold value THm (S445: NO), in S455, has the CPU 410 processed all candidate areas (FIG. 12) in the search area SA as attention candidate areas? Judge whether or not. If there is an unprocessed candidate area (S455: NO), the CPU 410 initializes the number of similar pixels SCm in S460, and returns to S405 to select the unprocessed candidate area as the attention candidate area. . When all candidate areas have been processed (S455: YES), the CPU 410 ends the corresponding area determination process without determining the corresponding area CP corresponding to the reference area SP.

  According to the fourth embodiment described above, the degree of similarity between the reduced reference image and the reduced candidate image reduced at the reduction rate SR determined based on the variation in the values of the plurality of pixels in the reference region SP ( SCm / Nmt) is calculated. As a result, since the similarity (SCm / Nmt) is calculated using a reduced reference image having an appropriate size (that is, the number of pixels), the corresponding region can be appropriately determined as in the first embodiment. it can. Therefore, it is possible to suppress a decrease in accuracy in determining the composite position of the right scan image 20R and the left scan image 20L in the composite image 30.

  Specifically, the CPU 410 determines the reduction rate SR so that the smaller the variation in the values of the plurality of pixels in the reference area SP of the right-side scan image 20R, the larger the number of pixels in the reduced reference image. As a result, as in the first embodiment, the processing speed can be improved while suppressing a decrease in the synthesis accuracy.

  Note that the generation method of the reduced candidate image and the reduced reference image is not limited to the nearest neighbor method. For example, in the generation method of the reduction candidate image or the reduction reference image, a simple average value or a weighted average value of a plurality of pixels in the image before reduction is used as a value of one pixel in the image after reduction. A method to be employed, for example, a bilinear method may be used. However, the nearest neighbor method is more preferable because it can suppress that characteristic pixel values in the image before reduction are averaged and lost.

E. Modified Example (1) In each of the above-described embodiments, composite image data representing a composite image obtained by combining two scan images is generated using two scan data. The present invention is not limited to this, and the composite image data may be generated using an arbitrary number of scan data. For example, composite image data representing a composite image obtained by combining four scan images may be generated using the four scan image data.

(2) In each of the above embodiments, the ratio of the number Nv of the variation pixels to the total number Nt of the pixels in the reference region SP is used as a value indicating the variation in the values of the plurality of pixels in the reference region SP of the right scan image 20R ( Nv / Nt) is used (S115 in FIG. 6). Instead of the value of (Nv / Nt), for example, the value indicating the variation in the values of the plurality of pixels in the reference region SP, for example, the variance σ ² of the values of the plurality of pixels in the reference region SP, the standard deviation σ and edge amount may be used.

(3) In the corresponding area determination processing in each of the above embodiments, the algorithm for determining the corresponding area CP corresponding to the reference area SP is an example, and other algorithms may be adopted. For example, in the corresponding region determination process of the first embodiment or the second embodiment, the CPU 410 calculates the value of each reference pixel in the reference region SP and the value of the pixel in the attention candidate region corresponding to the reference pixel. A value VPtotal obtained by summing the difference ΔVP for all reference pixels in the reference region SP may be calculated. And the candidate area | region where the said total value VPtotal is the minimum among the several candidate area | regions which can be arrange | positioned in search area | region SA may be determined as corresponding | compatible area | region CP.

(4) Formula (1) for calculating the thinning rate TR in the first embodiment, that is, TR = {0.2 + 0.7 × (Nv / Nt)} × 100 is an example. The thinning rate TR can be calculated by an arbitrary formula that is defined so as to increase as the variation in the reference region SP increases. For example, the values of the parameters “0.2” and “0.7” in Equation (1) may be different values, specifically, a combination of “0.3” and “0.6” may be used. A combination of “0.1” and “0.8” may be used.

(5) In the above embodiments, the two pieces of image data used for generating the composite image data are two pieces of scan data generated when the original is read by the scanner unit 250 of the multifunction device 200. Instead of this, two pieces of image data may be generated by photographing a plurality of areas of a document with a digital camera.

(6) The processing (for example, the processing of S25 to S40 in FIG. 2) executed by the CPU 410 of the server 400 in each of the above embodiments may be executed by the CPU 210 of the multifunction device 200, for example. In this case, the server 400 is unnecessary, and the multi-function device 200 may execute the processing of FIG. 3 alone. Further, the processing executed by the CPU 410 of the server 400 may be executed by the CPU (not shown) of the personal computer 500 (FIG. 1) connected to the multifunction device 200. For example, the CPU of the personal computer 500 may execute these processes by executing a scanner driver program installed in the personal computer 500. Further, the server 400 may be configured with one computer as in the present embodiment, or may be configured with a computer system including a plurality of computers.

(7) In each of the above embodiments, a part of the configuration realized by hardware may be replaced with software, and conversely, a part of the configuration realized by software may be replaced with hardware. May be.

  200 ... multifunction device, 210 ... CPU, 220 ... volatile storage device, 221 ... buffer area, 230 ... nonvolatile storage device, 231 ... control program, 240 ... printer , 250 ... scanner unit, 260 ... operation unit, 270 ... display unit, 280 ... communication unit, 400 ... server, 410 ... CPU, 420 ... volatile storage device 421 ... buffer area, 430 ... nonvolatile storage device, 431 ... computer program, 433 ... UI data group, 480 ... communication unit, 500 ... personal computer, 1000 ... Image processing system

Claims (11)

An image processing apparatus,
An acquisition unit that acquires first image data representing a first image and second image data representing a second image;
A reference area determination unit that determines a reference area that is a partial area in the first image;
A calculation unit that calculates a plurality of similarities between the reference region in the first image and a plurality of candidate regions in the second image;
A corresponding region determining unit that determines one candidate region of the plurality of candidate regions as a corresponding region corresponding to the reference region based on the plurality of similarities;
Using the first image data and the second image data, a synthesized image obtained by synthesizing the first image and the second image so that the reference area and the corresponding area overlap each other is obtained. A generating unit that generates composite image data to be represented;
With
The calculation unit includes:
Based on variations in the values of a plurality of pixels in the reference region, M pixels (N is an integer of 3 or more) used in the calculation of the similarity among the N pixels (N is an integer of 3 or more) in the reference region. M is an integer of 1 or more and less than N),
An image processing apparatus that calculates the degree of similarity using M pixels in the reference area and M pixels in each candidate area corresponding to the M pixels in the reference area. The image processing apparatus according to claim 1,
The calculation unit increases the number M of pixels in the reference region used for calculating the similarity as the variation in values of the plurality of pixels in the reference region of the first image is smaller. Processing equipment. The image processing apparatus according to claim 1,
The reference region in the first image, a first partial reference area, and a different second portion reference area and a plurality of pixels of variation of the first value of the partial reference region contained within The calculation unit includes:
Identifying the first partial reference region and the second partial reference region based on variations of a plurality of pixels in the reference region;
As a part of the M pixels used for calculating the similarity, M1 (M1 is 1 or more) out of N1 (N1 is an integer of 2 or more) pixels in the first partial reference region. N1 or less pixels),
As a part of the M pixels used for the calculation of the similarity, M2 (M2 is 1 or more) out of N2 (N2 is an integer of 2 or more) pixels in the second partial reference region. N2 or less integer) pixels,
The ratio (M1 / N1) of the M1 pixels to the N1 pixels in the first partial reference region, and the M2 pixels relative to the N2 pixels in the second partial reference region. An image processing apparatus different from the ratio (M2 / N2). The image processing apparatus according to claim 3,
The second partial reference region has a larger variation in the values of a plurality of pixels contained therein than the first partial reference region.
The image processing apparatus, wherein the ratio (M2 / N2) is greater than the ratio (M1 / N1). An image processing apparatus,
An acquisition unit that acquires first image data representing a first image and second image data representing a second image;
A reference area determination unit that determines a reference area that is a partial area in the first image;
Calculating a plurality of similarities between the reference region in the first image and a plurality of candidate regions in the second image corresponding to the reference region in the first image; A calculation unit;
A corresponding region determining unit that determines one candidate region of the plurality of candidate regions as a corresponding region corresponding to the reference region based on the plurality of similarities;
Using the first image data and the second image data, a synthesized image obtained by synthesizing the first image and the second image so that the reference area and the corresponding area overlap each other is obtained. A generating unit that generates composite image data to be represented;
With
The reference area in the first image includes a first partial reference area, a second partial reference area in which variations in values of a plurality of pixels included in the first image are different from the first partial reference area, Including,
Each of the candidate regions in the second image includes a first partial candidate region and a second partial region corresponding respectively to the first partial reference region and the second partial reference region in the first image. Partial candidate areas,
The calculation unit includes:
Identifying the first partial reference region and the second partial reference region based on variations of a plurality of pixels in the reference region;
Using a first weight value to calculate a first partial similarity between the first partial reference region and the first partial candidate region corresponding to the first partial reference region; and By calculating a second partial similarity between the second partial reference region and the second partial candidate region corresponding to the second partial reference region using a second weight value, Image processing for calculating the similarity between the reference region in the first image and each candidate region in the second image corresponding to the reference region in the first image apparatus. The image processing apparatus according to claim 5,
The second partial reference region has a larger variation in the values of a plurality of pixels contained therein than the first partial reference region.
The image processing apparatus, wherein the second weight value is larger than the first weight value. An image processing apparatus,
An acquisition unit that acquires first image data representing a first image and second image data representing a second image;
A reference area determination unit that determines a reference area that is a partial area in the first image;
A reduction rate determination unit that determines a reduction rate based on variations in values of a plurality of pixels in the reference region;
A reduced reference image obtained by reducing an image in the reference area in the first image at the reduction ratio, and a plurality of images obtained by reducing images in a plurality of candidate areas in the second image at the reduction ratio. A calculation unit that calculates a plurality of similarities between the reduced candidate images;
A corresponding region determining unit that determines one candidate region of the plurality of candidate regions as a corresponding region corresponding to the reference region based on the plurality of similarities;
Using the first image data and the second image data, a synthesized image obtained by synthesizing the first image and the second image so that the reference area and the corresponding area overlap each other is obtained. A generating unit that generates composite image data to be represented;
An image processing apparatus comprising: The image processing apparatus according to claim 7,
The reduction ratio determining unit determines the reduction ratio such that the smaller the variation in the values of the plurality of pixels in the reference area in the first image, the larger the number of pixels in the reduced reference image. An image processing apparatus. A computer program,
An acquisition function for acquiring first image data representing a first image and second image data representing a second image;
A reference area determination function for determining a reference area which is a partial area in the first image;
A calculation function for calculating a plurality of similarities between the reference region in the first image and a plurality of candidate regions in the second image;
A corresponding area determination function for determining one candidate area of the plurality of candidate areas as a corresponding area corresponding to the reference area based on the plurality of similarities;
Using the first image data and the second image data, a synthesized image obtained by synthesizing the first image and the second image so that the reference area and the corresponding area overlap each other is obtained. A generating function for generating composite image data to be represented;
Is realized on a computer,
The calculation function is
Based on variations in the values of a plurality of pixels in the reference region, M pixels (N is an integer of 3 or more) used in the calculation of the similarity among the N pixels (N is an integer of 3 or more) in the reference region. M is an integer of 1 or more and less than N),
A computer program that calculates the similarity using M pixels in the reference area and M pixels in each candidate area corresponding to the M pixels in the reference area. A computer program,
An acquisition function for acquiring first image data representing a first image and second image data representing a second image;
A reference area determination function for determining a reference area which is a partial area in the first image;
Calculating a plurality of similarities between the reference region in the first image and a plurality of candidate regions in the second image corresponding to the reference region in the first image; A calculation function;
A corresponding area determination function for determining one candidate area of the plurality of candidate areas as a corresponding area corresponding to the reference area based on the plurality of similarities;
Using the first image data and the second image data, a synthesized image obtained by synthesizing the first image and the second image so that the reference area and the corresponding area overlap each other is obtained. A generating function for generating composite image data to be represented;
Is realized on a computer,
The reference area in the first image includes a first partial reference area, a second partial reference area in which variations in values of a plurality of pixels included in the first image are different from the first partial reference area, Including,
Each of the candidate regions in the second image includes a first partial candidate region and a second partial region corresponding respectively to the first partial reference region and the second partial reference region in the first image. Partial candidate areas,
The calculation function is
Identifying the first partial reference region and the second partial reference region based on variations of a plurality of pixels in the reference region;
Using a first weight value to calculate a first partial similarity between the first partial reference region and the first partial candidate region corresponding to the first partial reference region; and By calculating a second partial similarity between the second partial reference region and the second partial candidate region corresponding to the second partial reference region using a second weight value, A computer program for calculating the similarity between the reference region in the first image and each candidate region in the second image corresponding to the reference region in the first image . A computer program,
An acquisition function for acquiring first image data representing a first image and second image data representing a second image;
A reference area determination function for determining a reference area which is a partial area in the first image;
A reduction rate determination function for determining a reduction rate based on variations in values of a plurality of pixels in the reference region;
A reduced reference image obtained by reducing an image in the reference area in the first image at the reduction ratio, and a plurality of images obtained by reducing images in a plurality of candidate areas in the second image at the reduction ratio. A calculation function for calculating a plurality of similarities between the reduced candidate images;
A corresponding area determination function for determining one candidate area of the plurality of candidate areas as a corresponding area corresponding to the reference area based on the plurality of similarities;
Using the first image data and the second image data, a synthesized image obtained by synthesizing the first image and the second image so that the reference area and the corresponding area overlap each other is obtained. A generating function for generating composite image data to be represented;
A computer program that causes a computer to realize

Images