JP6956269B2 - Image processing device and image processing method - Google Patents

Description

The disclosed technique relates to an image processing apparatus and an image processing method.

A scanner may be used to read a book page by page and convert it into electronic data.

Japanese Unexamined Patent Publication No. 09-186854 Japanese Unexamined Patent Publication No. 2000-011192

If the positions of the scanned series of multi-page images are misaligned between the pages, when the series of multi-page images are viewed in order, the images are viewed due to the misalignment of the images that occur between the pages. It makes people feel uncomfortable. In order to reduce such discomfort, the operator has conventionally required to carefully place the book on the platen of the scanner each time the page is read so that the image position does not shift between pages. rice field. Therefore, the work efficiency of the operator was poor.

The disclosed technique has been made in view of the above, and aims to improve the work efficiency of the operator.

In the disclosed aspect, the image processing apparatus has a storage unit and an image processing unit. The storage unit stores a series of a plurality of scanned images, each including a medium image. In each of the plurality of scanned images, the image processing unit sets the first reference position in the first region where the medium image exists, and sets the second reference position in the second region other than the first region. , The arrangement of the medium image in the scanned image is changed based on the positional relationship between the first reference position and the second reference position, and the plurality of images are changed based on the medium image after the arrangement change. The same extraction region is set in the scanned image of the above, and the medium image is extracted from each of the plurality of scanned images according to the extraction region.

According to the aspect of disclosure, the work efficiency of the operator can be improved.

FIG. 1 is a diagram showing a configuration example of the image processing apparatus of the first embodiment. FIG. 2 is a diagram showing an example of a processing flow of the image processing apparatus of the first embodiment. FIG. 3 is a diagram showing an example of a processing flow of the image processing apparatus of the first embodiment. FIG. 4A is a diagram provided for explaining an operation example of the image processing apparatus of the first embodiment. FIG. 4B is a diagram provided for explaining an operation example of the image processing apparatus of the first embodiment. FIG. 4C is a diagram provided for explaining an operation example of the image processing apparatus of the first embodiment. FIG. 4D is a diagram provided for explaining an operation example of the image processing apparatus of the first embodiment. FIG. 4E is a diagram provided for explaining an operation example of the image processing apparatus of the first embodiment. FIG. 5 is a diagram showing an example of a processing flow of the image processing apparatus of the first embodiment. FIG. 6A is a diagram provided for explaining an operation example of the image processing apparatus of the first embodiment. FIG. 6B is a diagram provided for explaining an operation example of the image processing apparatus of the first embodiment. FIG. 7 is a diagram showing an example of a processing flow of the image processing apparatus of the first embodiment. FIG. 8A is a diagram provided for explaining an operation example of the image processing apparatus of the first embodiment. FIG. 8B is a diagram provided for explaining an operation example of the image processing apparatus of the first embodiment. FIG. 8C is a diagram provided for explaining an operation example of the image processing apparatus of the first embodiment. FIG. 9 is a diagram showing an example of a processing flow at the time of the first reading of the first embodiment. FIG. 10A is a diagram provided for explaining an operation example of the image processing apparatus of the first embodiment. FIG. 10B is a diagram provided for explaining an operation example of the image processing apparatus of the first embodiment. FIG. 10C is a diagram provided for explaining an operation example of the image processing apparatus of the first embodiment. FIG. 11A is a diagram provided for explaining an operation example of the image processing apparatus of the first embodiment. FIG. 11B is a diagram provided for explaining an operation example of the image processing apparatus of the first embodiment. FIG. 11C is a diagram provided for explaining an operation example of the image processing apparatus of the first embodiment. FIG. 12A is a diagram provided for explaining an operation example of the image processing apparatus of the first embodiment. FIG. 12B is a diagram provided for explaining an operation example of the image processing apparatus of the first embodiment. FIG. 12C is a diagram provided for explaining an operation example of the image processing apparatus of the first embodiment. FIG. 13A is a diagram provided for explaining an operation example of the image processing apparatus of the first embodiment. FIG. 13B is a diagram provided for explaining an operation example of the image processing apparatus of the first embodiment. FIG. 13C is a diagram provided for explaining an operation example of the image processing apparatus of the first embodiment. FIG. 14A is a diagram provided for explaining an operation example of the image processing apparatus of the first embodiment. FIG. 14B is a diagram provided for explaining an operation example of the image processing apparatus of the first embodiment. FIG. 14C is a diagram provided for explaining an operation example of the image processing apparatus of the first embodiment. FIG. 14D is a diagram provided for explaining an operation example of the image processing apparatus of the first embodiment. FIG. 15A is a diagram provided for explaining an operation example of the image processing apparatus of the first embodiment. FIG. 15B is a diagram provided for explaining an operation example of the image processing apparatus of the first embodiment. FIG. 15C is a diagram provided for explaining an operation example of the image processing apparatus of the first embodiment. FIG. 15D is a diagram provided for explaining an operation example of the image processing apparatus of the first embodiment. FIG. 16A is a diagram provided for explaining an operation example of the image processing apparatus of the first embodiment. FIG. 16B is a diagram provided for explaining an operation example of the image processing apparatus of the first embodiment. FIG. 16C is a diagram provided for explaining an operation example of the image processing apparatus of the first embodiment. FIG. 17 is a diagram provided for explaining an operation example of the image processing apparatus of the first embodiment. FIG. 18A is a diagram provided for explaining an operation example of the image processing apparatus of the first embodiment. FIG. 18B is a diagram provided for explaining an operation example of the image processing apparatus of the first embodiment. FIG. 18C is a diagram provided for explaining an operation example of the image processing apparatus of the first embodiment. FIG. 19A is a diagram provided for explaining an operation example of the image processing apparatus of the first embodiment. FIG. 19B is a diagram provided for explaining an operation example of the image processing apparatus of the first embodiment. FIG. 19C is a diagram provided for explaining an operation example of the image processing apparatus of the first embodiment. FIG. 20 is a diagram provided for explaining an operation example of the image processing apparatus of the first embodiment. FIG. 21 is a diagram showing an example of an old book of Example 2. FIG. 22 is a diagram showing an example of a processing flow of the image processing apparatus of the second embodiment. FIG. 23 is a diagram showing an example of the processing flow of the image processing apparatus of the second embodiment. FIG. 24 is a diagram showing an example of a processing flow of the image processing apparatus of the second embodiment. FIG. 25 is a diagram showing an example of a processing flow of the image processing apparatus of the second embodiment. FIG. 26A is a diagram provided for explaining an operation example of the image processing apparatus of the second embodiment. FIG. 26B is a diagram provided for explaining an operation example of the image processing apparatus of the second embodiment. FIG. 26C is a diagram provided for explaining an operation example of the image processing apparatus of the second embodiment. FIG. 27A is a diagram provided for explaining an operation example of the image processing apparatus of the second embodiment. FIG. 27B is a diagram provided for explaining an operation example of the image processing apparatus of the second embodiment. FIG. 27C is a diagram provided for explaining an operation example of the image processing apparatus of the second embodiment. FIG. 28A is a diagram provided for explaining an operation example of the image processing apparatus of the second embodiment. FIG. 28B is a diagram provided for explaining an operation example of the image processing apparatus of the second embodiment. FIG. 28C is a diagram provided for explaining an operation example of the image processing apparatus of the second embodiment. FIG. 29A is a diagram provided for explaining an operation example of the image processing apparatus of the second embodiment. FIG. 29B is a diagram provided for explaining an operation example of the image processing apparatus of the second embodiment. FIG. 29C is a diagram provided for explaining an operation example of the image processing apparatus of the second embodiment. FIG. 30A is a diagram provided for explaining an operation example of the image processing apparatus of the second embodiment. FIG. 30B is a diagram provided for explaining an operation example of the image processing apparatus of the second embodiment. FIG. 30C is a diagram provided for explaining an operation example of the image processing apparatus of the second embodiment. FIG. 30D is a diagram provided for explaining an operation example of the image processing apparatus of the second embodiment. FIG. 31A is a diagram provided for explaining an operation example of the image processing apparatus of the second embodiment. FIG. 31B is a diagram provided for explaining an operation example of the image processing apparatus of the second embodiment. FIG. 31C is a diagram provided for explaining an operation example of the image processing apparatus of the second embodiment. FIG. 31D is a diagram provided for explaining an operation example of the image processing apparatus of the second embodiment. FIG. 32 is a diagram provided for explaining an operation example of the image processing apparatus of the second embodiment. FIG. 33A is a diagram provided for explaining an operation example of the image processing apparatus of the second embodiment. FIG. 33B is a diagram provided for explaining an operation example of the image processing apparatus of the second embodiment. FIG. 33C is a diagram provided for explaining an operation example of the image processing apparatus of the second embodiment. FIG. 34 is a diagram provided for explaining an operation example of the image processing apparatus of the second embodiment. FIG. 35A is a diagram provided for explaining an operation example of the image processing apparatus of the second embodiment. FIG. 35B is a diagram provided for explaining an operation example of the image processing apparatus of the second embodiment. FIG. 35C is a diagram provided for explaining an operation example of the image processing apparatus of the second embodiment. FIG. 36A is a diagram provided for explaining an operation example of the image processing apparatus of the second embodiment. FIG. 36B is a diagram provided for explaining an operation example of the image processing apparatus of the second embodiment. FIG. 36C is a diagram provided for explaining an operation example of the image processing apparatus of the second embodiment. FIG. 37 is a diagram showing an example of a processing flow of the image processing apparatus of the second embodiment. FIG. 38A is a diagram provided for explaining an operation example of the image processing apparatus of the second embodiment. FIG. 38B is a diagram provided for explaining an operation example of the image processing apparatus of the second embodiment. FIG. 38C is a diagram provided for explaining an operation example of the image processing apparatus of the second embodiment. FIG. 39 is a diagram showing an example of a processing flow of the image processing apparatus of the third embodiment. FIG. 40A is a diagram provided for explaining an operation example of the image processing apparatus of the third embodiment. FIG. 40B is a diagram provided for explaining an operation example of the image processing apparatus of the third embodiment. FIG. 40C is a diagram provided for explaining an operation example of the image processing apparatus of the third embodiment. FIG. 41A is a diagram provided for explaining an operation example of the image processing apparatus of the third embodiment. FIG. 41B is a diagram provided for explaining an operation example of the image processing apparatus of the third embodiment. FIG. 41C is a diagram provided for explaining an operation example of the image processing apparatus of the third embodiment. FIG. 42A is a diagram provided for explaining an operation example of the image processing apparatus of the third embodiment. FIG. 42B is a diagram provided for explaining an operation example of the image processing apparatus of the third embodiment. FIG. 42C is a diagram provided for explaining an operation example of the image processing apparatus of the third embodiment. FIG. 43A is a diagram provided for explaining an operation example of the image processing apparatus of the third embodiment. FIG. 43B is a diagram provided for explaining an operation example of the image processing apparatus of the third embodiment. FIG. 43C is a diagram provided for explaining an operation example of the image processing apparatus of the third embodiment. FIG. 44A is a diagram provided for explaining an operation example of the image processing apparatus of the third embodiment. FIG. 44B is a diagram provided for explaining an operation example of the image processing apparatus of the third embodiment. FIG. 44C is a diagram provided for explaining an operation example of the image processing apparatus of the third embodiment. FIG. 44D is a diagram provided for explaining an operation example of the image processing apparatus of the third embodiment. FIG. 45A is a diagram provided for explaining an operation example of the image processing apparatus of the third embodiment. FIG. 45B is a diagram provided for explaining an operation example of the image processing apparatus of the third embodiment. FIG. 45C is a diagram provided for explaining an operation example of the image processing apparatus of the third embodiment. FIG. 45D is a diagram provided for explaining an operation example of the image processing apparatus of the third embodiment. FIG. 46 is a diagram provided for explaining an operation example of the image processing apparatus of the third embodiment. FIG. 47A is a diagram provided for explaining an operation example of the image processing apparatus of the third embodiment. FIG. 47B is a diagram provided for explaining an operation example of the image processing apparatus of the third embodiment. FIG. 47C is a diagram provided for explaining an operation example of the image processing apparatus of the third embodiment. FIG. 48 is a diagram provided for explaining an operation example of the image processing apparatus of the third embodiment. FIG. 49A is a diagram provided for explaining an operation example of the image processing apparatus of the third embodiment. FIG. 49B is a diagram provided for explaining an operation example of the image processing apparatus of the third embodiment. FIG. 49C is a diagram provided for explaining an operation example of the image processing apparatus of the third embodiment. FIG. 50A is a diagram provided for explaining an operation example of the image processing apparatus of the third embodiment. FIG. 50B is a diagram provided for explaining an operation example of the image processing apparatus of the third embodiment. FIG. 50C is a diagram provided for explaining an operation example of the image processing apparatus of the third embodiment.

Hereinafter, examples of the image processing apparatus and the image processing method disclosed in the present application will be described with reference to the drawings. Note that this embodiment does not limit the image processing apparatus and image processing method disclosed in the present application. Further, in the embodiment, the same reference numerals are given to the configurations having the same functions and the steps for performing the same processing.

[Example 1]
<Configuration of image processing device>
FIG. 1 is a diagram showing a configuration example of the image processing apparatus of the first embodiment. In FIG. 1, the image processing device 1 includes a control unit 11, a storage unit 13, an image processing unit 15, and a display unit 17. The storage unit 13, the image processing unit 15, and the display unit 17 operate under the control of the control unit 11. The image processing device 10 is used, for example, mounted on a scanner or connected to a scanner.

The storage unit 13 stores an image read by the scanner (hereinafter, may be referred to as a “read image”). For example, when each page of one book is read by a scanner and one book is converted into electronic data, the storage unit 13 stores a series of a plurality of read images in which pages are connected over one book. do. Further, the scanned image includes an image of the medium to be scanned (hereinafter, may be referred to as a “medium image”), and the medium image is arranged in the scanned image. For example, when each page of a book is read by a scanner and one book is converted into electronic data, the medium image is an image of each page. That is, the storage unit 13 stores a series of a plurality of scanned images, each of which includes a medium image (in which a medium image is arranged).

<Processing and operation of image processing equipment>
2, FIG. 3, FIG. 5, and FIG. 7 are diagrams showing an example of the processing flow of the image processing apparatus of the first embodiment. 4A-E, 6A, B, 8A-C, 9, 10A-C, 11A-C, 12A-C, 13A-C, 14A-D, 15A-D, 16A. ~ C, FIGS. 17, 18A to 18C, 19A to 19C, and FIG. 20 are diagrams for explaining an operation example of the image processing apparatus of the first embodiment.

In the first embodiment, a case where each page of one old book is read by a scanner and one book is converted into electronic data will be described as an example. Further, in the first embodiment, a case where each page of an old book placed on a platen in a spread state is read by an overhead scanner will be described as an example.

The processing flow shown in FIG. 2 is started, for example, when a processing start button (not shown) included in the image reading device 10 is pressed by an operator.

In FIG. 2, in step S101, the control unit 11 selects a scanned image to be processed by the image processing unit 15 from a series of scanned images stored in the storage unit 13, and stores the stored image. The scanned image acquired from 13 is output to the image processing unit 15. However, the blank image has been removed in advance and is not stored in the storage unit 13.

For example, as a read image of one old book, the image of the front cover, the image of the spread of the first page and the second page (hereinafter, may be referred to as "spread 12 image"), the third page and the fourth page. When a series of images of spread images (hereinafter sometimes referred to as "spread 34 images"), ..., Back cover images are stored in the storage unit 13, the control unit 11 may perform steps S101 to S109. The front cover image in step S101 in the first processing loop, the spread 12 images in step S101 in the second processing loop of steps S101 to S109, and the spread 34 in step S101 in the third processing loop of steps S101 to S109. Images are sequentially acquired from the storage unit 13. Further, the control unit 11 acquires an image of the back cover from the storage unit 13 in step S101 in the final processing loop of steps S101 to S109. Further, for example, a series of images of the front cover image, the spread 12 image, the spread 34 image, ..., The back cover image are stored in the storage unit 13 as one file, and the control unit 11 is from the file to be processed. Each image of the front cover image, the spread 12 image, the spread 34 image, ..., The back cover image is selected in order. The file to be processed is arbitrarily selected by the operator, for example.

In step S103, the image processing unit 15 performs medium image position detection processing on the scanned image selected in step S101.

FIG. 3 shows an example of the processing flow of the medium image position detection process. Further, FIG. 4A shows an example of the scanned image RI selected in step S101. In FIG. 4A, the scanned image RI is, for example, the medium images MIA and MIB and the image of the platen on which the medium to be scanned is placed (hereinafter, may be referred to as “document stand image”) MTI. And include. For example, the medium image MIA is a double-page spread image of an old book, and the medium image MIB is an image of a color chart placed next to the old book.

In step S151 of FIG. 3, the image processing unit 15 performs edge detection ED using the near gradation difference with respect to the scanned image RI (FIG. 4A) (FIG. 4B).

Next, in step S153, the image processing unit 15 performs linear detection SLD using the huff detection and the least squares method on the read image RI after the edge detection ED (FIG. 4C).

Next, in step S155, the image processing unit 15 performs curve detection CLD using labeling on the read image RI after the edge detection ED (FIG. 4D).

Then, in step S157, the image processing unit 15 performs the circumscribed rectangle detection RTD on the read image RI based on the results of the straight line detection SLD and the curve detection CLD (FIG. 4E). By this circumscribing rectangle detection RTD, the circumscribing rectangle BR1 of the medium image MIA is detected as the existing position of the medium image MIA in the scanned image RI, and the circumscribing rectangle BR2 of the medium image MIB is used as the existing position of the medium image MIB in the scanned image RI. Detected.

Returning to FIG. 2, in step S105, the image processing unit 15 calculates the inclination and the center of gravity of each medium image based on each circumscribed rectangle detected in step S157. For example, the image processing unit 15 calculates the inclination of the circumscribing rectangle as the inclination of the medium image, and calculates the center of gravity of the extrinsic rectangle as the center of gravity of the medium image.

Next, in step S107, the image processing unit 15 performs the medium image reference position setting process.

FIG. 5 shows an example of the processing flow of the medium image reference position setting process. Since each process of steps S151, S153, and S155 in FIG. 5 is the same as that shown in FIG. 3, description thereof will be omitted.

In step S161 of FIG. 5, the image processing unit 15 uses the results of the straight line detection SLD and the curve detection CLD to obtain a figure showing the outline of the medium image (hereinafter, may be referred to as a “medium contour figure”) as the medium image. Detect every time.

Next, in step S163, the image processing unit 15 determines whether or not the upper side of the medium contour figure is a straight line for each medium image. If the upper side of the medium contour figure is a straight line (step S163: Yes), the process proceeds to step S165, and if the upper side of the medium contour figure is not a straight line (step S163: No), the process proceeds to step S167.

In step S165, the image processing unit 15 sets the midpoint of the upper side of the medium contour figure (that is, the midpoint of the straight line) as the medium image reference position, and stores the scanned image after setting the medium image reference position in the storage unit 13. Let me.

On the other hand, in step S167, the image processing unit 15 sets a straight line (hereinafter, may be referred to as an “approximate straight line”) connecting the corner points at both ends of the upper side of the medium contour figure in the medium image.

Then, in step S169, the image processing unit 15 sets the midpoint of the approximate straight line at the medium image reference position, and stores the scanned image after setting the medium image reference position in the storage unit 13.

6A and 6B show an example of setting the medium image reference position. FIG. 6A shows an example in which the midpoint of the approximate straight line set in the medium image is set as the medium image reference position. In FIG. 6A, the scanned image RI includes a medium image MI and a platen image MTI. By the process of step S157, the circumscribed rectangle BR with respect to the medium image MI is detected. Further, by the process of step S161, the medium contour figure DL is detected with respect to the medium image MI. In FIG. 6A, since the upper side of the medium contour figure DL is not a straight line (step S163: No), the corner points CR1 and CR2 at both ends of the upper side of the medium contour figure DL are connected to the medium image MI by the processing of step S167. An approximate straight line SL is set. Then, by the process of step S169, the midpoint of the approximate straight line SL is set to the medium image reference position RRP.

When the upper side of the medium contour figure DL is not a straight line, the image processing unit 15 sets the midpoint of the approximate straight line SL to the medium image reference position RRP, as shown in FIG. 6B, instead of setting the midpoint of the medium contour figure DL. The midpoint of the upper side may be set to the medium image reference position RRP.

Returning to FIG. 2, in step S109, the control unit 11 determines whether or not the processing of steps S101 to S107 has been completed for all of the series of scanned images stored in the storage unit 13. When the processing of steps S101 to S107 is completed for all of the series of scanned images stored in the storage unit 13 (step S109: Yes), the processing proceeds to step S111. On the other hand, in a series of a series of scanned images stored in the storage unit 13, if the scanned images that have not been processed in steps S101 to S107 remain (step S109: No), the processing proceeds to step S101. return. For example, when a series of images of the front cover image, the spread 12 image, the spread 34 image, ..., The back cover image are stored in the storage unit 13 as the read image of one old book, the back cover image. When the processing of steps S101 to S107 is completed, the processing proceeds to step S111.

In step S111, the image processing unit 15 sets the scanned image reference position for each scanned image. 6A and 6B show an example of setting the scanned image reference position. As shown in FIGS. 6A and 6B, the image processing unit 15 sets, for example, the midpoint of the upper side of the platen image MTI included in the scanned image RI at the scanned image reference position MRP1.

Next, in step S113, as in the process of step S101, the control unit 11 stores a series of a plurality of scanned images stored in the storage unit 13 (that is, a series of read images after setting the medium image reference position and the scanned image reference position). A scanned image to be processed by the image processing unit 15 is selected from the plurality of scanned images), acquired from the storage unit 13, and the acquired scanned image is output to the image processing unit 15.

Next, in step S115, the image processing unit 15 performs upright correction on the medium image included in the scanned image selected in step S113. That is, in step S115, the image processing unit 15 corrects the inclination of the medium image calculated in step S105.

Next, in step S117, the image processing unit 15 determines whether or not the scanned image selected in step S113 is the first scanned image in a series of a plurality of scanned images. If the scanned image selected in step S113 is the first scanned image (step S117: Yes), the process proceeds to step S119. On the other hand, if the scanned image selected in step S113 is not the first scanned image (step S117: No), the process proceeds to step S121.

In step S121, the image processing unit 15 performs paired image detection processing. FIG. 7 shows an example of the processing flow of the image detection process.

In FIG. 7, in step S171, the image processing unit 15 may refer to a medium image (hereinafter, referred to as “current medium image”) included in the Nth selected scanned image (that is, the scanned image selected this time). ) (Hereinafter referred to as "the center of gravity of the medium image") and the medium image included in the N-1th selected scanned image (that is, the previously selected scanned image) (hereinafter, "" The center of gravity (hereinafter sometimes referred to as "previous medium image center of gravity") of the previous medium image) is calculated. For example, the image processing unit 15 calculates the center of gravity of the circumscribing rectangle detected in step S157 as the center of gravity of the medium image this time and the center of gravity of the previous medium image.

Next, in step S173, the image processing unit 15 determines the distance between the scanned image reference position and the current medium image center of gravity (hereinafter, may be referred to as “this time distance”), and the scanned image reference position and the previous medium image center of gravity. Calculate the distance between and (hereinafter sometimes referred to as the "previous distance").

Next, in step S175, the image processing unit 15 calculates the vertical length H and the horizontal length W of the medium image this time and the vertical length H'and the horizontal length W'of the previous medium image. For example, the image processing unit 15 calculates the vertical length and the horizontal length of the circumscribed rectangle of the medium image this time as the vertical length H and the horizontal length W of the medium image this time, and calculates the vertical length and the horizontal length of the circumscribed rectangle of the previous medium image as the previous time. It is calculated as the vertical length H'and the horizontal length W'of the medium image.

Next, in step S177, the image processing unit 15 determines whether or not the absolute value of the difference between the current distance and the previous distance (hereinafter, may be referred to as “distance difference”) is less than the threshold value TH1. .. If the distance difference is less than the threshold TH1 (step S177: Yes), the process proceeds to step S179, and if the distance difference is greater than or equal to the threshold TH1 (step S177: No), the process proceeds to step S185.

In step S179, the image processing unit 15 sets a threshold value of the absolute value of the difference between the vertical length H of the current medium image and the vertical length H'of the previous medium image (hereinafter, may be referred to as "vertical length difference"). It is determined whether or not it is less than TH2. If the vertical length difference is less than the threshold value TH2 (step S179: Yes), the process proceeds to step S181, and if the vertical length difference is greater than or equal to the threshold value TH2 (step S179: No), the process proceeds to step S191.

In step S181, the image processing unit 15 has a threshold value of the absolute value of the difference between the horizontal length W of the medium image this time and the horizontal length W'of the previous medium image (hereinafter, may be referred to as "horizontal length difference"). It is determined whether or not it is less than TH3. If the lateral length difference is less than the threshold TH3 (step S181: Yes), the process proceeds to step S183, and if the lateral length difference is greater than or equal to the threshold TH3 (step S181: No), the process proceeds to step S187.

In step S183, the image processing unit 15 indicates that the current medium image and the previous medium image are both spread images or both are color chart images, and the current medium image and the previous medium image are paired with each other. Is determined. That is, in step S183, the image processing unit 15 determines that the current medium image and the previous medium image are a pair of images of the spread images or a pair of images of the color chart images. The image processing unit 15 determines whether the current medium image and the previous medium image are paired images of spread images or paired images of color chart images, for example, based on the horizontal length of the current medium image. To judge. For example, when the horizontal length of the medium image this time is the threshold TH4 or more, the image processing unit 15 determines that the medium image this time and the previous medium image are paired images of the spread images, and determines that the horizontal length of the medium image this time is a pair image. When is less than the threshold value TH4, it is determined that the current medium image and the previous medium image are paired images of the color chart images.

On the other hand, in step S185, the image processing unit 15 determines whether or not the vertical length difference is less than the threshold value TH2. If the vertical length difference is less than the threshold value TH2 (step S185: Yes), the process proceeds to step S187, and if the vertical length difference is greater than or equal to the threshold value TH2 (step S185: No), the process proceeds to step S191.

In step S187, the image processing unit 15 determines whether or not the horizontal length W of the medium image this time is within a predetermined range of the horizontal length W'of the previous medium image. In the image processing unit 15, for example, the horizontal length W of the medium image this time is more than half (that is, W'/ 2) or twice (that is, 2W') of the horizontal length W'of the previous medium image. It is determined whether or not it is within a predetermined range of less than. If the horizontal length W of the medium image this time is within this predetermined range (step S187: Yes), the process proceeds to step S189, and if the horizontal length W of the medium image this time is not within this predetermined range (step S187: No). , The process proceeds to step S191.

In step S189, the image processing unit 15 determines that one of the current medium image and the previous medium image is a cover image and the other is a spread image, and the current medium image and the previous medium image are paired with each other. judge. That is, in step S183, the image processing unit 15 determines that the current medium image and the previous medium image are paired images of the cover image and the spread image.

On the other hand, in step S191, the image processing unit 15 determines that the current medium image and the previous medium image are not paired with each other. That is, in step S191, the image processing unit 15 determines that the current medium image and the previous medium image are not paired images.

8A, B, and C show an example of determining the image pair.

In FIG. 8A, for example, the medium image MIA included in the scanned image RIA is the previous medium image, and the medium image MIB included in the scanned image RIB is the current medium image. In FIG. 8A, the distance difference between the previous distance DTA between the scanned image reference position MRP1 and the previous medium image center of gravity GBA and the current distance DTB between the scanned image reference position MRP1 and the current media image center of gravity GBB is less than the threshold TH1. (Step S177: Yes). Further, the vertical length difference between the vertical length H5 of the medium image MIA and the vertical length H6 of the medium image MIB is less than the threshold value TH2 (step S179: Yes). Further, the lateral length difference between the lateral length W5 of the medium image MIA and the lateral length W6 of the medium image MIB is less than the threshold TH3 (step S181: Yes). Therefore, it is determined that the medium image MIA and the medium image MIB are paired images of the spread images (step S183).

Further, in FIG. 8B, for example, the medium image MIC included in the scanned image RIC is the previous medium image, and the medium image MID included in the scanned image RID is the medium image this time. In FIG. 8B, the distance difference between the previous distance DTC between the scanned image reference position MRP1 and the previous media image center of gravity GBC and the current distance DTD between the scanned image reference position MRP1 and the current media image center of gravity GBD is less than the threshold TH1. (Step S177: Yes). Further, the vertical length difference between the vertical length H3 of the medium image MIC and the vertical length H4 of the medium image MID is less than the threshold value TH2 (step S179: Yes). However, the horizontal length difference between the horizontal length W3 of the medium image MIC and the horizontal length W4 of the medium image MID is the threshold value TH3 or more (step S181: No). However, the horizontal length W4 of the medium image MID is in the range of more than half and less than twice the horizontal length W3 of the medium image MIC (step S187: Yes). Therefore, it is determined that the medium image MIC and the medium image MID are paired images of the cover image and the spread image (step S189).

Further, in FIG. 8C, for example, the medium image MIE included in the scanned image RIE is the previous medium image, and the medium image MIF included in the scanned image RIF is the medium image this time. In FIG. 8C, the distance difference between the previous distance DTE between the scanned image reference position MRP1 and the previous media image center of gravity GBE and the current distance DTF between the scanned image reference position MRP1 and the current media image center of gravity GBF is the threshold TH1 or more. (Step S177: No). However, the vertical difference between the vertical length H1 of the medium image MIE and the vertical length H2 of the medium image MIF is less than the threshold value TH2 (step S185: Yes). Further, the horizontal length W2 of the medium image MIF is in the range of more than half and less than twice the horizontal length W1 of the medium image MIE (step S187: Yes). Therefore, it is determined that the medium image MIE and the medium image MIF are paired images of the cover image and the spread image (step S189).

The image processing unit 15 performs the processing of steps S171 to S191 (FIG. 7) for the scanned image selected this time and all the medium images included in the scanned image selected last time.

Returning to FIG. 2, in step S123, the image processing unit 15 determines whether or not a pair of images exists in the processing of steps S171 to S191 (FIG. 7). When the processing of step S183 or step S189 is performed in FIG. 7, it is determined that the paired image exists, and when the processing of step S191 is performed in FIG. 7, it is determined that the paired image does not exist. .. If the paired image exists (step S123: Yes), the process proceeds to step S125, and if the paired image does not exist (step S123: No), the process proceeds to step S131.

In step S125, the image processing unit 15 determines whether or not the two paired images are spread images or color chart images. When the process of step S183 is performed in FIG. 7, it is determined that the two paired images are spread images or color chart images, and when the process of step S189 is performed in FIG. 7, the pair is paired. It is determined that the two images are not the spread images and the color chart images. If the two paired images are spread images or color chart images (step S125: Yes), the process proceeds to step S127, and if the two paired images are not spread images or color chart images. (Step S125: No), the process proceeds to step S129.

The image processing unit 15 performs the position setting process A in step S127, while the position setting process B is performed in step S129. Details of the position setting processes A and B will be described later. After the processing of steps S127 and S129, the processing proceeds to step S135.

On the other hand, in step S131, the image processing unit 15 outputs a warning indicating that the pair of images does not exist. For example, the image processing unit 15 displays a warning message indicating that there is no medium image paired with the medium image in the N-1st selected scanned image in the Nth selected scanned image. Display on 17. That is, the image processing unit 15 outputs a warning when the current medium image paired with the previous medium image does not exist (step S123: No).

Next, in step S133, the image processing unit 15 determines whether or not to continue the processing. For example, when the operator gives an instruction to the image processing apparatus 10 to continue the processing in response to the warning output in step S131, the image processing unit 15 determines that the processing is continued (step S133: Yes). ). On the other hand, when the operator gives an instruction to the image processing apparatus 10 to stop the processing in response to the warning output in step S131, the image processing unit 15 determines that the processing is not continued (step S133). : No). If the process is continued (step S133: Yes), the process proceeds to step S119, and if the process is not continued (step S133: No), the process ends.

In step S119, the image processing unit 15 performs the position setting process C. The details of the position setting process C will be described later. After the process of step S119, the process proceeds to step S135.

Next, in step S135, the image processing unit 15 rearranges the medium image in the scanned image based on the results of the position setting processes A, B, and C (steps S127, S129, S119). Details of the process in step S135 will be described later.

Next, in step S136, the image processing unit 15 determines whether or not the processing of steps S115 to S135 has been completed for all the medium images included in the scanned image selected in step S113. When the processing of steps S115 to S135 is completed for all the medium images included in the scanned image selected in step S113 (step S136: Yes), the processing proceeds to step S137. On the other hand, if the scanned image selected in step S113 still has a medium image that has not been processed in steps S115 to S135 (step S136: No), the process returns to step S115.

In step S137, the control unit 11 determines whether or not the processing of steps S113 to S135 has been completed for all of the series of scanned images stored in the storage unit 13. When the processing of steps S113 to S135 is completed for all of the series of scanned images stored in the storage unit 13 (step S137: Yes), the processing proceeds to step S139. On the other hand, if there are still scanned images that have not been processed in steps S113 to S135 in the series of scanned images stored in the storage unit 13 (step S137: No), the processing proceeds to step S113. return.

In step S139, the image processing unit 15 sets an extraction area for the scanned image. Details of the process in step S139 will be described later.

Next, in step S141, the image processing unit 15 extracts the medium image based on the extraction region set in step S139. Details of the process in step S141 will be described later.

Hereinafter, an operation example based on the series of processes shown in FIGS. 2, 3, 5, and 7 will be described.

In the following, as shown in FIG. 9, the image processing unit 15 sets the X coordinate in the right direction and the Y coordinate in the downward direction with the upper left vertex of the scanned image as the origin on all the scanned images of the rectangle. That is, in the scanned image, the X coordinate is the horizontal coordinate and the Y coordinate is the vertical coordinate.

Further, for example, the scanned images RI11, RI12, RI13, and RI14 (FIGS. 10A, 11A, 12A, and 13A) correspond to a series of a plurality of scanned images of one old book. The scanned image RI 11 includes a medium image MI 11 which is an image of the front cover (FIG. 10A). That is, the medium image MI11 is arranged on the scanned image RI11. The scanned image RI 12 includes a medium image MI 121 which is a spread 12 image and a medium image MI 122 which is an image of a color chart (FIG. 11A). That is, the medium image MI121 and the medium image MI122 are arranged in the scanned image RI12. The scanned image RI 13 includes a medium image MI131 which is a spread 34 image and a medium image MI132 which is an image of a color chart (FIG. 12A). That is, the medium image MI131 and the medium image MI132 are arranged in the scanned image RI13. The scanned image RI 14 includes a medium image MI 14 which is an image of the back cover (FIG. 13A). That is, the medium image MI14 is arranged on the scanned image RI14. Further, the scanned images RI11, RI12, RI13, and RI14 include the platen image MTI.

First, according to the processing of steps S103 to S107 (FIG. 2), the image processing unit 15 sets the scanned images RI11, RI12, RI13, and RI14 as shown in FIGS. 10B, 11B, 12B, and 13B. Operate.

That is, as shown in FIG. 10B, the image processing unit 15 detects the circumscribed rectangle BR11 of the medium image MI11 and calculates the center of gravity GB11 of the detected circumscribed rectangle BR11. Further, since the upper side of the medium contour figure of the medium image MI11 is a straight line, the image processing unit 15 sets the midpoint of the upper side of the medium contour figure to the medium image reference position RRP11 in the medium image MI11.

Further, as shown in FIG. 11B, the image processing unit 15 detects the circumscribed rectangle BR121 of the medium image MI121 and calculates the center of gravity GB121 of the detected circumscribed rectangle BR121. Further, since the upper side of the medium contour figure of the medium image MI121 is not a straight line, the image processing unit 15 uses the middle point of the approximate straight line SL121 connecting the corner points CR121a and CR121b at both ends of the upper side of the medium contour figure as a medium in the medium image MI121. The image reference position is set to RRP121. On the other hand, the image processing unit 15 detects the circumscribed rectangle BR122 of the medium image MI122 and calculates the center of gravity GB122 of the detected circumscribed rectangle BR122. Further, since the upper side of the medium contour figure of the medium image MI 122 is a straight line, the image processing unit 15 sets the midpoint of the upper side of the medium contour figure to the medium image reference position RRP 122 in the medium image MI 122.

Further, as shown in FIG. 12B, the image processing unit 15 detects the circumscribed rectangle BR131 of the medium image MI131 and calculates the center of gravity GB131 of the detected circumscribed rectangle BR131. Further, since the upper side of the medium contour figure of the medium image MI131 is not a straight line, the image processing unit 15 uses the middle point of the approximate straight line SL131 connecting the corner points CR131a and CR131b at both ends of the upper side of the medium contour figure as a medium in the medium image MI131. Set to the image reference position RRP131. On the other hand, the image processing unit 15 detects the circumscribed rectangle BR132 of the medium image MI132 and calculates the center of gravity GB132 of the detected circumscribed rectangle BR132. Further, since the upper side of the medium contour figure of the medium image MI132 is a straight line, the image processing unit 15 sets the midpoint of the upper side of the medium contour figure to the medium image reference position RRP132 in the medium image MI132.

Further, as shown in FIG. 13B, the image processing unit 15 detects the circumscribed rectangle BR14 of the medium image MI14 and calculates the center of gravity GB14 of the detected circumscribed rectangle BR14. Further, since the upper side of the medium contour figure of the medium image MI14 is a straight line, the image processing unit 15 sets the midpoint of the upper side of the medium contour figure to the medium image reference position RRP14 in the medium image MI14.

Here, the extrinsic rectangle BR11 indicates the region where the medium image MI11 exists in the scanned image RI11, the extrinsic rectangle BR121 indicates the region where the medium image MI121 exists in the scanned image RI12, and the extrinsic rectangle BR122 indicates the region where the medium image MI121 exists in the scanned image RI12. The area where the medium image MI 122 exists is shown. Further, the extrinsic rectangle BR131 indicates a region in which the medium image MI131 exists in the scanned image RI13, the extrinsic rectangle BR132 indicates the region in which the medium image MI132 exists in the scanned image RI13, and the extrinsic rectangle BR14 indicates the region in which the medium image MI132 exists in the scanned image RI14. The area where the image MI14 exists is shown.

As described above, the image processing unit 15 has the medium image reference positions RRP11, RRP121 in each of the scanned images RI11, RI12, RI13, and RI14 in the region where the medium images MI11, MI121, MI122, MI131, MI132, and MI14 are present. , RRP122, RRP131, RRP132, RRP14, respectively.

Since the scanned image RI14 is the last image in a series of a plurality of scanned images of one old book, when the processing of steps S101 to S107 is completed for the scanned image RI14 (step S109: Yes), The process proceeds to step S111.

Next, the image processing unit 15 operates as shown in FIGS. 10B, 11B, 12B, and 13B for each of the scanned images RI11, RI12, RI13, and RI14 according to the process of step S111 (FIG. 2). .. That is, as shown in FIGS. 10B, 11B, 12B, and 13B, the image processing unit 15 uses the midpoint of the upper side of the platen image MTI as a scanning image reference in each of the scanned images RI11, RI12, RI13, and RI14. Set to position MRP1.

In this way, the image processing unit 15 sets the scanned image reference position in each of the scanned images RI11, RI12, RI13, and RI14 in a region other than the region in which the medium images MI11, MI121, MI122, MI131, MI132, and MI14 are present. Set MRP1.

Next, the image processing unit 15 rotates the circumscribing rectangle BR11 clockwise around the center of gravity GB11 in the scanned image RI11 (FIG. 10B) selected by the control unit 11 in the process of step S113 (FIG. 2), thereby causing the medium. The inclination of the image MI11 is corrected (step S115, FIG. 10C).

Further, since the scanned image RI 11 is the first scanned image in a series of a plurality of scanned images (step S117: Yes), the image processing unit 15 performs the position setting process C on the medium image MI 11 (step S119). ). In the position setting process C for the medium image MI11, as shown in FIG. 10C, the image processing unit 15 starts from the scanned image reference position MRP1 (coordinates (x _s0 , y _s0 )), and the medium image MI11 after the upright correction. _{The vector Vec b0} whose end point is the medium image reference position RRP11 (coordinates (x _b0 , y _b0 )) in the above is set as the position of the medium image MI11 with respect to the platen image MTI. That is, the vector Vec _b0 set in the medium image MI11 indicates the positional relationship between the medium image reference position RRP11 and the scanned image reference position MRP1.

Then, as shown in FIG. 10C, the image processing unit 15 rearranges the medium image MI11 _{based on the vector Vec b0} set in the medium image MI11 in the scanned image RI11 (step S135). That is, the image processing unit 15 rearranges the medium image MI11 so that the medium image reference position RRP11 coincides with _{the coordinates (x b0} , y _b0 ) of the end point of the _{vector Vec b0.} The image processing unit 15 stores the read image RI11F after rearranging the medium image MI11 in the storage unit 13. The arrangement of the medium image MI11 after the rearrangement is the same as the arrangement of the medium image MI11 before the rearrangement (that is, the arrangement of the medium image MI11 after the upright correction).

Further, in the scanned image RI12 (FIG. 11B) selected by the control unit 11 in the process of step S113 (FIG. 2), the image processing unit 15 rotates the circumscribing rectangle BR121 counterclockwise around the center of gravity GB121 to form a medium. The inclination of the image MI 121 is corrected (step S115, FIG. 11C). Similarly, the image processing unit 15 corrects the inclination of the medium image MI 122 by rotating the circumscribed rectangle BR 122 clockwise around the center of gravity GB 122 (step S115, FIG. 11C).

Further, since the scanned image RI 12 is not the first scanned image in a series of a plurality of scanned images (step S117: No), the image processing unit 15 performs anti-image detection processing on the scanned image RI 12 (step S121). .. By this pair image detection process, the medium image MI11 and the medium image MI121 are determined to be paired images of the cover image and the spread image, while the medium image MI11 and the medium image MI122 are not paired images. It is judged.

Since it is determined that the medium image MI 11 and the medium image MI 121 are a pair of images of the cover image and the spread image, the image processing unit 15 performs the position setting process B on the medium image MI 121 (step S129). In the position setting process B for the medium image MI 121, as shown in FIG. 11C, the image processing unit 15 starts from the scanned image reference position MRP1 (coordinates (x _s0 , y _s0 )) and media image reference position RRP121 (coordinates (x)). _{The vector Vec b1} whose end point is b1, y _b0 )) is set as the position of the medium image MI121 with respect to the platen image MTI. _{That is, the vector Vec b1} set in the medium image MI 121 indicates the positional relationship between the medium image reference position RRP121 and the scanned image reference position MRP1. Here, in FIG. 11C, the X-coordinate value x _{b1 of the medium image reference position RRP121 is the same as the X-coordinate value x b1} of the medium image reference position RRP121 in the medium image MI121 after the upright correction, while the medium image. the value of Y-coordinate of the reference position RRP121 y _b0 is the same as the value y _b0 Y coordinate of the media image reference position RRP11 in FIG 10C.

Further, since it is determined that the medium image MI 11 and the medium image MI 122 are not paired images, the image processing unit 15 determines that the processing is continued in step S133 after the processing in step S131, with respect to the medium image MI 122. The position setting process C is performed (step S119). In the position setting process C for the medium image MI 122, as shown in FIG. 11C, the image processing unit 15 starts from the scanned image reference position MRP1 (coordinates (x _s0 , y _s0 )) and performs the upright correction of the medium image MI 122. _{The vector Vec c1} whose end point is the medium image reference position RRP122 (coordinates (x _c0 , y _c0 )) in the above is set as the position of the medium image MI122 with respect to the platen image MTI. _{That is, the vector Vec c1} set in the medium image MI 122 indicates the positional relationship between the medium image reference position RRP 122 and the scanned image reference position MRP1.

Then, as shown in FIG. 11C, the image processing unit 15 rearranges the medium image MI121 _{based on the vector Vec b1} set in the medium image MI121 in the scanned image RI12, and rearranges the medium image MI121 and sets the vector Vec _{c1 in the medium image MI122.} The medium image MI122 is rearranged based on (step S135). That is, the image processing unit 15 rearranges the medium image MI 121 so that the medium image reference position RRP 121 coincides with _{the coordinates (x b1} , y _b0 ) of the end point of the _{vector Vec b1.} Further, the image processing unit 15 rearranges the medium image MI 122 so that the medium image reference position RRP 122 coincides with _{the coordinates (x c0} , y _c0 ) of the end point of the _{vector Vec c1.} The image processing unit 15 stores the read image RI12F after rearranging the medium images MI121 and MI122 in the storage unit 13. Due to the rearrangement of the medium image MI121, the arrangement of the medium image MI121 after the rearrangement is changed from the arrangement of the medium image MI121 before the rearrangement (that is, the arrangement of the medium image MI121 after the upright correction). The arrangement of the medium image MI122 after the rearrangement is the same as the arrangement of the medium image MI122 before the rearrangement (that is, the arrangement of the medium image MI122 after the upright correction).

As described above, the image processing unit 15 rearranges the medium image MI121 so that the medium image reference position RRP121 coincides with _{the coordinates (x b1} , y _b0 ) of the end point of the _{vector Vec b1 (FIG. 11C).} Further, as described above, the X coordinate value x _{b1 of the medium image reference position RRP121 is the same as the X coordinate value x b1} of the medium image reference position RRP121 in the medium image MI121 after the upright correction, while the medium image reference. position value y _b0 Y coordinate of RRP121 is identical to the value y _b0 Y coordinate of the media image reference position RRP11 in FIG 10C. Further, as described above, the medium image MI11 and the medium image MI121 are paired images of the cover image and the spread image. Further, it is determined that one medium image and the other medium image are paired images of the cover image and the spread image (step S189), that is, the vertical length of one medium image and the vertical length of the other medium image. The difference is less than the threshold TH2 (step S179: Yes), the difference between the landscape of one medium image and the landscape of the other medium image is greater than or equal to the threshold TH3 (step S181: No), and of the other medium image. This is a case where the horizontal length is within a predetermined range of the horizontal length of one medium image (step S187: Yes). That is, in the image processing unit 15, the difference between the vertical length of the medium image MI11 and the vertical length of the medium image MI121 is less than the threshold value TH2, and the difference between the horizontal length of the medium image MI11 and the horizontal length of the medium image MI121 is the threshold value TH3 or more. When the horizontal length of the medium image MI 121 is within the predetermined range of the horizontal length of the medium image MI 11, the vertical arrangement of the medium image MI 121 is changed when the medium image MI 121 is rearranged, while the medium image MI 121 is rearranged. Do not change the horizontal arrangement of.

Further, the image processing unit 15 rotates the circumscribing rectangle BR131 clockwise around the center of gravity GB131 in the scanned image RI13 (FIG. 12B) selected by the control unit 11 in the process of step S113 (FIG. 2), thereby causing the medium. The inclination of the image MI131 is corrected (step S115, FIG. 12C). Similarly, the image processing unit 15 corrects the inclination of the medium image MI132 by rotating the circumscribed rectangle BR132 counterclockwise around the center of gravity GB132 (step S115, FIG. 12C).

Further, since the scanned image RI 13 is not the first scanned image in a series of a plurality of scanned images (step S117: No), the image processing unit 15 performs anti-image detection processing on the scanned image RI 13 (step S121). .. By this pair image detection process, it is determined that the medium image MI121 and the medium image MI131 are paired images of the spread images, and that the medium image MI122 and the medium image MI132 are paired images of the color chart images. ..

Since it is determined that the medium image MI 121 and the medium image MI 131 are paired images of the spread images, the image processing unit 15 performs the position setting process A on the medium image MI 131 (step S127). In the position setting process A for the medium image MI131, as shown in FIG. 12C, the image processing unit 15 starts from the scanned image reference position MRP1 (coordinates (x _s0 , y _s0 )) and media image reference position RRP131 (coordinates (x)). _{The vector Vec b1} whose end point is b1, y _b0 )) is set as the position of the medium image MI131 with respect to the platen image MTI. That is, the vector Vec _b1 set in the medium image MI131 indicates the positional relationship between the medium image reference position RRP131 and the scanned image reference position MRP1. Here, in FIG. 12C, the X coordinate value x _{b1 of the medium image reference position RRP131 is the same as the X coordinate value x b1} of the medium image reference position RRP121 in FIG. 11C, and the Y coordinate value of the medium image reference position RRP131. y _b0 is the same as _{the Y coordinate value y b0} of the medium image reference position RRP121 in FIG. 11C. That is, the vector Vec _b1 set in the medium image MI131 is the same _{as the vector Vec b1 set in the medium image MI121.}

Further, since it is determined that the medium image MI 122 and the medium image MI 132 are paired images of the color chart images, the image processing unit 15 performs the position setting process A on the medium image MI 132 (step S127). In the position setting process A for the medium image MI 132, as shown in FIG. 12C, the image processing unit 15 starts from the scanned image reference position MRP1 (coordinates (x _s0 , y _s0 )) and media image reference position RRP132 (coordinates (x)). _{The vector Vec c1} whose end point is c0, y _c0 )) is set as the position of the medium image MI132 with respect to the platen image MTI. That is, the vector Vec _c1 set in the medium image MI132 indicates the positional relationship between the medium image reference position RRP132 and the scanned image reference position MRP1. Here, in FIG. 12C, the X coordinate value x _{c0 of the medium image reference position RRP 132 is the same as the X coordinate value x c0} of the medium image reference position RRP 122 in FIG. 11C, and the Y coordinate value of the medium image reference position RRP 132. y _c0 is the same as _{the Y coordinate value y c0} of the medium image reference position RRP122 in FIG. 11C. That is, the vector Vec _c1 set in the medium image MI132 is the same _{as the vector Vec c1 set in the medium image MI122.}

Then, as shown in FIG. 12C, the image processing unit 15 rearranges the medium image MI131 _{based on the vector Vec b1} set in the medium image MI131 in the scanned image RI13, and rearranges the medium image MI131 and sets the vector Vec _{c1 in the medium image MI132.} The medium image MI132 is rearranged based on (step S135). That is, the image processing unit 15 rearranges the medium image MI131 so that the medium image reference position RRP131 coincides with _{the coordinates (x b1} , y _b0 ) of the end point of the _{vector Vec b1.} Further, the image processing unit 15 rearranges the medium image MI132 so that the medium image reference position RRP132 coincides with _{the coordinates (x c0} , y _c0 ) of the end point of the _{vector Vec c1.} The image processing unit 15 stores the read image RI13F after rearranging the medium images MI131 and MI132 in the storage unit 13. Due to the rearrangement of the medium image MI131, the arrangement of the medium image MI131 after the rearrangement is changed from the arrangement of the medium image MI131 before the rearrangement (that is, the arrangement of the medium image MI131 after the upright correction). Further, due to the rearrangement of the medium image MI132, the arrangement of the medium image MI132 after the rearrangement is changed from the arrangement of the medium image MI132 before the rearrangement (that is, the arrangement of the medium image MI132 after the upright correction).

As described above, the medium image MI121 and the medium image MI131 are paired images of spread images. Further, as described above, the vector Vec _b1 set in the medium image MI131 is the same _{as the vector Vec b1 set in the medium image MI121.} Further, as described above, the vector Vec _b1 set in the medium image MI121 indicates the positional relationship between the medium image reference position RRP121 and the scanned image reference position MRP1, and the vector Vec _b1 set in the medium image MI131 indicates the positional relationship. The positional relationship between the medium image reference position RRP131 and the scanned image reference position MRP1 is shown. Also as described above, while the medium image MI121 is repositioned to medium image reference position RRP121 the coordinates (x _b1, y _b0) of end point of the vector Vec _b1 match, the end point of the vector Vec _b1 coordinates ( The medium image MI131 is rearranged so that the medium image reference position RRP131 coincides with _{x b1} , y _b0). That is, the image processing unit 15 rearranges the medium image MI131 paired with the medium image MI121 so that the positional relationship between the medium image MI121 and the medium image MI131 is the same.

Further, as described above, the medium image MI122 and the medium image MI132 are paired images of color chart images. Further, as described above, the vector Vec _c1 set in the medium image MI132 is the same _{as the vector Vec c1 set in the medium image MI122.} Further, as described above, the vector Vec _c1 set in the medium image MI122 indicates the positional relationship between the medium image reference position RRP122 and the scanned image reference position MRP1, and the vector Vec _c1 set in the medium image MI132 is The positional relationship between the medium image reference position RRP132 and the scanned image reference position MRP1 is shown. Also as described above, while the medium image MI122 is repositioned to medium image reference position RRP122 the coordinates (x _c0, y _c0) of the end point of the vector Vec _c1 match, the end point of the vector Vec _c1 coordinates ( The medium image MI132 is rearranged so that the medium image reference position RRP132 coincides with _{x c0} , y _c0). That is, the image processing unit 15 rearranges the medium image MI132 paired with the medium image MI122 so that the positional relationship between the medium image MI122 and the medium image MI132 is the same.

Further, since the circumscribed rectangle BR14 of the medium image MI14 included in the scanned image RI14 selected by the control unit 11 by the process of step S113 (FIG. 2) has no inclination (FIG. 13B), the upright correction is performed with respect to the medium image MI14. Even if this is done (step S115), the medium image MI14 is not rotated.

Further, since the scanned image RI 14 is not the first scanned image in a series of a plurality of scanned images (step S117: No), the image processing unit 15 performs anti-image detection processing on the scanned image RI 14 (step S121). .. By this pair image detection process, it is determined that the medium image MI131 and the medium image MI14 are paired images of the cover image and the spread image.

Since it is determined that the medium image MI131 and the medium image MI14 are paired images of the cover image and the spread image, the image processing unit 15 performs the position setting process B on the medium image MI14 (step S129). In the position setting process B for the medium image MI14, as shown in FIG. 13C, the image processing unit 15 starts from the scanned image reference position MRP1 (coordinates (x _s0 , y _s0 )) and media image reference position RRP14 (coordinates (x)). _{The vector Vec b3} ending in b3, y _b0 )) is set as the position of the medium image MI14 with respect to the platen image MTI. That is, the vector Vec _b3 set in the medium image MI14 indicates the positional relationship between the medium image reference position RRP14 and the scanned image reference position MRP1. Here, in FIG. 13C, the X-coordinate value x _{b3 of the medium image reference position RRP14 is the same as the X-coordinate value x b3} of the medium image reference position RRP14 of the medium image MI14 in FIG. 13B, while the medium image reference position. the value y _b0 Y coordinate of RRP14 is identical to the value y _b0 Y coordinate of the media image reference position RRP131 in FIG 12C.

Then, as shown in FIG. 13C, the image processing unit 15 rearranges the medium image MI14 _{based on the vector Vec b3} set in the medium image MI14 in the scanned image RI14 (step S135). That is, the image processing unit 15 rearranges the medium image MI14 so that the medium image reference position RRP14 coincides with _{the coordinates (x b3} , y _b0 ) of the end point of the _{vector Vec b3.} The image processing unit 15 stores the read image RI14F after rearranging the medium image MI14 in the storage unit 13. Due to the rearrangement of the medium image MI14, the arrangement of the medium image MI14 after the rearrangement is changed from the arrangement of the medium image MI14 before the rearrangement (that is, the arrangement of the medium image MI14 after the upright correction).

As described above, the image processing unit 15 rearranges the medium image MI14 so that the medium image reference position RRP14 coincides with _{the coordinates (x b3} , y _b0 ) of the end point of the _{vector Vec b3 (FIG. 13C).} Further, as described above, the X coordinate value x _{b3 of the medium image reference position RRP14 is the same as the X coordinate value x b3} of the medium image reference position RRP14 in the medium image MI14 after the erecting correction, while the medium image reference. position value y _b0 Y coordinate of RRP14 is identical to the value y _b0 Y coordinate of the media image reference position RRP131 in FIG 12C. Further, as described above, the medium image MI14 and the medium image MI131 are paired images of the cover image and the spread image. Further, it is determined that one medium image and the other medium image are paired images of the cover image and the spread image (step S189), that is, the vertical length of one medium image and the vertical length of the other medium image. The difference is less than the threshold TH2 (step S179: Yes), the difference between the landscape of one medium image and the landscape of the other medium image is greater than or equal to the threshold TH3 (step S181: No), and of the other medium image. This is a case where the horizontal length is within a predetermined range of the horizontal length of one medium image (step S187: Yes). That is, in the image processing unit 15, the difference between the vertical length of the medium image MI131 and the vertical length of the medium image MI14 is less than the threshold value TH2, and the difference between the horizontal length of the medium image MI131 and the horizontal length of the medium image MI14 is the threshold value TH3 or more. When the horizontal length of the medium image MI131 is within the predetermined range of the horizontal length of the medium image MI14, the vertical arrangement of the medium image MI14 is changed when the medium image MI14 is rearranged, while the medium image MI14 is rearranged. Do not change the horizontal arrangement of.

Since the scanned image RI14 is the last image in a series of a plurality of scanned images of one old book, when the processing of steps S113 to S135 is completed for the scanned image RI14 (step S137: Yes), The process proceeds to step S139 (extraction area setting).

In setting the extraction area, the image processing unit 15 acquires the read images RI11F, RI12F, RI13F, and RI14F from the storage unit 13. The scanned image RI11F includes the medium image MI11 after the erecting correction (FIG. 14A), and the scanned image RI12F includes the medium image MI121 after the erecting correction and the arrangement change, and the medium image MI122 after the erecting correction. And are included (Fig. 14B). Further, the scanned image RI13F includes a medium image MI131 after the upright correction and the arrangement change, and a medium image MI132 after the upright correction and the arrangement change (FIG. 14C). Further, the scanned image RI14F includes the medium image MI14 after the upright correction and the arrangement change (FIG. 14D).

As shown in FIGS. 14A to 14D, the image processing unit 15 is based on the medium images MI11, MI122 after the erecting correction and the medium images MI121, MI131, MI132, MI14 after the erecting correction and the arrangement change. An extraction region EA11 having the same position and size is set for the scanned images RI11F, RI12F, RI13F, and RI14F (step S139). That is, the vertical length DstH1 of the rectangular extraction region EA11 is the same in all of the scanned images RI11F, RI12F, RI13F, and RI14F. Further, the horizontal length DstW1 of the rectangular extraction region EA11 is the same in all of the scanned images RI11F, RI12F, RI13F, and RI14F. _{Further, the vector VecL d0} starting from the scanned image reference position MRP1 and ending at the upper left vertex of the rectangular extraction region EA11 is the same in all of the scanned images RI11F, RI12F, RI13F, and RI14F. _{Further, the vector VecR d0} starting from the scanned image reference position MRP1 and ending at the upper right vertex of the rectangular extraction region EA11 is the same in all of the scanned images RI11F, RI12F, RI13F, and RI14F. Further, the medium image MI11 exists in the extraction area EA11 set in the scanned image RI11F (FIG. 14A), and the medium images MI121 and 122 exist in the extraction area EA11 set in the scanned image RI12F (FIG. 14B). The medium images MI131 and 132 exist in the extraction area EA11 set in the scanned image RI13F (FIG. 14C), and the medium image MI14 exists in the extraction area EA11 set in the scanned image RI14F (FIG. 14D).

Therefore, as shown in FIGS. 15A to 15D, the image processing unit 15 extracts the medium image MI11 from the scanned image RI11F according to the same extraction region EA11 set in step S139, and extracts the medium images MI121 and MI122 from the scanned image RI12F. The media images MI131 and MI132 are extracted from the scanned image RI13F, and the medium image MI14 is extracted from the scanned image RI14F (step S141).

16A to 16C, FIGS. 17, 18A to 18C, and 19A to 19C show an example of setting an extraction area for the scanned image after rearranging the medium image. As an example, the scanned image RI1F includes a rearranged medium image MI1 having an extrinsic rectangle BR1 (FIG. 16A), and the scanned image RI2F includes a rearranged medium image MI2 having an extrinsic rectangle BR2 (FIG. 16A). 16B), the scanned image RI3F includes a rearranged medium image MI3 having an extrinsic rectangle BR3 and a rearranged medium image MI4 having an extrinsic rectangle BR4 (FIG. 16C). Further, the scanned image reference position MRP1 is set in all of the scanned images RI1F, RI2F, and RI3F (FIGS. 16A to 16C).

The image processing unit 15 sets the extraction region EA1 based on the circumscribed rectangle including all the medium images MI1, MI2, MI3, MI4 after the rearrangement in the scanned images RI1F, RI2F, and RI3F. For example, as shown in FIG. 17, the image processing unit 15 sets a rectangle including all of the circumscribed rectangles BR1, BR2, BR3, and BR4 in the extraction region EA1. Further, as shown in FIG. 17, the image processing unit 15 has a vector VEA1a whose starting point is the scanned image reference position MRP1 and whose end point is the upper left apex of the extraction area EA1 and whose start point is the scanned image reference position MRP1. A vector VEA1b whose end point is the upper right vertex of the EA1 is set with respect to the extraction region EA1.

Next, the image processing unit 15 sets the extraction region EA1 in which the vectors VEA1a and VEA1b are set in each of the scanned images RI1F, RI2F, and RI3F (FIGS. 18A to 18C). Therefore, the medium image MI1 exists in the extraction region EA1 set in the scanned image RI1F (FIG. 18A), and the medium image MI2 exists in the extraction region EA1 set in the scanned image RI2F (FIG. 18B). The medium images MI3 and MI4 are present in the extraction region EA1 set in RI3F (FIG. 18C). The extraction region EA1 set in each of the images RI1F, RI2F, and RI3F has the same position and size in all of the images RI1F, RI2F, and RI3F.

Therefore, as shown in FIGS. 19A to 19C, the image processing unit 15 extracts the medium image MI1 from the scanned image RI1F and extracts the medium image MI2 from the scanned image RI2F according to the extraction region EA1 having the same position and size. The medium images MI3 and MI4 are extracted from the scanned image RI3F.

FIG. 20 shows an example of a series of scanned images of the front cover image of an old book, 12 spread images, 34 spread images, ..., Back cover images, and after the processed images shown in FIG. 2 are performed on the scanned images. An example of the image (that is, the processed image) is shown.

As described above, in the first embodiment, the image processing device 10 has a storage unit 13 and an image processing unit 15. The storage unit 13 stores a series of a plurality of scanned images, each including a medium image. In each of the plurality of scanned images, the image processing unit 15 sets a medium image reference position in an area where the medium image exists (hereinafter, may be referred to as a “medium image area”), and sets the medium image reference position in an area other than the medium image area. Set the scanned image reference position. Further, the image processing unit 15 rearranges the medium image in the scanned image based on the positional relationship between the medium image reference position and the scanned image reference position. Further, the image processing unit 15 sets the same extraction region in a plurality of scanned images based on the medium image after rearrangement. Then, the image processing unit 15 extracts a medium image from each of the plurality of scanned images according to the set extraction area.

By doing so, it is possible to suppress the positional deviation between the medium images included in each of the series of a plurality of scanned images. Therefore, it is possible to reduce the discomfort of a person who continuously views the medium images included in each of a series of a plurality of scanned images. For this reason, for example, when an operator installs a book on the platen of an overhead scanner, it is not necessary to carefully install the book while being careful not to shift the position of the medium image between pages. Therefore, for example, it is possible to improve the work efficiency of the operator when converting one book into electronic data by using an overhead scanner.

Further, in the first embodiment, the storage unit 13 stores at least two read images. The image processing unit 15 includes a medium image included in one of the two scanned images and a medium image included in the other scanned image of the two scanned images and included in one of the scanned images. The medium image included in the other scanned image is rearranged so that the positional relationship between the image and the paired medium image is the same.

By doing so, it is possible to suppress the positional deviation between the medium images paired with each other (for example, between the spread images or between the color chart images).

Further, in the first embodiment, the image processing unit 15 sets the extraction region based on the rectangle including the circumscribed rectangle of all the medium images after rearrangement in the plurality of scanned images.

By doing so, an appropriate extraction area can be set, especially when the medium image is a spread image. Further, when the scanned image includes different types of medium images such as a spread image and a color chart image, an appropriate extraction region can be set.

Further, in the first embodiment, the storage unit 13 stores at least two read images. In the image processing unit 15, the difference between the vertical length of the medium image included in one of the two scanned images and the vertical length of the medium image included in the other scanned image of the two scanned images is the threshold TH2. Less than, the difference between the horizontal length of the medium image included in one scanned image and the horizontal length of the medium image included in the other scanned image is equal to or more than the threshold TH3, and the horizontal length of the medium image included in the other scanned image. When the width is within a predetermined range of the horizontal length of the medium image included in one scanned image, when rearranging the medium image included in the other scanned image, the vertical arrangement is changed while the horizontal arrangement is changed. Do not change the placement.

By doing so, for example, when the cover image (that is, the image for one page) and the spread image (that is, the image for two pages) are mixed as the medium image, the arrangement of the medium image is excessively changed. It can be suppressed. Therefore, it is possible to reduce the discomfort of a person who continuously views the cover image and the spread image.

Further, in the first embodiment, the image processing unit 15 outputs a warning when there is no paired medium image.

By doing so, the operator can know that there is no paired medium image. Therefore, the operator can appropriately select whether or not to continue the process when the paired medium image does not exist.

The first embodiment has been described above.

[Example 2]
In the second embodiment, similarly to the first embodiment, a case where each page of one old book is read by a scanner and one book is converted into electronic data will be described as an example. However, in the second embodiment, as shown in FIG. 21, the operator cuts the old book BK turned to the right and cuts off the back, so that each page separated one by one is used as a scanner with an ADF (Auto Document Feeder). The case of continuous reading will be described as an example. FIG. 21 is a diagram showing an example of an old book of Example 2. A plurality of manuscripts generated by cutting one old book BK are put together by an operator on a shooter. Multiple originals placed on the shooter are continuously scanned into the scanner one by one by the ADF, and the scanner reads the front side (front side) image and the back side (back side) image of each document. Be done.

<Configuration of image processing device>
Since the configuration example of the image processing apparatus of the second embodiment is the same as that of the first embodiment (FIG. 1), the description thereof will be omitted.

<Processing and operation of image processing equipment>
22, FIG. 23, FIG. 24, FIG. 25, and FIG. 37 are diagrams showing an example of the processing flow of the image processing apparatus of the second embodiment. 26A-C, 27A-C, 28A-C, 29A-C, 30A-D, 31A-D, 32, 33A-C, 34, 35A-C, 36A-C. , And FIGS. 38A to 38C are diagrams for explaining an operation example of the image processing apparatus of the second embodiment.

The processing flow shown in FIG. 22 is started when, for example, the processing start button (not shown) of the image reading device 10 is pressed by the operator, as in the first embodiment.

In step S101 of FIG. 22, the control unit 11 selects a scanned image to be processed by the image processing unit 15 from a series of a plurality of scanned images stored in the storage unit 13, and the storage unit 13 And outputs the acquired scanned image to the image processing unit 15. However, the blank image has been removed in advance and is not stored in the storage unit 13.

For example, when a series of images of the front cover image, the first page image, the second page image, ..., The back cover image are stored in the storage unit 13 as the read image of one old book, The control unit 11 displays the image of the front cover in step S101 in the first processing loop of steps S101 to S109, the image of the first page in step S101 in the second processing loop of steps S101 to S109, and the image of the first page in steps S101 to S109. In step S101 in the third processing loop of the above, the images on the third page are sequentially acquired from the storage unit 13. Further, the control unit 11 acquires an image of the back cover from the storage unit 13 in step S101 in the final processing loop of steps S101 to S109. Further, for example, a series of images of the front cover image, the first page image, the second page image, ..., The back cover image are stored as one file in the storage unit 13, and the control unit 11 processes them. From the target file, each image of the first page image, the second page image, ..., The back cover image is selected in order. The file to be processed is arbitrarily selected by the operator, for example.

In FIG. 22, the processes of steps S103 to S105 are the same as those of the first embodiment (FIG. 2), and thus the description thereof will be omitted.

In step S201 of FIG. 22, the image processing unit 15 performs the medium image reference position setting process.

FIG. 23 shows an example of the processing flow of the medium image reference position setting process. The processing of steps S151, S153, S155, and S161 in FIG. 23 is the same as that shown in the first embodiment (FIGS. 3 and 5), and thus the description thereof will be omitted.

In step S211 of FIG. 23, the image processing unit 15 performs cutting location detection processing.

24 and 25 show an example of the processing flow of the cutting portion detection process. FIG. 24 shows an example of the first processing of the cutting location detection process, and FIG. 25 shows an example of the second processing of the cutting location detection process. Hereinafter, the cutting portion detection process will be described separately for the first process example and the second process example.

<Example of first processing of cutting location detection processing (FIG. 24)>
In step S221 of FIG. 24, the image processing unit 15 determines whether or not the series of scanned images sequentially selected in step S101 is a left-turned image. Whether the old book to be converted into electronic data is a left-turning book or a right-turning book is instructed in advance by the operator to the image processing device 10, and the instruction result of "left-turning" or "right-turning" is stored. It is stored in advance in the unit 13. When the instruction result stored in the storage unit 13 is "left-turning", the image processing unit 15 determines that a series of a plurality of read images are left-turning images, and stores them in the storage unit 13. When the instruction result is "right-turning", it is determined that a series of a plurality of scanned images are right-turning images. When the series of the plurality of scanned images is a left-turned image (step S221: Yes), the process proceeds to step S223, and when the series of the plurality of scanned images is a right-turned image (step S221: No). , The process proceeds to step S229.

In step S223, the image processing unit 15 determines whether or not the scanned image selected in step S101 is an odd-numbered page image. If the scanned image selected in step S101 is an odd-numbered page image (step S223: Yes), the process proceeds to step S225, and if the scanned image selected in step S101 is an even-numbered page image (step). S223: No), the process proceeds to step S227.

Also in step S229, the image processing unit 15 determines whether or not the scanned image selected in step S101 is an odd-numbered page image. If the scanned image selected in step S101 is an odd-numbered page image (step S229: Yes), the process proceeds to step S227, and if the scanned image selected in step S101 is an even-numbered page image (step). S229: No), the process proceeds to step S225.

In step S225, the image processing unit 15 determines that the cut portion (hereinafter, may be referred to as “media cut portion”) in the medium image exists on the left side of the medium image.

On the other hand, in Tep S227, the image processing unit 15 determines that the medium cutting portion exists on the right side of the medium image.

<Example of second processing of cutting location detection processing (Fig. 25)>
In step S231 of FIG. 25, the image processing unit 15 calculates the corner angle LA on the left side of the medium image included in the scanned image selected in step S101. For example, the image processing unit 15 calculates the angle of the upper left corner of the medium image as the corner angle LA.

Next, in step S233, the image processing unit 15 calculates the difference angle DLA, which is the absolute value of the difference between the corner angle LA and the right angle (90 °).

Next, in step S235, the image processing unit 15 calculates the corner angle RA on the right side of the medium image included in the scanned image selected in step S101. For example, the image processing unit 15 calculates the angle of the upper right corner of the medium image as the corner angle RA.

Next, in step S237, the image processing unit 15 calculates the difference angle DRA, which is the absolute value of the difference between the corner angle RA and the right angle (90 °).

Next, in step S239, the image processing unit 15 determines whether or not the difference angle DLA is less than the difference angle DRA. If the difference angle DLA is less than the difference angle DRA (step S239: Yes), the process proceeds to step S227, and if the difference angle DLA is greater than or equal to the difference angle DRA (step S239: No), the process proceeds to step S225. move on.

In step S227, the image processing unit 15 determines that the medium cutting portion exists on the right side of the medium image.

On the other hand, in step S225, the image processing unit 15 determines that the medium cutting portion exists on the left side of the medium image.

The processing example of the cutting portion detection processing has been described above.

Returning to FIG. 23, in step S213, the image processing unit 15 determines whether or not the medium cutting portion exists on the left side of the medium image. If the medium cut portion exists on the left side of the medium image (that is, when the process of step S225 is performed) (step S213: Yes), the process proceeds to step S215. On the other hand, if the medium cut portion exists on the right side of the medium image (that is, when the process of step S227 is performed) (step S213: No), the process proceeds to step S217.

In step S215, the image processing unit 15 sets the right corner point of the upper side of the medium image included in the scanned image as the medium image reference position.

On the other hand, in step S217, the image processing unit 15 sets the left corner point of the upper side of the medium image included in the scanned image as the medium image reference position.

Returning to FIG. 22, in step S109, the control unit 11 determines whether or not the processing of steps S101 to S201 has been completed for all of the series of scanned images stored in the storage unit 13. When the processing of steps S101 to S201 is completed for all of the series of scanned images stored in the storage unit 13 (step S109: Yes), the processing proceeds to step S111. On the other hand, if the scanned images that have not been processed in steps S101 to S201 remain in the series of scanned images stored in the storage unit 13 (step S109: No), the process proceeds to step S101. return. For example, when a series of images of the front cover image, the first page image, the second page image, ..., The back cover image, and the back cover image are stored in the storage unit 13 as the read image of one old book, When the processing of steps S101 to S201 is completed for the back cover image, the processing proceeds to step S111.

Since the processes of steps S111 to S133 in FIG. 22 are the same as those shown in the first embodiment (FIG. 2), the description thereof will be omitted. However, since the scanned image of Example 2 does not include a spread image or a color chart image, in the target image detection process (FIG. 7) performed in step S121 of Example 2, the process of step S183 or step S189 is performed. When this is done, it is simply determined that the current medium image and the previous medium image are paired images.

In step S123, the image processing unit 15 determines whether or not a pair of images exists in the processing of steps S171 to S191 (FIG. 7). When the processing of step S183 or step S189 is performed in FIG. 7, it is determined that the paired image exists, and when the processing of step S191 is performed in FIG. 7, it is determined that the paired image does not exist. .. If the paired image exists (step S123: Yes), the process proceeds to step S203, and if the paired image does not exist (step S123: No), the process proceeds to step S131.

In step S203, the image processing unit 15 performs the position setting process D. The details of the position setting process D will be described later. After the process of step S203, the process proceeds to step S135.

Next, in step S135, the image processing unit 15 rearranges the medium image in the scanned image based on the results of the position setting processes C and D (steps S119 and S203). Details of the process in step S135 will be described later.

Next, in step S137, the control unit 11 determines whether or not the processes of steps S113 to S135 and S203 have been completed for all of the series of scanned images stored in the storage unit 13. When the processes of steps S113 to S135 and S203 are completed for all of the series of scanned images stored in the storage unit 13 (step S137: Yes), the process proceeds to step S205. On the other hand, in a series of a series of scanned images stored in the storage unit 13, if the scanned images that have not been processed in steps S113 to S135 and S203 remain (step S137: No), the processing is a step. Return to S113.

In step S205, the image processing unit 15 sets an extraction area for the scanned image. Details of the process in step S205 will be described later.

Next, in step S141, the image processing unit 15 extracts the medium image based on the extraction region set in step S205. Details of the process in step S141 will be described later.

Hereinafter, an operation example based on the series of processes shown in FIGS. 22, 23, 24, and 25 will be described.

In the following, as in the first embodiment, the image processing unit 15 sets the X coordinate in the right direction and the Y coordinate in the downward direction with the upper left vertex of the scanned image as the origin on all the scanned images of the rectangle. That is, in the scanned image, the X coordinate is the horizontal coordinate and the Y coordinate is the vertical coordinate.

Further, for example, the scanned images RI21, RI22, RI23, and RI24 (FIGS. 26A, 27A, 28A, and 29A) correspond to a series of a plurality of scanned images of one old book. The scanned image RI21 includes the medium image MI21 which is an image of the front cover, and the medium cut portion CP in the medium image MI21 exists on the right side of the medium image MI21 (FIG. 26A). That is, the medium image MI21 is arranged on the scanned image RI21. The scanned image RI 22 includes the medium image MI 22 which is the image of the first page, and the medium cutting portion CP in the medium image MI 22 exists on the left side of the medium image MI 22 (FIG. 27A). That is, the medium image MI22 is arranged on the scanned image RI22. The scanned image RI23 includes the medium image MI23 which is the image of the second page, and the medium cutting portion CP in the medium image MI23 exists on the right side of the medium image MI23 (FIG. 28A). That is, the medium image MI23 is arranged on the scanned image RI23. The scanned image RI24 includes a medium image MI24 which is an image of the back cover, and the medium cut portion CP in the medium image MI24 is on the left side of the medium image MI24 (FIG. 29A). That is, the medium image MI24 is arranged on the scanned image RI24.

First, according to the processing of steps S103 to S105 and S201 (FIG. 22), the image processing unit 15 shows FIGS. 26B, 27B, 28B, and 29B for each of the scanned images RI21, RI22, RI23, and RI24. Works like this.

That is, as shown in FIG. 26B, the image processing unit 15 detects the circumscribed rectangle BR21 of the medium image MI21 and calculates the center of gravity GB21 of the detected circumscribed rectangle BR21. Further, since the medium cutting portion CP exists on the right side of the medium image MI21, the image processing unit 15 sets the upper left apex of the circumscribed rectangle BR21 at the medium image reference position RRP21.

Further, as shown in FIG. 27B, the image processing unit 15 detects the circumscribed rectangle BR22 of the medium image MI22 and calculates the center of gravity GB22 of the detected circumscribed rectangle BR22. Further, since the medium cutting portion CP exists on the left side of the medium image MI22, the image processing unit 15 sets the upper right apex of the circumscribed rectangle BR22 at the medium image reference position RRP22.

Further, as shown in FIG. 28B, the image processing unit 15 detects the circumscribed rectangle BR23 of the medium image MI23 and calculates the center of gravity GB23 of the detected circumscribed rectangle BR23. Further, since the medium cutting portion CP exists on the right side of the medium image MI23, the image processing unit 15 sets the upper left apex of the circumscribed rectangle BR23 at the medium image reference position RRP23.

Further, as shown in FIG. 29B, the image processing unit 15 detects the circumscribed rectangle BR24 of the medium image MI24 and calculates the center of gravity GB24 of the detected circumscribed rectangle BR24. Further, since the medium cutting portion CP exists on the left side of the medium image MI24, the image processing unit 15 sets the upper right apex of the circumscribed rectangle BR24 at the medium image reference position RRP24.

Here, the extrinsic rectangle BR21 indicates the region where the medium image MI21 exists in the scanned image RI21, the extrinsic rectangle BR22 indicates the region where the medium image MI22 exists in the scanned image RI22, and the extrinsic rectangle BR23 indicates the region where the medium image MI22 exists in the scanned image RI23. The region where the medium image MI23 exists is shown, and the circumscribing rectangle BR24 shows the region where the medium image MI24 exists in the scanned image RI24.

As described above, the image processing unit 15 has the medium image reference positions RRP21, RRP22, RRP23, and RRP24 in the regions where the medium images MI21, MI22, MI23, and MI24 are present in each of the scanned images RI21, RI22, RI23, and RI24. To set each.

Since the scanned image RI24 is the last image in a series of a plurality of scanned images of one old book, when the processing of steps S101 to S105 and S201 is completed for the scanned image RI24 (step S109: Yes). ), The process proceeds to step S111.

Next, the image processing unit 15 operates as shown in FIGS. 26B, 27B, 28B, and 29B for each of the scanned images RI21, RI22, RI23, and RI24 according to the process of step S111 (FIG. 2). .. That is, as shown in FIGS. 26B, 27B, 28B, and 29B, the image processing unit 15 has read images in each of the rectangular read images RI21, RI22, RI23, and RI24 having the same vertical and horizontal lengths. The midpoint of the upper side of RI21, RI22, RI23, and RI24 is set to the scanned image reference position MRP2. When the horizontal length of the rectangular scanned images RI21, RI22, RI23, and RI24 is expressed as SrcW, the coordinates of the scanned image reference position MRP2 are expressed as (SrcW / 2,0).

In this way, the image processing unit 15 sets the scanned image reference position MRP2 in each of the scanned images RI21, RI22, RI23, and RI24 in a region other than the region in which the medium images MI21, MI22, MI23, and MI24 are present. ..

Next, since the circumscribed rectangle BR21 of the medium image MI21 included in the scanned image RI21 selected by the control unit 11 by the process of step S113 (FIG. 22) has no inclination (FIG. 26B), the upright correction is performed on the medium image MI21. Even if this is done (step S115), the medium image MI21 is not rotated.

Further, since the scanned image RI 21 is the first scanned image in a series of a plurality of scanned images (step S117: Yes), the image processing unit 15 performs the position setting process C on the medium image MI 21 (step S119). ). In the position setting process C for the medium image MI21, as shown in FIG. 26C, the image processing unit 15 starts from the scanned image reference position MRP2 (coordinates (SrcW / 2,0)), and the medium image MI21 after erect correction is performed. _{The vector VecL b0} whose end point is the medium image reference position RRP21 (coordinates (x _b0 , y _b0 )) in the above is set as the position of the medium image MI21 with respect to the scanned image RI21. That is, the vector VecL _b0 set in the medium image MI21 indicates the positional relationship between the medium image reference position RRP21 and the scanned image reference position MRP2.

Then, as shown in FIG. 26C, the image processing unit 15 rearranges the medium image MI21 _{based on the vector VecL b0} set in the medium image MI21 in the scanned image RI21 (step S135). That is, the image processing unit 15 rearranges the medium image MI 21 so that the medium image reference position RRP 21 coincides with _{the coordinates (x b0} , y _b0 ) of the end point of the _{vector VecL b0.} The image processing unit 15 stores the read image RI21F after rearranging the medium image MI21 in the storage unit 13. The arrangement of the medium image MI21 after the rearrangement is the same as the arrangement of the medium image MI21 before the rearrangement (that is, the arrangement of the medium image MI21 after the upright correction).

Further, since the circumscribed rectangle BR22 of the medium image MI22 included in the scanned image RI22 selected by the control unit 11 by the process of step S113 (FIG. 22) has no inclination (FIG. 27B), the upright correction is performed with respect to the medium image MI22. Even if this is done (step S115), the medium image MI22 is not rotated.

Further, since the scanned image RI 22 is not the first scanned image in a series of a plurality of scanned images (step S117: No), the image processing unit 15 performs anti-image detection processing on the scanned image RI 22 (step S121). .. By this pair image detection process, it is determined that the medium image MI21 and the medium image MI22 are paired images.

Since it is determined that the medium image MI21 and the medium image MI22 are paired images, the image processing unit 15 performs the position setting process D on the medium image MI22 (step S203). In the position setting process D for the medium image MI22, as shown in FIG. 27C, the image processing unit 15 starts at the scanned image reference position MRP2 (coordinates (SrcW / 2,0)) and starts at the medium image reference position RRP22 (coordinates (x)). _{The vector VecR b0} ending in b1, y _b0 )) is set as the position of the medium image MI22 with respect to the scanned image RI22. _{That is, the vector VecR b0} set in the medium image MI 22 indicates the positional relationship between the medium image reference position RRP 22 and the scanned image reference position MRP2. Here, the vector VecR _b0 set in the medium image MI22 is a vector symmetric with the vector _{VecL b0} set in the medium image MI21 in the X coordinate direction with respect to the scanned image reference position MRP2. That, Y-coordinate y _b0 of the end point of the vector VecR _b0 is the same as the Y-coordinate y _b0 of the end point of the vector VecL _b0. The distance from the read image reference position MRP2 to X-coordinate x _b1 of the end point of the vector VecR _b0 is the same as the distance from the read image reference position MRP2 to X-coordinate x _b0 of the end point of the vector VecL _b0.

Then, as shown in FIG. 27C, the image processing unit 15 rearranges the medium image MI22 _{based on the vector VecR b0} set in the medium image MI22 in the scanned image RI22 (step S135). That is, the image processing unit 15 rearranges the medium image MI 22 so that the medium image reference position RRP 22 coincides with _{the coordinates (x b1} , y _b1 ) of the end point of the _{vector VecR b0.} The image processing unit 15 stores the read image RI22F after rearranging the medium image MI22 in the storage unit 13. Due to the rearrangement of the medium image MI22, the arrangement of the medium image MI22 after the rearrangement is changed from the arrangement of the medium image MI22 before the rearrangement (that is, the arrangement of the medium image MI22 after the upright correction).

Further, the image processing unit 15 rotates the circumscribing rectangle BR23 clockwise around the center of gravity GB23 in the scanned image RI23 (FIG. 28B) selected by the control unit 11 in the process of step S113 (FIG. 2), thereby causing the medium. The inclination of the image MI23 is corrected (step S115, FIG. 28C).

Further, since the scanned image RI23 is not the first scanned image in a series of a plurality of scanned images (step S117: No), the image processing unit 15 performs anti-image detection processing on the scanned image RI23 (step S121). .. By this pair image detection process, it is determined that the medium image MI22 and the medium image MI23 are paired images.

Since it is determined that the medium image MI 22 and the medium image MI 23 are paired images, the image processing unit 15 performs the position setting process D on the medium image MI 23 (step S203). In the position setting process D for the medium image MI23, as shown in FIG. 28C, the image processing unit 15 starts at the scanned image reference position MRP2 (coordinates (SrcW / 2,0)) and starts at the medium image reference position RRP23 (coordinates (x)). _{The vector VecL b0} ending in b0, y _b0 )) is set as the position of the medium image MI23 with respect to the scanned image RI23. That is, the vector VecL _b0 set in the medium image MI23 indicates the positional relationship between the medium image reference position RRP23 and the scanned image reference position MRP2. Here, the vector VecL _b0 set in the medium image MI23 is the same _{vector as the vector VecL b0 set in the medium image MI21.}

Then, as shown in FIG. 28C, the image processing unit 15 rearranges the medium image MI23 _{based on the vector VecL b0} set in the medium image MI23 in the scanned image RI23 (step S135). That is, the image processing unit 15 rearranges the medium image MI23 so that the medium image reference position RRP23 coincides with _{the coordinates (x b0} , y _b0 ) of the end point of the _{vector VecL b0.} The image processing unit 15 stores the read image RI23F after rearranging the medium image MI23 in the storage unit 13. Due to the rearrangement of the medium image MI23, the arrangement of the medium image MI23 after the rearrangement is changed from the arrangement of the medium image MI23 before the rearrangement (that is, the arrangement of the medium image MI23 after the upright correction).

As described above, the medium image MI21 and the medium image MI22 are paired images, and the medium image MI22 and the medium image MI23 are paired images. Therefore, the medium image MI21 and the medium image MI23 are paired images. Further, as described above, the vector VecL _b0 set in the medium image MI23 is the same _{as the vector VecL b0 set in the medium image MI21.} Further, as described above, the vector VecL _b0 set in the medium image MI21 indicates the positional relationship between the medium image reference position RRP21 and the scanned image reference position MRP2, and the vector VecL _b0 set in the medium image MI23 is The positional relationship between the medium image reference position RRP23 and the scanned image reference position MRP2 is shown. Also as described above, while the medium image MI21 are repositioned to medium image reference position RRP21 the coordinates (x _b0, y _b0) of end point of the vector VecL _b0 match, the end point of the vector VecL _b0 coordinates ( The medium image MI23 is rearranged so that the medium image reference position RRP23 coincides with _{x b0} , y _b0). That is, the image processing unit 15 rearranges the medium image MI23 paired with the medium image MI21 so that the positional relationship between the medium image MI21 and the medium image MI23 is the same.

Further, since the circumscribing rectangle BR24 of the medium image MI24 included in the scanned image RI24 selected by the control unit 11 by the process of step S113 (FIG. 22) has no inclination (FIG. 29B), the upright correction is performed with respect to the medium image MI24. Even if this is done (step S115), the medium image MI24 is not rotated.

Further, since the scanned image RI24 is not the first scanned image in a series of a plurality of scanned images (step S117: No), the image processing unit 15 performs anti-image detection processing on the scanned image RI24 (step S121). .. By this pair image detection process, it is determined that the medium image MI23 and the medium image MI24 are paired images.

Since it is determined that the medium image MI23 and the medium image MI24 are paired images, the image processing unit 15 performs the position setting process D on the medium image MI24 (step S203). In the position setting process D for the medium image MI24, as shown in FIG. 29C, the image processing unit 15 starts at the scanned image reference position MRP2 (coordinates (SrcW / 2,0)) and starts at the medium image reference position RRP24 (coordinates (x)). _{The vector VecR b0} ending in b1, y _b0 )) is set as the position of the medium image MI24 with respect to the scanned image RI24. That is, the vector VecR _b0 set in the medium image MI24 indicates the positional relationship between the medium image reference position RRP24 and the scanned image reference position MRP2. Here, the vector VecR _b0 set in the medium image MI24 is the same _{vector as the vector VecR b0 set in the medium image MI22.}

Then, as shown in FIG. 29C, the image processing unit 15 rearranges the medium image MI24 _{based on the vector VecR b0} set in the medium image MI24 in the scanned image RI24 (step S135). That is, the image processing unit 15 rearranges the medium image MI24 so that the medium image reference position RRP24 coincides with _{the coordinates (x b1} , y _b1 ) of the end point of the _{vector VecR b0.} The image processing unit 15 stores the read image RI24F after rearranging the medium image MI24 in the storage unit 13. Due to the rearrangement of the medium image MI24, the arrangement of the medium image MI24 after the rearrangement is changed from the arrangement of the medium image MI24 before the rearrangement (that is, the arrangement of the medium image MI24 after the upright correction).

As described above, the medium image MI22 and the medium image MI23 are paired images, and the medium image MI23 and the medium image MI24 are paired images. Therefore, the medium image MI22 and the medium image MI24 are paired images. Further, as described above, the vector VecR _b0 set in the medium image MI22 is the same _{as the vector VecR b0 set in the medium image MI24.} Further, as described above, the vector VecR _b0 set in the medium image MI22 indicates the positional relationship between the medium image reference position RRP22 and the scanned image reference position MRP2, and the vector VecR _b0 set in the medium image MI24 is The positional relationship between the medium image reference position RRP24 and the scanned image reference position MRP2 is shown. Also as described above, while the medium image MI22 are repositioned to medium image reference position RRP22 the coordinates (x _b1, y _b0) of end point of the vector VecR _b0 match, the end point of the vector VecR _b0 coordinates ( The medium image MI24 is rearranged so that the medium image reference position RRP24 coincides with _{x b1} , y _b0). That is, the image processing unit 15 rearranges the medium image MI24 paired with the medium image MI22 so that the positional relationship between the medium image MI22 and the medium image MI24 is the same.

Since the scanned image RI24 is the last image in a series of a plurality of scanned images of one old book, when the processing of steps S113 to S135 and S203 is completed for the scanned image RI24 (step S137: Yes). ), The process proceeds to step S205 (extraction area setting).

In setting the extraction area, the image processing unit 15 acquires the read images RI21F, RI22F, RI23F, and RI24F from the storage unit 13. The scanned image RI21F includes the medium image MI21 (FIG. 30A), the scanned image RI22F includes the media image MI22 after the arrangement change (FIG. 30B), and the scanned image RI23F has the upright correction and arrangement. The modified medium image MI23 is included (FIG. 30C), and the scanned image RI24F includes the modified medium image MI24 (FIG. 30D).

As shown in FIGS. 30A to 30D, the image processing unit 15 sets the extraction regions EA21 and EA22 having the same size as the scanned images RI21F, RI22F, RI23F and RI24F based on the medium images MI21, MI22, MI23 and MI24. Set (step S139). That is, the vertical length DstH2 of the rectangular extraction region EA21 is the same in all of the scanned images RI21F, RI22F, RI23F, and RI24F. Further, the horizontal length DstW2 of the rectangular extraction region EA21 is the same in all of the scanned images RI21F, RI22F, RI23F, and RI24F. Further, as described above, is identical to the vector VecL _b0 set in vector VecL _b0 and MI23 set in the medium image MI21, it was set to the vector VecR _b0 and medium image MI24 set in the medium image MI22 Same as the vector VecR _b0 . Therefore, the extraction region EA21 having the same position and size is set for the scanned image RI21F and the scanned image RI23F. Further, for the scanned image RI22F and the scanned image RI24F, an extraction region EA22 having the same position and size is set. Further, the extraction area EA21 and the extraction area EA22 have the same size and the position in the Y coordinate direction on the scanned image, but are different in the position in the X coordinate direction on the scanned image. The distance from the scanned image reference position MRP2 to the upper left apex of the extraction area EA21 is the same as the distance from the scanned image reference position MRP2 to the upper right apex of the extraction area EA22. Further, the extraction region EA21 set in the scanned image RI21F has a main part of the medium image MI21 (FIG. 30A), and the extraction region EA22 set in the scanned image RI22F has the main part of the medium image MI22 (FIG. 30A). FIG. 30B), the extraction region EA21 set in the scanned image RI23F has a main part of the medium image MI23 (FIG. 30C), and the extraction region EA22 set in the scanned image RI24F has the main part of the medium image MI24. (Fig. 30D).

Here, the "main part" of the medium image is a part in the medium image in which at least a part of some information such as characters, photographs, figures, and tables exists.

Therefore, as shown in FIGS. 31A to 31D, the image processing unit 15 extracts the main portion MI21C of the medium image MI21 from the scanned image RI21F according to the extraction regions EA21 and EA22 set in step S139, and media images from the scanned image RI22F. The main part MI22C of MI22 is extracted, the main part MI23C of the medium image MI23 is extracted from the scanned image RI23F, and the main part MI24C of the medium image MI24 is extracted from the scanned image RI24F (step S141).

Then, as shown in FIG. 32, the image processing unit 15 causes the display unit 17 to display the main parts MI21C, MI22C, MI23C, and MI24C. That is, the image processing unit 15 displays the main part MI21C of the image on the front cover, the main part MI22C of the image on the first page, the main part MI23C of the image on the second page, ..., The main part MI24C of the image on the back cover in order. It is displayed on the unit 17. Further, the image processing unit 15 causes the display unit 17 to display the main part MI21C of the front cover image and the main part MI24C of the back cover image as independent pages, while the main part of the image on the first page. The MI22C and the main part MI23C of the image on the second page are joined at the cut portion and displayed on the display unit 17 as a double-page spread.

33A to 33C, FIGS. 34A to 35C, and FIGS. 36A to 36C show an example of setting an extraction area for the scanned image after rearranging the medium image. As an example, the scanned image RI6F includes the rearranged medium image MI6 (FIG. 33A), the scanned image RI7F contains the rearranged medium image MI7 (FIG. 33B), and the scanned image RI8F is relocated. The arranged medium image MI8 is included (FIG. 33C).

As shown in FIG. 33A, since the medium cutting portion CP in the medium image MI6 exists on the right side of the medium image MI6, the image processing unit 15 moves the right side of the circumscribing rectangle of the medium image MI6 in parallel to the left. In the scanned image RI6F, a rectangle IR6 that does not include the medium cutting portion CP is created.

Further, as shown in FIG. 33B, since the medium cutting portion CP in the medium image MI7 exists on the left side of the medium image MI7, the image processing unit 15 moves the left side of the circumscribing rectangle of the medium image MI7 in parallel to the right. Therefore, in the scanned image RI7F, a rectangle IR7 that does not include the medium cutting portion CP is created.

Further, as shown in FIG. 33C, since the medium cutting portion CP in the medium image MI8 exists on the left side of the medium image MI8, the image processing unit 15 translates the right side of the circumscribing rectangle of the medium image MI8 to the left. Therefore, in the scanned image RI8F, a rectangle IR8 that does not include the medium cutting portion CP is created.

Next, as shown in FIG. 34, the image processing unit 15 determines a rectangular region in which all of the rectangles IR6, IR7, and IR8 overlap as the extraction region EA2. Further, the image processing unit 15 obtains a vector VL2 starting from the scanned image reference position MRP2 and ending at the upper left vertex of the rectangular extraction region EA2.

Then, as shown in FIG. 35A, the image processing unit 15 sets the extraction region EA2 in which the vector VL2 is set in the scanned image RI6F.

Further, as shown in FIG. 35B, the image processing unit 15 sets the extraction region EA3 in which the vector VL3 is set in the scanned image RI7F. Here, the size of the extraction region EA3 is the same as the size of the extraction region EA2. Further, the vector VL3 is a vector symmetrical with respect to the vector VL2 in the X coordinate direction with respect to the scanned image reference position MRP2. That is, the position of the extraction area EA3 in the Y coordinate direction is the same as the position of the extraction area EA2 in the Y coordinate direction. The distance from the scanned image reference position MRP2 to the upper right apex of the extraction area EA3 is the same as the distance from the scanned image reference position MRP2 to the upper left apex of the extraction area EA23.

Further, as shown in FIG. 35C, the image processing unit 15 sets the extraction region EA2 in which the vector VL2 is set in the scanned image RI8F. Therefore, for the scanned image RI6F and the scanned image RI8F, the scanned image RI8F having the same position and size is set.

Therefore, the main part of the medium image MI6 exists in the extraction area EA2 set in the scanned image RI6F, and the main part of the medium image MI7 exists in the extraction area EA3 set in the scanned image RI7F. The main part of the medium image MI8 exists in the set extraction region EA2.

Therefore, as shown in FIGS. 36A to 36C, the image processing unit 15 extracts the main portion MI6C of the medium image MI6 from the scanned image RI6F according to the extraction regions EA2 and EA3, and extracts the main portion MI7C of the medium image MI7 from the scanned image RI7F. Is extracted, and the main part MI8C of the medium image MI8 is extracted from the scanned image RI8F.

FIG. 37 shows an example of a rectangle creation process that does not include the medium cutting portion.

In step S251 of FIG. 37, the image processing unit 15 detects an inflection point in the medium contour figure.

Next, in step S253, the image processing unit 15 determines whether or not the medium cutting portion exists on the left side of the medium image. If the medium cut portion is on the left side of the medium image (step S253: Yes), the process proceeds to step S255, and if the medium cut portion is on the right side of the medium image (step S253: No), the process is a step. Proceed to S257.

In step S255, the image processing unit 15 detects the position of the inflection point having the maximum value of the X coordinate (hereinafter, may be referred to as “the maximum value position of the X coordinate”) among the inflection points detected in step S251. do.

On the other hand, in step S257, the image processing unit 15 may refer to the position of the inflection point having the minimum value of the X coordinate among the inflection points detected in step S251 (hereinafter, referred to as "the minimum value position of the X coordinate"). ) Is detected.

Then, in step S259, when the medium cutting portion exists on the left side of the medium image, the image processing unit 15 translates the left side of the circumscribing rectangle of the medium image to the rightward to the X coordinate maximum value position detected in step S255. By doing so, a rectangle that does not include the medium cut portion is created in the scanned image. Further, when the medium cutting portion exists on the right side of the medium image, the image processing unit 15 translates the right side of the circumscribing rectangle of the medium image to the left to the minimum value position of the X coordinate detected in step S257. In the scanned image, a rectangle that does not include the medium cut portion is created.

Hereinafter, as shown in FIG. 38A, an operation example will be described when the scanned image RI includes the medium image MI and the medium cutting portion exists on the left side of the medium image MI in which the circumscribed rectangle BR is detected. As shown in FIG. 38B, the image processing unit 15 detects a plurality of inflection point IPs in the medium contour figure of the medium image MI. Since the medium cutting point CP exists on the left side of the medium image MI, the image processing unit 15 then, as shown in FIG. 38C, of the inflection point IP having the maximum value of the X coordinate among the plurality of inflection point IPs. Detect position IPM. Then, as shown in FIG. 38C, the image processing unit 15 translates the left side of the circumscribing rectangle BR to the position IPM (that is, the X coordinate maximum value position) in the right direction, thereby causing the medium cutting portion in the scanned image RI. Create a rectangular IR that does not include CP.

As described above, in the second embodiment, the image processing device 10 has a storage unit 13 and an image processing unit 15. The storage unit 13 stores a series of a plurality of scanned images, each including a medium image. The image processing unit 15 sets the medium image reference position in the medium image area and sets the scanned image reference position in the area other than the medium image area in each of the plurality of scanned images. Further, the image processing unit 15 rearranges the medium image in the scanned image based on the positional relationship between the medium image reference position and the scanned image reference position. Further, the image processing unit 15 sets the same extraction region in a plurality of scanned images based on the medium image after rearrangement. Then, the image processing unit 15 extracts a medium image from each of the plurality of scanned images according to the set extraction area.

By doing so, it is possible to suppress the positional deviation between the medium images included in each of the series of a plurality of scanned images. For this reason, for example, when one book is cut and each page separated one by one is converted into electronic data using ADF, the positional deviation between the medium images due to the influence of skew during transportation is suppressed. be able to. Therefore, it is possible to reduce the discomfort of a person who continuously views a medium image converted into electronic data using ADF over a plurality of pages.

Further, in the second embodiment, the storage unit 13 stores at least two read images. The image processing unit 15 includes a medium image included in one of the two scanned images and a medium image included in the other scanned image of the two scanned images and included in one of the scanned images. The medium image included in the other scanned image is rearranged so that the positional relationship between the image and the paired medium image is the same.

By doing so, it is possible to suppress the positional deviation between the medium images paired with each other (for example, between the medium images of odd-numbered pages or between the medium images of even-numbered pages).

Further, in the second embodiment, the image processing unit 15 sets the extraction area based on the rectangular area where the circumscribed rectangles of all the medium images after rearrangement in the plurality of scanned images overlap.

By doing so, an appropriate extraction area can be set, especially when the medium image includes a medium cut portion.

Further, in the second embodiment, the image processing unit 15 detects the medium cut portion in the medium image and sets the medium image reference position on the medium image based on the detected medium cut portion.

By doing so, it is possible to set an appropriate medium image reference position for the medium image including the medium cut portion.

The second embodiment has been described above.

[Example 3]
In the third embodiment, the method of setting the medium image reference position and the method of setting the extraction area are different from those of the second embodiment. Hereinafter, the differences from the second embodiment will be described.

<Processing and operation of image processing equipment>
FIG. 39 is a diagram showing an example of a processing flow of the image processing apparatus of the third embodiment. 40A-C, 41A-C, 42A-C, 43A-C, 44A-D, 45A-D, 46, 47A-C, 48, 49A-C, and 50A. ~ C are diagrams provided for explaining an operation example of the image processing apparatus of the third embodiment.

FIG. 39 shows an example of the processing flow of the medium image reference position setting process. Since the processes of steps S151, S153, S155, and S161 in FIG. 39 are the same as those shown in the second embodiment (FIG. 23), the description thereof will be omitted.

In step S271 of FIG. 39, the image processing unit 15 sets the center of gravity of the circumscribed rectangle of the medium image at the medium image reference position.

Hereinafter, an operation example of the image processing apparatus of the third embodiment will be described.

For example, the scanned images RI31, RI32, RI33, and RI34 (FIGS. 40A, 41A, 42A, 43A) correspond to a series of plurality of scanned images of one old book. The scanned image RI31 includes the medium image MI31 which is an image of the front cover, and the medium cut portion CP in the medium image MI31 exists on the right side of the medium image MI31 (FIG. 40A). That is, the medium image MI31 is arranged on the scanned image RI31. The scanned image RI32 includes the medium image MI32 which is the image of the first page, and the medium cutting portion CP in the medium image MI32 exists on the left side of the medium image MI32 (FIG. 41A). That is, the medium image MI32 is arranged on the scanned image RI32. The scanned image RI33 includes the medium image MI33 which is the image of the second page, and the medium cut portion CP in the medium image MI33 exists on the right side of the medium image MI33 (FIG. 42A). That is, the medium image MI33 is arranged on the scanned image RI33. The scanned image RI34 includes the medium image MI34 which is an image of the back cover, and the medium cut portion CP in the medium image MI34 exists on the left side of the medium image MI34 (FIG. 43A). That is, the medium image MI34 is arranged on the scanned image RI34.

First, as shown in FIG. 40B, the image processing unit 15 detects the circumscribed rectangle BR31 of the medium image MI31 and calculates the center of gravity GB31 of the detected circumscribed rectangle BR31. Further, the image processing unit 15 sets the center of gravity GB31 at the medium image reference position of the medium image MI31.

Further, as shown in FIG. 41B, the image processing unit 15 detects the circumscribed rectangle BR32 of the medium image MI32 and calculates the center of gravity GB32 of the detected circumscribed rectangle BR32. Further, the image processing unit 15 sets the center of gravity GB32 at the medium image reference position of the medium image MI32.

Further, as shown in FIG. 42B, the image processing unit 15 detects the circumscribed rectangle BR33 of the medium image MI33 and calculates the center of gravity GB33 of the detected circumscribed rectangle BR33. Further, the image processing unit 15 sets the center of gravity GB33 at the medium image reference position of the medium image MI33.

Further, as shown in FIG. 43B, the image processing unit 15 detects the circumscribed rectangle BR34 of the medium image MI34 and calculates the center of gravity GB34 of the detected circumscribed rectangle BR34. Further, the image processing unit 15 sets the center of gravity GB34 at the medium image reference position of the medium image MI34.

Next, the image processing unit 15 sets a rectangular cropping region CA31 with respect to the scanned image RI31 (FIG. 40B). The cropping area CA31 includes the circumscribed rectangle BR31, the vertical length of the cropping area CA31 is the vertical length of the circumscribed rectangle BR31 with a predetermined margin MG1, and the horizontal length of the cropping area CA31 is the horizontal length of the circumscribed rectangle BR31. A predetermined margin MG2 is attached. Then, the image processing unit 15 cuts out the cropping image RI31F from the scanned image RI31 according to the cropping region CA31 (FIG. 40C).

Further, the image processing unit 15 sets a rectangular cropping region CA32 for the scanned image RI32 (FIG. 41B). The cropping area CA32 includes the circumscribed rectangle BR32, the vertical length of the cropping area CA32 is the vertical length of the circumscribed rectangle BR32 with a predetermined margin MG1, and the horizontal length of the cropping area CA32 is the horizontal length of the circumscribed rectangle BR32. A predetermined margin MG2 is attached. Then, the image processing unit 15 cuts out the cropping image RI32F from the scanned image RI32 according to the cropping region CA32 (FIG. 41C).

Further, the image processing unit 15 sets a rectangular cropping region CA33 with respect to the scanned image RI33 (FIG. 42B). The cropping area CA33 includes the circumscribed rectangle BR33, the vertical length of the cropping area CA33 is the vertical length of the circumscribed rectangle BR33 with a predetermined margin MG1, and the horizontal length of the cropping area CA33 is the horizontal length of the circumscribed rectangle BR33. A predetermined margin MG2 is attached. Then, the image processing unit 15 cuts out the cropped image RI33F after the upright correction from the scanned image RI33 according to the cropping region CA33 (FIG. 42C).

Further, the image processing unit 15 sets a rectangular cropping region CA34 with respect to the scanned image RI34 (FIG. 43B). The cropping area CA34 includes the circumscribed rectangle BR34, the vertical length of the cropping area CA34 is the vertical length of the circumscribed rectangle BR34 with a predetermined margin MG1, and the horizontal length of the cropping area CA34 is the horizontal length of the circumscribed rectangle BR34. A predetermined margin MG2 is attached. Then, the image processing unit 15 cuts out the cropping image RI34F from the scanned image RI34 according to the cropping region CA34 (FIG. 43C).

Next, as shown in FIGS. 44A to 44D, the image processing unit 15 has an extraction region having the same position and size as the cropping images RI31F, RI32F, RI33F, and RI34F based on the medium images MI31, M322, MI33, and MI34. Set EA31. The vertical length DstH3 of the rectangular extraction region EA31 is the same in all of the cropping images RI31F, RI32F, RI33F, and RI34F. Further, the horizontal length DstW3 of the rectangular extraction region EA31 is the same in all of the cropping images RI31F, RI32F, RI33F, and RI34F. Further, the main part of the medium image MI31 exists in the extraction region EA31 set in the cropping image RI31F (FIG. 44A), and the main part of the medium image MI32 exists in the extraction region EA31 set in the cropping image RI32F (FIG. 44A). 44B), the main part of the medium image MI33 is present in the extraction region EA31 set in the cropping image RI331F (FIG. 44C), and the main part of the medium image MI34 is present in the extraction region EA31 set in the cropping image RI34F. (Fig. 44D).

Therefore, as shown in FIGS. 45A to 45D, the image processing unit 15 extracts the main part MI31C of the medium image MI31 from the cropping image RI31F and extracts the main part MI32C of the medium image MI32 from the cropping image RI32F according to the extraction region EA31. Then, the main part MI33C of the medium image MI33 is extracted from the cropped image RI33F, and the main part MI34C of the medium image MI34 is extracted from the cropped image RI34F.

Then, as shown in FIG. 46, the image processing unit 15 causes the display unit 17 to display the main parts MI31C, MI32C, MI33C, and MI34C. That is, the image processing unit 15 displays the main part MI31C of the image on the front cover, the main part MI32C of the image on the first page, the main part MI33C of the image on the second page, ..., The main part MI34C of the image on the back cover in order. It is displayed on the unit 17. Further, the image processing unit 15 causes the display unit 17 to display the main part MI31C of the front cover image and the main part MI34C of the back cover image as independent pages, while the main part of the image on the first page. The MI32C and the main part MI33C of the image on the second page are joined at the cut portion and displayed on the display unit 17 as a double-page spread.

47A to 47C, FIGS. 48, 49A to 49C, and 50A to 50C show examples of setting the extraction region for the scanned image. As an example, the cropping image RI41F includes a medium image MI41 (FIG. 47A), the cropping image RI42F includes a medium image MI42 (FIG. 47B), and the cropping image RI43F includes a medium image MI43 (FIG. 47C). .. Further, the cropping image RI41F is set with the center of gravity GB41 of the circumscribing rectangle of the medium image MI41, the cropping image RI42F is set with the center of gravity GB42 of the circumscribing rectangle of the medium image MI42, and the cropping image RI43F is set with the center of gravity GB42 of the medium image MI43. The center of gravity GB43 of the circumscribing rectangle is set.

As shown in FIG. 47A, since the medium cutting portion CP in the medium image MI41 exists on the right side of the medium image MI41, the image processing unit 15 moves the right side of the circumscribing rectangle of the medium image MI41 in parallel to the left. In the cropping image RI41F, a rectangle IR41 that does not include the medium cutting portion CP is created.

Further, as shown in FIG. 47B, since the medium cutting portion CP in the medium image MI42 exists on the left side of the medium image MI42, the image processing unit 15 moves the left side of the circumscribing rectangle of the medium image MI42 in parallel to the right. Therefore, in the cropping image RI42F, a rectangle IR42 that does not include the medium cutting portion CP is created.

Further, as shown in FIG. 47C, since the medium cutting portion CP in the medium image MI43 exists on the right side of the medium image MI43, the image processing unit 15 moves the right side of the circumscribing rectangle of the medium image MI43 in parallel to the left. Therefore, in the cropping image RI43F, a rectangle IR43 that does not include the medium cutting portion CP is created.

Next, as shown in FIG. 48, the image processing unit 15 arranges the rectangles IR41, IR42, and IR43 by aligning the centers of gravity GB41, GB42, and GB43 with each other. Then, the image processing unit 15 determines a rectangular region in which all of the rectangles IR41, IR42, and IR43 overlap as the extraction region EA4. Therefore, the center of gravity GB5 of the rectangular extraction region EA4 coincides with the centers of gravity GB41, GB42, and GB43.

Then, as shown in FIG. 49A, the image processing unit 15 sets the extraction region EA4 in the cropping image RI41F by matching the center of gravity GB5 with the center of gravity GB41.

Further, as shown in FIG. 49B, the image processing unit 15 sets the extraction region EA4 in the cropping image RI42F by matching the center of gravity GB5 with the center of gravity GB42.

Further, as shown in FIG. 49C, the image processing unit 15 sets the extraction region EA4 in the cropping image RI43F by matching the center of gravity GB5 with the center of gravity GB43.

Therefore, the main part of the medium image MI41 exists in the extraction area EA4 set in the cropping image RI41F, and the main part of the medium image MI42 exists in the extraction area EA4 set in the cropping image RI42F. The main part of the medium image MI43 exists in the set extraction region EA4.

Therefore, as shown in FIGS. 50A to 50C, the image processing unit 15 extracts the main part MI41C of the medium image MI41 from the cropping image RI41F and extracts the main part MI42C of the medium image MI42 from the cropping image RI42F according to the extraction region EA4. Then, the main part MI43C of the medium image MI43 is extracted from the cropping image RI43F.

The third embodiment has been described above.

[Example 4]
In the first embodiment, the image processing unit 15 sets the midpoint of the upper side of the medium contour figure or the midpoint of the straight line connecting the corner points at both ends of the upper side of the medium contour figure as the medium image reference position.

However, the image processing unit 15 may set the midpoint of the lower side of the medium contour figure or the midpoint of the straight line connecting the corner points at both ends of the lower side of the medium contour figure as the medium image reference position. Further, the image processing unit 15 may set the midpoint of the left side of the medium contour figure or the midpoint of the straight line connecting the corner points at both ends of the left side of the medium contour figure as the medium image reference position. Further, the image processing unit 15 may set the midpoint of the right side of the medium contour figure or the midpoint of the straight line connecting the corner points at both ends of the right side of the medium contour figure as the medium image reference position.

Further, the image processing unit 15 may set the upper end of the throat portion of the book as the medium image reference position in the medium image.

[Example 5]
In the first embodiment, the image processing unit 15 sets the midpoint of the upper side of the platen image included in the scanned image as the scanned image reference position.

However, the image processing unit 15 may set the midpoint of the lower side of the platen image included in the scanned image as the scanned image reference position.

Further, the image processing unit 15 may set any vertex of the rectangular scanned image as the scanned image reference position.

[Example 6]
In the first embodiment, the image processing unit 15 sets the extraction region based on the rectangle including the circumscribed rectangle of all the medium images after rearrangement in the plurality of scanned images. Further, in the second embodiment, the image processing unit 15 sets the extraction area based on the rectangular area where the circumscribed rectangles of all the medium images after rearrangement in the plurality of scanned images overlap. When setting the extraction area in Examples 1 and 2, the image processing unit 15 may set the extraction area based on the area in which the circumscribed rectangle is provided with a predetermined margin. By doing so, it is possible to reduce the possibility that a part of the medium image is missing when the medium image is extracted from the scanned image.

[Example 7]
The storage unit 13 is realized as hardware, for example, by a memory. Examples of the memory include RAM (Random Access Memory) such as SDRAM (Synchronous Dynamic Random Access Memory), ROM (Read Only Memory), and flash memory.

The control unit 11 and the image processing unit 15 are realized as hardware, for example, by a processor. Examples of processors include CPUs (Central Processing Units), DSPs (Digital Signal Processors), FPGAs (Field Programmable Gate Arrays), and the like. Further, the control unit 11 and the image processing unit 15 may be realized by an LSI (Large Scale Integrated circuit) including a processor and peripheral circuits. Further, the control unit 11 and the image processing unit 15 may be realized by using a GPU (Graphics Processing Unit), an ASIC (Application Specific Integrated Circuit), or the like.

The display unit 17 is realized as hardware, for example, by a display. An example of a display used in the image processing device 10 is a liquid crystal display.

All or part of each process in the above description in the image processing device 10 may be realized by causing a processor included in the image processing device 10 to execute a program corresponding to each process. For example, the program corresponding to each process in the above description may be stored in the memory, and the program may be read from the memory by the processor and executed. Further, the program is stored in a program server connected to the image processing device 10 via an arbitrary network, downloaded from the program server to the image processing device 10 and executed, or a record that can be read by the image processing device 10. It may be stored in a medium, read from the recording medium, and executed. Recording media that can be read by the image processing device 10 include, for example, memory cards, USB memory, SD cards, flexible disks, magneto-optical disks, CD-ROMs, DVDs, and Blu-ray (registered trademark) disks. Carrying storage media is included. The program is a data processing method described in an arbitrary language or an arbitrary description method, and may be in any format such as source code or binary code. In addition, the program is not necessarily limited to a single program, but is distributed as multiple modules or multiple libraries, or cooperates with a separate program represented by the OS to achieve its function. Including things.

The specific form of dispersion / integration of the image processing device 10 is not limited to the one shown in the drawing, and all or a part of the image processing device 10 can be used as an arbitrary unit according to various additions or functional loads. It can be functionally or physically distributed and integrated.

1 Image processing device 11 Control unit 13 Storage unit 15 Image processing unit 17 Display unit

Claims (12)

A storage unit, each of which stores a series of a plurality of scanned images including a medium image,
In each of the plurality of scanned images, the first reference position is set in the first region where the medium image exists, the second reference position is set in the second region other than the first region, and the first reference position is set. The medium image is rearranged in the scanned image based on the positional relationship between the second reference position and the second reference position.
An image processing unit that sets the same extraction region in the plurality of scanned images based on the rearranged medium image and extracts the medium image from each of the plurality of scanned images according to the extracted region.
An image processing device comprising. The plurality of scanned images include two scanned images.
The image processing unit includes the first medium image included in one of the two scanned images and the other scanned image of the two scanned images, and the first medium image. The second medium image is rearranged so that the positional relationship is the same between the second medium image and the paired second medium image.
The image processing apparatus according to claim 1. The image processing unit sets the extraction region based on a rectangle including the circumscribed rectangles of all the medium images after rearrangement in the plurality of scanned images.
The image processing apparatus according to claim 1. The image processing unit sets the extraction region based on a rectangular region in which the circumscribed rectangles of all the media images after rearrangement in the plurality of scanned images overlap.
The image processing apparatus according to claim 1. The image processing unit sets the midpoint of the upper side of the platen image included in the scanned image at the second reference position.
The image processing apparatus according to claim 1. The image processing unit sets the apex of the rectangular scanned image at the second reference position.
The image processing apparatus according to claim 1. The image processing unit sets the midpoint of the straight line connecting the two corner points on the medium image at the first reference position.
The image processing apparatus according to claim 1. The image processing unit sets the extraction area based on the area in which the circumscribed rectangle is provided with a predetermined margin.
The image processing apparatus according to claim 3 or 4. The image processing unit detects a cut portion of the medium in the medium image and sets the first reference position on the medium image based on the cut portion.
The image processing apparatus according to claim 1. The plurality of scanned images include two scanned images.
The image processing unit has the vertical length of the first medium image included in one of the two scanned images and the second medium image included in the other scanned image of the two scanned images. The difference between the vertical length and the vertical length of the first medium image is less than the first threshold value, the difference between the horizontal length of the first medium image and the horizontal length of the second medium image is greater than or equal to the second threshold value, and the second is the second. When the horizontal length of the medium image is within a predetermined range of the horizontal length of the first medium image, the vertical arrangement of the second medium image is changed, while the horizontal direction of the second medium image is changed. Do not change the placement of
The image processing apparatus according to claim 1. The image processing unit outputs a warning when there is no medium image paired with the first medium image.
The image processing apparatus according to claim 2. Stores a series of multiple scanned images, each containing a medium image,
In each of the plurality of scanned images, the first reference position is set in the first region where the medium image exists, the second reference position is set in the second region other than the first region, and the first reference position is set. The medium image is rearranged in the scanned image based on the positional relationship between the second reference position and the second reference position.
The same extraction region is set in the plurality of scanned images based on the rearranged medium image, and the medium image is extracted from each of the plurality of scanned images according to the extraction region.
Image processing method.

Images