JP2020141274A - Image forming device and program - Google Patents

Abstract

To provide an image forming device and a program that can prevent unnecessary correction compared to a case where a reading method that reduces the correction range is not considered when the distortion that occurs in a binding part at the time of reading a book is corrected.SOLUTION: An image forming device 10A includes a conversion unit 30 that converts first reading data obtained by reading a spread page including a binding part of a book into multi-gradation data, a detection unit 32 that detects the correction range of the binding part from the multi-gradation data, an estimation unit 34 that estimates a reading method that reduces the correction range of the binding part from the correction range of the binding part detected by the detection unit 32 and the correction range of the binding part when the reading method is performed, and a display control unit 36 that performs control to display an image indicating the correction range of the binding part estimated by the estimation unit 34 together with the reading method.SELECTED DRAWING: Figure 2

Description

The present invention relates to an image forming apparatus and a program.

For example, Patent Document 1 describes an image correction device that corrects a scanned image obtained by scanning a book document in contact with the top or bottom of a contact glass by an image reading means. This image correction device includes a "H-shaped" distortion angle detecting means for detecting a "H-shaped" distortion angle of a scanned image by using a line-shaped portion included in a scanned image of a book document. This image correction device corrects the "H" distortion of the scanned image based on the "H" distortion angle detected by the "H" distortion angle detecting means, and then determines the sub-scanning direction of the scanned image. It is provided with a "H-shaped" distortion correction means for correcting shape distortion.

Further, Patent Document 2 describes an image reading device that reads a spread document placed on a platen. This image reading device includes a document stand on which the spread document is placed face down, and a reading unit for reading the spread document on the document stand. This image scanning device repeatedly scans a spread document by a scanning unit, and acquires an image from the scanned images in which deterioration that occurs in the image corresponding to a portion of the spread document that rises from the platen satisfies an allowable condition. It has a control unit.

Further, Patent Document 3 describes an image processing device that corrects the background with high accuracy even when the background is darker than characters or the like. This image processing device uses an image reading means for reading a book document placed on a contact glass and a color with the highest frequency of use as the background color based on the pixel value of the scanned image of the book document read by the image reading means. It has a background color detecting means for detecting. This image processing device has a background color correcting means for correcting the background color near the binding portion of the scanned image based on the background color detected by the background color detecting means and the pixel value of the flat portion of the book document.

Japanese Unexamined Patent Publication No. 2003-304390 JP-A-2010-268245 JP-A-2007-166430

By the way, when a spread page including a binding portion of a book is placed on a platen and distortion generated in the binding portion when reading the spread page is corrected, a reading method for reducing the correction range is not considered. Therefore, there is a possibility that the image quality may be deteriorated due to extra correction being performed even in a portion that should not be corrected.

INDUSTRIAL APPLICABILITY According to the present invention, when correcting distortion generated in a binding portion when reading a book, it is possible to prevent unnecessary correction from being performed as compared with the case where a reading method for reducing the correction range is not considered. It is an object of the present invention to provide a forming apparatus and a program.

In order to achieve the above object, the image forming apparatus according to the first aspect includes a conversion unit that converts the first read data obtained by reading the spread page including the binding part of the book into multi-gradation data, and the multi-level image. A detection unit that detects the correction range of the binding portion from the adjustment data, a reading method that reduces the correction range of the binding portion from the correction range of the binding portion detected by the detection unit, and the reading method are carried out. It includes an estimation unit that estimates the correction range of the binding portion in the case, and a control unit that controls to display an image showing the correction range of the binding portion estimated by the estimation unit together with the reading method. ..

Further, in the image forming apparatus according to the second aspect, in the image forming apparatus according to the first aspect, the estimation unit carried out the reading method and the reading method using the learning data obtained by machine learning. The correction range of the binding portion in the case is estimated.

Further, the image forming apparatus according to the third aspect is the learning reading data obtained by reading the spread page including the binding portion of the book in a predetermined state in the image forming apparatus according to the second aspect. The image showing the correction range of the binding portion detected from is associated with a reading method for reducing the correction range of the binding portion and an image showing the correction range of the binding portion when the reading method is performed. It is said that the data was created.

Further, in the image forming apparatus according to the fourth aspect, in the image forming apparatus according to the third aspect, the learning data is further associated with an image showing the background portion of the spread page detected from the reading reading data for learning. ing.

Further, the image forming apparatus according to the fifth aspect is the correction range of the binding portion detected by the detecting portion in the image forming apparatus according to any one of the first to fourth aspects. Estimate different reading methods depending on the shape of.

Further, in the image forming apparatus according to the sixth aspect, in the image forming apparatus according to the fifth aspect, the reading method is a method using a jig for pressing the book from above, and the binding portion is corrected. It is said that the installation position of the jig differs depending on the shape of the range.

Further, in the image forming apparatus according to the seventh aspect, in the image forming apparatus according to any one of the first to sixth aspects, the conversion unit uses the reading method displayed by the control unit. The second read data obtained by re-reading the spread page is converted into multi-gradation data, the detection unit detects the correction range of the binding portion from the multi-gradation data, and the control unit controls the first. Control is performed so that an image showing the correction range of the binding portion obtained from the 1 reading data and an image showing the correction range of the binding portion obtained from the second reading data are displayed side by side.

Further, the image forming apparatus according to the eighth aspect corrects the second reading data in the image forming apparatus according to the seventh aspect, with the correction range of the binding portion obtained from the second reading data as the correction target. It has more parts.

Further, in the image forming apparatus according to the ninth aspect, in the image forming apparatus according to any one aspect of the first to eighth aspects, the conversion unit further converts the first read data into binary data. , The detection unit further detects the shape of the shadow of the binding portion from the binary data, and the estimation unit further detects the shape of the shadow of the binding portion from the correction range of the binding portion and the shape of the shadow of the binding portion detected by the detection unit. , A reading method for reducing the correction range of the binding portion, and an estimation of the correction range of the binding portion and the shape of the shadow of the binding portion when the reading method is performed, and the control unit estimates by the estimation unit. Control is performed to display the corrected range of the binding portion and the image showing the shape of the shadow of the binding portion together with the reading method.

Further, in the image forming apparatus according to the tenth aspect, in the image forming apparatus according to the ninth aspect, the estimation unit carried out the reading method and the reading method using the learning data obtained by machine learning. The correction range of the binding portion and the shape of the shadow of the binding portion in the case are estimated.

In addition, the image forming apparatus according to the eleventh aspect is the learning reading data obtained by reading the spread page including the binding portion of the book in a predetermined state in the image forming apparatus according to the tenth aspect. A reading method for reducing the correction range of the binding portion and a correction range of the binding portion when the reading method is performed on an image showing the correction range of the binding portion and the shape of the shadow of the binding portion detected from And the data is learned by associating an image showing the shape of the shadow of the binding portion.

Further, in order to achieve the above object, the program according to the twelfth aspect causes the computer to function as each part included in the image forming apparatus according to any one aspect of the first aspect to the eleventh aspect.

According to the first aspect and the twelfth aspect, when correcting the distortion generated in the binding portion when reading a book, an extra correction is performed as compared with the case where the reading method for reducing the correction range is not considered. Has the effect of being able to prevent.

According to the second aspect, there is an effect that an appropriate reading method and correction range can be estimated as compared with the case where the learning data is not used.

According to the third aspect, the image showing the correction range obtained from the learning read data is compared with the case where the reading method for reducing the correction range and the image showing the correction range when the reading method is performed are not associated with each other. Therefore, it has the effect of being able to obtain appropriate learning data.

According to the fourth aspect, there is an effect that appropriate learning data can be obtained as compared with the case where the image showing the background portion is not associated.

According to the fifth aspect, there is an effect that an appropriate reading method can be estimated as compared with the case where the same reading method is estimated according to the shape of the correction range.

According to the sixth aspect, there is an effect that an appropriate reading method can be obtained as compared with the case where the jigs are installed at the same reading method according to the shape of the correction range.

According to the seventh aspect, the correction range according to the reading method is compared with the case where the image showing the correction range obtained from the first read data and the image showing the correction range obtained from the second read data are not displayed side by side. It has the effect of being able to see the difference at a glance.

According to the eighth aspect, there is an effect that it is possible to prevent extra correction from being performed as compared with the case where the correction range obtained from the second read data is not the correction target.

According to the ninth aspect, there is an effect that an appropriate correction can be performed on the binding portion as compared with the case where only the correction range obtained from the multi-gradation data is considered.

According to the tenth aspect, there is an effect that an appropriate reading method, a correction range, and a shadow shape can be estimated as compared with the case where the learning data is not used.

According to the eleventh aspect, the image showing the correction range obtained from the learning reading data is associated with the reading method for reducing the correction range and the image showing the correction range and the shape of the shadow when the reading method is performed. It has the effect that appropriate learning data can be obtained as compared with the case without it.

It is a block diagram which shows an example of the electric structure of the image forming apparatus which concerns on 1st Embodiment. It is a block diagram which shows an example of the functional structure of the image forming apparatus which concerns on 1st Embodiment. It is a flowchart which shows an example of the processing flow of the correction processing program which concerns on 1st Embodiment. It is a figure which shows an example of the correction range of the binding part which concerns on 1st Embodiment. It is a front view which shows an example of the estimation result screen which concerns on 1st Embodiment. It is a front view which shows an example of the correction range detection result screen which concerns on 1st Embodiment. (A) is a side view which shows an example of the 1st reading method by the image forming apparatus which concerns on 1st Embodiment. (B) is a side view showing an example of a second reading method by the image forming apparatus according to the first embodiment. (A) is a figure which shows an example of the 1st image obtained by the 1st reading method which concerns on 1st Embodiment. (B) is a figure which shows an example of the 2nd image obtained by the 2nd reading method which concerns on 1st Embodiment. It is a figure which provides the explanation of the learning example of the background color of the binding part which concerns on 1st Embodiment. It is a front view which shows another example of the estimation result screen which concerns on 1st Embodiment. It is a block diagram which shows an example of the functional structure of the image forming apparatus which concerns on 2nd Embodiment. It is a flowchart which shows an example of the processing flow of the correction processing program which concerns on 2nd Embodiment. It is a figure which shows an example of the correction range of the binding part and the shape of the shadow of the binding part which concerns on 2nd Embodiment. It is a front view which shows an example of the estimation result screen which concerns on 2nd Embodiment. It is a front view which shows an example of the correction range detection result screen which concerns on 2nd Embodiment. (A) is a figure which shows an example of the 1st image obtained by the 1st reading method which concerns on 2nd Embodiment. (B) is a diagram showing an example of a second image obtained by the second reading method according to the second embodiment.

Hereinafter, an example of a mode for carrying out the present invention will be described in detail with reference to the drawings.

[First Embodiment]
FIG. 1 is a block diagram showing an example of an electrical configuration of the image forming apparatus 10A according to the first embodiment.
As shown in FIG. 1, the image forming apparatus 10A according to the present embodiment includes a control unit 12, a storage unit 14, a display unit 16, an operation unit 18, an image forming unit 20, a document reading unit 22, and a document reading unit 22. It includes a communication unit 24.

The control unit 12 includes a CPU (Central Processing Unit) 12A, a ROM (Read Only Memory) 12B, a RAM (Random Access Memory) 12C, and an input / output interface (I / O) 12D, and each of these units is via a bus. Are connected to each other.

Each functional unit including a storage unit 14, a display unit 16, an operation unit 18, an image forming unit 20, a document reading unit 22, and a communication unit 24 is connected to the I / O 12D. Each of these functional units can communicate with the CPU 12A via the I / O 12D.

The control unit 12 may be configured as a sub control unit that controls a part of the operation of the image forming apparatus 10A, or may be configured as a part of a main control unit that controls the entire operation of the image forming apparatus 10A. Good. For example, an integrated circuit such as an LSI (Large Scale Integration) or an IC (Integrated Circuit) chipset is used for a part or all of each block of the control unit 12. An individual circuit may be used for each of the above blocks, or a circuit in which a part or all of them are integrated may be used. Each of the above blocks may be provided integrally, or some blocks may be provided separately. In addition, a part of each of the above blocks may be provided separately. The integration of the control unit 12 is not limited to the LSI, and a dedicated circuit or a general-purpose processor may be used.

As the storage unit 14, for example, an HDD (Hard Disk Drive), an SSD (Solid State Drive), a flash memory, or the like is used. The correction processing program 14A according to the present embodiment is stored in the storage unit 14. The correction processing program 14A may be stored in the ROM 12B. Further, the learning data 14B, which will be described later, is stored in the storage unit 14. The learning data 14B may also be stored in the ROM 12B.

The correction processing program 14A may be pre-installed in, for example, the image forming apparatus 10A. The correction processing program 14A may be realized by storing it in a non-volatile storage medium or distributing it via a network and appropriately installing it in the image forming apparatus 10A. Examples of non-volatile storage media include CD-ROMs (Compact Disc Read Only Memory), optical magnetic disks, HDDs, DVD-ROMs (Digital Versatile Disc Read Only Memory), flash memories, memory cards, and the like. To.

For the display unit 16, for example, a liquid crystal display (LCD), an organic EL (Electro Luminescence) display, or the like is used. The display unit 16 integrally has a touch panel. The operation unit 18 is provided with various operation keys such as a numeric keypad and a start key. The display unit 16 and the operation unit 18 receive various instructions from the user of the image forming apparatus 10A. These various instructions include, for example, an instruction to start reading the original, an instruction to start copying the original, and the like. The display unit 16 displays various information such as the result of the process executed in response to the instruction received from the user and the notification for the process.

The document reading unit 22 captures the documents placed on the paper feed tray of the automatic document feeder (not shown) provided above the image forming apparatus 10A one by one, and optically reads the captured documents to obtain image information. .. Alternatively, the document reading unit 22 optically reads the document placed on the platen such as platen glass to obtain image information.

The image forming unit 20 obtains image information obtained by reading by the document reading unit 22 or an image based on image information obtained from an external personal computer (PC) or the like connected via a network. Form on a recording medium such as paper. In the present embodiment, the electrophotographic method will be described as an example of the method for forming an image, but another method such as an inkjet method may be adopted.

When the method of forming an image is an electrophotographic method, the image forming unit 20 includes a photoconductor drum, a charging unit, an exposure unit, a developing unit, a transfer unit, and a fixing unit. The charging unit applies a voltage to the photoconductor drum to charge the surface of the photoconductor drum. The exposed unit forms an electrostatic latent image on the photoconductor drum by exposing the photoconductor drum charged by the charged unit with light corresponding to the image information. The developing unit develops an electrostatic latent image formed on the photoconductor drum with toner to form a toner image on the photoconductor drum. The transfer unit transfers the toner image formed on the photoconductor drum to the recording medium. The fixing unit fixes the toner image transferred to the recording medium by heating and pressurizing.

The communication unit 24 is connected to the network, and can communicate with an external PC via the network. The communication unit 24 and the network may be connected by wire or wirelessly. As an example, the Internet, LAN (Local Area Network), WAN (Wide Area Network), and the like are applied to the network.

By the way, as described above, when a double-page spread including a binding portion of a book is placed on a platen and distortion generated in the binding portion when reading the double-page spread is corrected, a reading method for reducing the correction range is considered. It has not been. In this case, there is a possibility that the image quality may be deteriorated due to extra correction being performed on a portion that should not be corrected.

Therefore, the CPU 12A of the image forming apparatus 10A according to the present embodiment functions as each unit shown in FIG. 2 by writing the correction processing program 14A stored in the storage unit 14 to the RAM 12C and executing the program.

FIG. 2 is a block diagram showing an example of a functional configuration of the image forming apparatus 10A according to the first embodiment.
As shown in FIG. 2, the CPU 12A of the image forming apparatus 10A according to the present embodiment functions as a conversion unit 30, a detection unit 32, an estimation unit 34, a display control unit 36, and a correction unit 38. The display control unit 36 is an example of a control unit. Further, as described above, the learning data 14B is stored in the storage unit 14.

The conversion unit 30 according to the present embodiment converts the first read data obtained by reading the spread page including the binding part of the book into multi-gradation data. The term "book" as used herein includes books, magazines, etc., and is in a state where one end of a plurality of sheets of paper is bound. Further, the multi-gradation data is, for example, data represented by gray scale or the like. A known method is used for the conversion to the multi-gradation data.

As an example, the detection unit 32 according to the present embodiment detects the correction range of the binding unit from the multi-gradation data obtained by conversion by the conversion unit 30 as shown in FIG. 4 described later.

The estimation unit 34 according to the present embodiment sets a reading method for reducing the correction range of the binding portion from the correction range of the binding portion detected by the detection unit 32, and a correction range of the binding portion when the reading method is implemented. presume. These reading methods and estimation of the correction range are performed using the learning data 14B obtained by machine learning as an example. The estimation unit 34 estimates different reading methods according to the shape of the correction range of the binding unit detected by the detection unit 32. As an example, this reading method is a method using a jig that presses a book from above, as shown in FIG. 5 described later, and the installation position of the jig is set according to the shape of the correction range of the binding portion. It's a different method.

As an example, the display control unit 36 according to the present embodiment controls the display unit 16 to display an image showing the correction range of the binding unit estimated by the estimation unit 34 together with the reading method, as shown in FIG. 5 described later. I do.

Here, when the user rereads the spread page using the reading method displayed on the display unit 16, the conversion unit 30 reads the spread page again and obtains the second read data in multiple gradations. Convert to data.

In this case, the detection unit 32 detects the correction range of the binding unit from the multi-gradation data obtained by converting the second read data. As an example, the display control unit 36 has an image showing a correction range of the binding portion obtained from the first read data and an image showing a correction range of the binding portion obtained from the second read data, as shown in FIG. 6 described later. Is controlled to be displayed on the display unit 16 side by side.

The correction unit 38 according to the present embodiment corrects the second read data with the correction range of the binding portion obtained from the second read data as the correction target, and outputs the image corresponding to the corrected second read data to the subsequent stage. To do.

Next, the operation of the image forming apparatus 10A according to the first embodiment will be described with reference to FIG. Note that FIG. 3 is a flowchart showing an example of the processing flow of the correction processing program 14A according to the first embodiment.

First, for the image forming apparatus 10A, as an example, when a double-page spread including a binding portion of a book is placed on a platen and an instruction to start execution of a book scanning mode is given, a correction is made. The processing program 14A is started, and each of the following steps is executed.

In step 100 of FIG. 3, the conversion unit 30 acquires the first reading data by reading the spread page by the document reading unit 22.

In step 102, the conversion unit 30 converts the first read data acquired in step 100 into multi-gradation data.

In step 104, as an example, the detection unit 32 detects the correction range of the binding unit from the multi-gradation data converted in step 102, as shown in FIG.

FIG. 4 is a diagram showing an example of the correction range of the binding portion according to the first embodiment.
As shown in FIG. 4, in the multi-gradation data, the range surrounded by the rectangle is detected as the correction range of the binding portion.

For example, when a double-page spread is placed on a platen, the binding portion is lifted. The floating of the binding portion is largest near the center portion in the width direction of the spread page, gradually decreases from there to both the left and right sides, and the page surface comes into contact with the platen and disappears. Due to this floating, a corresponding band-shaped portion appears in the multi-gradation data. In this band-shaped portion, the concentration near the central portion in the width direction becomes the highest (higher), and the density gradually becomes lighter (lower) toward both the left and right sides. When the background color of the double-page spread is white, the strip-shaped portion also becomes white and disappears at the position where the page surface contacts the platen.

Therefore, the detection unit 32 detects the band-shaped portion in the multi-gradation data as the correction range of the binding portion. Specifically, in the multi-gradation data, a region where the density is higher than the reference density is detected as the correction range of the binding portion.

In step 106, the estimation unit 34 estimates from the correction range of the binding portion detected in step 104, a reading method for reducing the correction range of the binding portion and a correction range of the binding portion when the reading method is performed. Specifically, as described above, the learning data (that is, the learning model) 14B obtained by machine learning is used. As an example, the learning data 14B is learning reading data obtained by reading a spread page including a binding portion of one or more books in a predetermined state (for example, a state in which a certain part is floating) (hereinafter, "" It is the data learned by using the "read image for learning"). Specifically, in the image showing the correction range of the binding portion detected from the read image for learning, a reading method for reducing the correction range of the binding portion and a correction range of the binding portion when the reading method is performed are added. It is the data learned by associating the shown images. A specific method for generating the learning data 14B will be described later.

In step 108, as an example, the display control unit 36 controls the display unit 16 to display an image showing the correction range of the binding unit estimated in step 106 on the display unit 16 together with the reading method, as shown in FIG.

FIG. 5 is a front view showing an example of the estimation result screen 16A according to the first embodiment.
The estimation result screen 16A shown in FIG. 5 is displayed on the display unit 16.

The estimation result screen 16A shown in FIG. 5 shows an image 56A showing the correction range of the binding portion detected in step 104, an explanatory diagram of the reading method estimated in step 106, and correction of the binding portion when the reading method is performed. Image 56B showing the range is included.

Image 56A is an image showing the correction range R1 of the binding portion obtained from the first read data. The shape of the correction range R1 indicates that the front side (lower side in the figure) of the book is raised. In this case, using the learning data 14B, a reading method for reducing the correction range according to the shape of the correction range R1 is estimated. Specifically, since the front side of the book is floating, a reading method in which the front side of the book is pressed while reading is presumed. In this case, on the estimation result screen 16A, as an example, an explanatory diagram of a reading method in which reading is performed while pressing the front side of the book using a jig is displayed. According to this explanatory view, the front side of the book 56 placed on the platen 52 is pressed by the jig 50A, and the upper side of the jig 50A is pressed by the platen cover 54. In addition, on the estimation result screen 16A, along with this explanatory diagram, as an example, "The front is floating. The correction range is reduced by pressing the front side of the book with a jig and reading as shown in the figure below. The explanation "I will do it." Is also displayed. The image 56B is an image showing the correction range R2 of the binding portion when the reading method displayed on the estimation result screen 16A is performed. It can be seen that the correction range R2 of the image 56B is smaller than the correction range R1 of the image 56A. Further, on the estimation result screen 16A, a "Yes" button and a "No" button are displayed so that they can be pressed, along with a message "Do you want to read again?".

In step 110, as an example, the display control unit 36 determines whether or not to perform re-reading by the reading method displayed on the estimation result screen 16A shown in FIG. 5 described above. If it is determined that rereading is to be performed (in the case of affirmative determination), that is, if it is determined that the "Yes" button has been pressed on the estimation result screen 16A, the process proceeds to step 112. On the other hand, if it is determined that rereading is not performed (in the case of a negative determination), that is, if it is determined that the "No" button has been pressed on the estimation result screen 16A, the process proceeds to step 126.

In step 112, as an example, the conversion unit 30 acquires the second reading data by re-reading the spread page by the document reading unit 22 by using the reading method displayed on the estimation result screen 16A shown in FIG.

In step 114, the conversion unit 30 converts the second read data acquired in step 112 into multi-gradation data.

In step 116, the detection unit 32 detects the correction range of the binding unit from the multi-gradation data converted in step 114.

In step 118, as an example, as shown in FIG. 6, the display control unit 36 has an image (previous) showing a correction range of the binding portion obtained from the first read data and a binding portion obtained from the second read data. Control is performed so that the image (this time) showing the correction range is displayed side by side on the display unit 16.

FIG. 6 is a front view showing an example of the correction range detection result screen 16B according to the first embodiment.
The correction range detection result screen 16B shown in FIG. 6 is displayed on the display unit 16.

On the correction range detection result screen 16B shown in FIG. 6, an image 56A (previous) showing the correction range R1 of the binding portion detected in step 104 and an image 56C showing the correction range R3 of the binding portion detected in step 116 ( This time) is included. Further, on the correction range detection result screen 16B, a "Yes" button and a "No" button are displayed so that they can be pressed, along with a message "Do you want to read again?".

In step 120, as an example, the display control unit 36 determines whether or not rereading is unnecessary for the correction range R3 displayed on the correction range detection result screen 16B shown in FIG. 6 above. If it is determined that rereading is unnecessary (in the case of affirmative determination), that is, if it is determined that the "No" button has been pressed on the correction range detection result screen 16B, the process proceeds to step 122. On the other hand, when it is determined that rereading is necessary (in the case of a negative determination), that is, when it is determined that the "Yes" button has been pressed on the correction range detection result screen 16B, the process returns to step 112 and the process is repeated.

In step 122, as an example, the correction unit 38 corrects the correction range R3 displayed on the correction range detection result screen 16B shown in FIG. 6, that is, the correction range R3 of the binding portion obtained from the second read data. As a target, the second read data is corrected.

In step 124, the correction unit 38 outputs an image corresponding to the corrected second read data to the subsequent stage, and ends a series of processing by the correction processing program 14A. The output destination may be, for example, the image forming unit 20 or the storage unit 14.

On the other hand, in step 126, as an example, the correction unit 38 corrects the correction range R1 displayed on the estimation result screen 16A shown in FIG. 5, that is, the correction range R1 of the binding portion obtained from the first read data. As a target, the first read data is corrected.

In step 128, the correction unit 38 outputs an image corresponding to the corrected first read data to the subsequent stage, and ends a series of processing by the correction processing program 14A.

Next, with reference to FIGS. 7 (A), 7 (B), 8 (A), and 8 (B), a learning example of a reading method for reducing the correction range of the binding portion of the book will be described. .. Here, a normal reading method that does not use the jig 50A will be referred to as a first reading method, and a reading method that uses the jig 50A to reduce the correction range will be referred to as a second reading method.

FIG. 7A is a side view showing an example of the first reading method by the image forming apparatus 10A according to the first embodiment.
In the first scanning method shown in FIG. 7A, the back side of the book 58 placed on the platen 52 is pressed by the platen cover 54, and the front side of the book 58 is floating. Book 58 is a learning book. The right side of the book 58 (the side far from the opening / closing mechanism (not shown) of the platen cover 54) is the front side, and the left side of the book 58 (the side close to the opening / closing mechanism of the platen cover 54) is the back side.

FIG. 7B is a side view showing an example of the second reading method by the image forming apparatus 10A according to the first embodiment.
In the second scanning method shown in FIG. 7B, the front side of the book 58 placed on the platen 52 is pressed by the jig 50A, and the upper side of the jig 50A is pressed by the platen cover 54. Indicates the state.

FIG. 8A is a diagram showing an example of the first image 58A obtained by the first reading method according to the first embodiment.
The first image 58A shown in FIG. 8A is an image showing the correction range R11 of the binding portion of the book 58. The correction range R11 is detected from the learning scanned image obtained by scanning the spread page of the book 58 by the first scanning method.

FIG. 8B is a diagram showing an example of the second image 58B obtained by the second reading method according to the first embodiment.
The second image 58B shown in FIG. 8B is an image showing the correction range R12 of the binding portion of the book 58. It can be seen that the correction range R12 of the second image 58B is smaller than the correction range R11 of the first image 58A.

When learning the reading method, reading is performed using the above two types of first reading method and second reading method for each spread page of the book 58, and the first image 58A and the second spread page of the book 58 are read. Acquire a pair of images 58B. At this time, from the viewpoint of estimation accuracy, it is desirable to change the reading direction (for example, vertical direction, horizontal direction, etc.) of the book 58 and acquire a pair of the first image 58A and the second image 58B for each direction. Further, the floating part of the book 58 (for example, the front side part, the right side part, the left side part, the back side part, etc.) is changed, and a pair of the first image 58A and the second image 58B is acquired for each part. Is desirable.

Then, the position of the correction range R11 of the first image 58A and the position of the correction range R12 of the second image 58B are designated, and the second image 58B and the second reading method are associated with the first image 58A. The associated first image 58A, second image 58B, and second reading method are machine-learned as learning data to generate the learning data 14B shown in FIG. 2 above. This machine learning method may be any method used for image recognition and is not particularly limited, but a neural network or the like is applied as an example.

By rotating the first image 58A within a certain angle range (for example, 5 degrees or more and 60 degrees or less) and performing machine learning in association with the rotated angles, it is appropriate even when the book is placed at an angle. It is desirable that the second image and the second reading method be estimated.

Further, when enlarging or correcting the density of the correction range R12 of the second image 58B, it is desirable to perform machine learning by associating the correction factor and the correction position with the pair of the first image 58A and the second image 58B. ..

Further, the learning data 14B may be associated with an image showing the background portion of the spread page detected from the scanned image of the book 58. Regarding the background portion of the book 58, the background portion differs depending on the type of book. In this case, as an example, as shown in FIG. 9, various types of books with different layouts (specifically, dictionaries, magazines, textbooks, etc.) are machine-learned about the background color of the binding part at the time of reading. Estimate the background color of the binding part and use it to correct the binding part.

FIG. 9 is a diagram provided for explaining a learning example of the background color of the binding portion according to the first embodiment.

(S1) shows a learning scanned image 60 obtained by scanning a spread page of a book by the first scanning method. The learning scanned image 60 includes a binding portion 60A.

In (S2), the read image 60 for learning is divided perpendicularly to the binding portion 60A, and a plurality of rows L1, L2, ... Are acquired.

In (S3), for the line L1, the background portion JL1 and the background correction range HL1 on the left page are specified, and the background portion JR1 and the background correction range HR1 on the right page are specified, and the background portion and the background correction on each of the left and right pages are specified. Machine learn the range. Similarly, for the line L2, the background portion JL2 and the background correction range HL2 on the left page are specified, and the background portion JR2 and the background correction range HR2 on the right page are specified, and the background portion and the background correction on each of the left and right pages are specified. Machine learn the range. For the other lines, the background part and the background correction range of each of the left and right pages are machine-learned in the same manner.

Here, in the above, a jig is used as a reading method for reducing the correction range of the binding portion, but as another example, both hands of the user may be used as shown in FIG.

FIG. 10 is a front view showing another example of the estimation result screen 16A according to the first embodiment.
The estimation result screen 16A shown in FIG. 10 is displayed on the display unit 16 in step 108 shown in FIG. 3 described above.

The estimation result screen 16A shown in FIG. 10 shows an image 56A showing the correction range of the binding portion detected in step 104, an explanatory diagram of the reading method estimated in step 106, and the same as the example shown in FIG. Image 56B showing the correction range of the binding portion when the reading method is carried out is included.

Image 56A is an image showing the correction range R1 of the binding portion obtained from the first read data, as in the example shown in FIG. On the estimation result screen 16A shown in FIG. 10, an explanatory diagram of a reading method for reading while pressing the front side of the book with both hands is displayed instead of the jig. According to this explanatory view, a state in which the front side of the book 56 placed on the platen 52 is pressed by both hands 50B is shown. Further, on the estimation result screen 16A shown in FIG. 10, along with this explanatory diagram, as an example, "the front is floating. Correction is made by pressing the front side of the book with both hands as shown in the figure below to read. The description is also displayed. Image 56B is an image showing the correction range R2 of the binding portion when the reading method displayed on the estimation result screen 16A is performed, as in the example shown in FIG. It can be seen that the correction range R2 of the image 56B is smaller than the correction range R1 of the image 56A. Further, on the estimation result screen 16A, a "Yes" button and a "No" button are displayed so that they can be pressed, along with a message "Do you want to read again?".

In the case of using both hands, as in the case of using the jig, for each spread page of the book 58, as shown in FIGS. 8 (A) and 8 (B) above, the first image 58A and the second image 58A and the second Learning data 14B is generated by acquiring a pair of images 58B and performing machine learning.

Further, when the book is pressed with both hands of the user, the user may not be able to operate the start button at the time of re-reading because both hands are occupied. Therefore, it is desirable that the image forming apparatus 10A has a timer function that automatically starts re-reading. For example, after the "Yes" button shown in FIG. 10 is pressed, it is set to automatically start rereading after a certain period of time.

As described above, according to the present embodiment, a reading method for reducing the correction range of the binding portion and correction of the binding portion when the reading method is performed using the multi-gradation data obtained from the first reading data of the book. An image showing the range is displayed. By implementing the scanning method displayed by the user, a scanned image in which the correction range of the binding portion is reduced is acquired, and it is possible to prevent unnecessary correction from being performed on the scanned image.

[Second Embodiment]
In the first embodiment, a reading method for reducing the correction range of the binding portion using multi-gradation data and a mode for displaying an image showing the correction range of the binding portion when the reading method is performed have been described. In the present embodiment, a reading method for reducing the correction range of the binding portion using multi-gradation data and binary data, and the correction range of the binding portion and the shape of the shadow of the binding portion when the reading method is performed are shown. A form for displaying an image will be described.

FIG. 11 is a block diagram showing an example of a functional configuration of the image forming apparatus 10B according to the second embodiment.
As shown in FIG. 11, the CPU 12A of the image forming apparatus 10B according to the present embodiment functions as a conversion unit 40, a detection unit 42, an estimation unit 44, a display control unit 46, and a correction unit 48. The display control unit 46 is an example of a control unit. Further, the learning data 14C is stored in the storage unit 14.

The conversion unit 40 according to the present embodiment converts the first reading data obtained by reading the spread page including the binding part of the book into multi-gradation data and binary data. The binary data is data represented in black and white. A known method is used for the conversion to the binary data.

As an example, the detection unit 42 according to the present embodiment detects the correction range of the binding unit from the multi-gradation data converted by the conversion unit 40 and obtained from the binary data, as shown in FIG. 13 described later. Detects the shape of the shadow of the binding part.

When the estimation unit 44 according to the present embodiment implements a reading method for reducing the correction range of the binding portion from the correction range of the binding portion and the shape of the shadow of the binding portion detected by the detection unit 42, and the reading method. The correction range of the binding part and the shape of the shadow of the binding part are estimated. These reading methods, correction ranges, and shadow shape estimates are performed using, for example, learning data 14C obtained by machine learning. The estimation unit 44 estimates different reading methods according to the shape of the correction range of the binding portion detected by the detection unit 42 and the shape of the shadow of the binding portion. As an example, this reading method is a method using a jig that presses the book from above, as shown in FIG. 14 described later, and also depends on the shape of the correction range of the binding portion and the shape of the shadow of the binding portion. This is a method in which the jigs are installed in different positions.

As an example, the display control unit 46 according to the present embodiment reads an image showing the correction range of the binding portion estimated by the estimation unit 44 and the shape of the shadow of the binding portion together with the reading method, as shown in FIG. 14 described later. Control is performed so that the display unit 16 displays the image.

Here, when the user re-reads the spread page using the reading method displayed on the display unit 16, the conversion unit 40 re-reads the spread page and multi-gradations the second read data obtained. Convert to data and binary data.

In this case, the detection unit 42 detects the correction range of the binding portion from the multi-gradation data obtained by converting the second read data, and detects the shape of the shadow of the binding portion from the binary data. As an example, the display control unit 46 has, as shown in FIG. 15 described later, an image showing the correction range of the binding portion obtained from the first reading data and the shape of the shadow of the binding portion, and the binding obtained from the second reading data. Control is performed so that the correction range of the portion and the image showing the shape of the shadow of the binding portion are displayed side by side on the display unit 16.

The correction unit 48 according to the present embodiment corrects the second reading data with the correction range of the binding portion obtained from the second reading data and the shape of the shadow of the binding portion as the correction target, and uses the corrected second reading data as the correction target. The corresponding image is output to the latter stage.

Next, the operation of the image forming apparatus 10B according to the second embodiment will be described with reference to FIG. Note that FIG. 12 is a flowchart showing an example of the processing flow of the correction processing program 14A according to the second embodiment.

First, for the image forming apparatus 10B, as an example, when a double-page spread including a binding portion of a book is placed on a platen and an instruction is given to start execution of a book scanning mode, a correction is made. The processing program 14A is started, and each of the following steps is executed.

In step 130 of FIG. 12, the conversion unit 40 acquires the first reading data by reading the spread page by the document reading unit 22.

In step 132, the conversion unit 40 converts the first read data acquired in step 130 into multi-gradation data and binary data.

In step 134, as an example, as shown in FIG. 13, the detection unit 42 detects the correction range of the binding portion from the multi-gradation data converted in step 132, and detects the shape of the shadow of the binding portion from the binary data. To do.

FIG. 13 is a diagram showing an example of the correction range of the binding portion and the shape of the shadow of the binding portion according to the second embodiment.
As shown in FIG. 13, in the binary data, the range surrounded by the black rectangle is detected as the shadow shape of the binding portion, and in the multi-gradation data, the range surrounded by the rectangle is detected as the correction range of the binding portion. ..

In step 136, a reading method in which the estimation unit 44 reduces the correction range of the binding portion from the correction range of the binding portion and the shape of the shadow of the binding portion detected in step 134, and the binding portion when the reading method is performed. Estimate the correction range and the shape of the shadow at the binding part. Specifically, as described above, the learning data (that is, the learning model) 14C obtained by machine learning is used. As an example, the learning data 14C is learned by using a reading image obtained by reading a spread page including a binding portion of one or more books in a predetermined state (for example, a state in which a certain part is floating). It is the data that has been created. Specifically, in the image showing the correction range of the binding portion detected from the read image for learning, a reading method for reducing the correction range of the binding portion, and the correction range of the binding portion when the reading method is performed, and This is data learned by associating an image showing the shape of the shadow of the binding portion.

In step 138, as an example, as shown in FIG. 14, the display control unit 36 displays an image showing the correction range of the binding portion estimated in step 136 and the shape of the shadow of the binding portion on the display unit 16 together with the reading method. Control to make it.

FIG. 14 is a front view showing an example of the estimation result screen 16C according to the second embodiment.
The estimation result screen 16C shown in FIG. 14 is displayed on the display unit 16.

On the estimation result screen 16C shown in FIG. 14, an image 56D showing the correction range of the binding portion detected in step 134 and the shape of the shadow of the binding portion, an explanatory diagram of the reading method estimated in step 136, and the reading method are carried out. An image 56E showing the correction range of the binding portion and the shape of the shadow of the binding portion in the case of the binding portion is included.

The image 56D is an image showing the correction range R21 of the binding portion and the shadow D21 of the binding portion obtained from the first read data. Each shape of the correction range R21 and the shadow D21 indicates that the front side (lower side in the figure) of the book is raised. In this case, using the learning data 14C, a reading method for reducing the correction range according to the shapes of the correction range R21 and the shadow D21 is estimated. Specifically, since the front side of the book is floating, a reading method in which the front side of the book is pressed while reading is presumed. In this case, on the estimation result screen 16C, as an example, an explanatory diagram of a reading method in which reading is performed while pressing the front side of the book using a jig is displayed. According to this explanatory view, the front side of the book 56 placed on the platen 52 is pressed by the jig 50A, and the upper side of the jig 50A is pressed by the platen cover 54. In addition, on the estimation result screen 16C, along with this explanatory diagram, as an example, "The front is floating. The correction range is reduced by pressing the front side of the book with a jig and reading as shown in the figure below. The explanation "I will do it." Is also displayed. The image 56E is an image showing the correction range R22 of the binding portion and the shadow D22 of the binding portion when the reading method displayed on the estimation result screen 16C is performed. It can be seen that the correction range R22 and the shadow D22 of the image 56E are smaller than the correction range R21 and the shadow D21 of the image 56D. Further, on the estimation result screen 16C, a "Yes" button and a "No" button are displayed so that they can be pressed, along with a message "Do you want to read again?".

In step 140, as an example, the display control unit 46 determines whether or not to perform re-reading by the reading method displayed on the estimation result screen 16C shown in FIG. 14 described above. If it is determined that rereading is to be performed (in the case of affirmative determination), that is, if it is determined that the "Yes" button has been pressed on the estimation result screen 16C, the process proceeds to step 142. On the other hand, if it is determined that rereading is not performed (in the case of a negative determination), that is, if it is determined that the "No" button has been pressed on the estimation result screen 16C, the process proceeds to step 156.

In step 142, as an example, the conversion unit 40 acquires the second reading data by re-reading the spread page by the document reading unit 22 by using the reading method displayed on the estimation result screen 16C shown in FIG.

In step 144, the conversion unit 40 converts the second read data acquired in step 142 into multi-gradation data and binary data.

In step 146, the detection unit 42 detects the correction range of the binding portion from the multi-gradation data converted in step 144, and detects the shape of the shadow of the binding portion from the binary data.

In step 148, as an example, as shown in FIG. 15, the display control unit 46 obtains an image (previous) showing the correction range and shadow shape of the binding portion obtained from the first read data and the second read data. Control is performed so that the correction range of the binding portion and the image (this time) showing the shape of the shadow are displayed side by side on the display unit 16.

FIG. 15 is a front view showing an example of the correction range detection result screen 16D according to the second embodiment.
The correction range detection result screen 16D shown in FIG. 15 is displayed on the display unit 16.

On the correction range detection result screen 16D shown in FIG. 15, an image 56D (previous) showing the correction range R21 and the shadow D21 of the binding portion detected in step 134, and the correction range R23 and the shadow of the binding portion detected in step 146 are displayed. Image 56F (this time) showing D23 is included. Further, on the correction range detection result screen 16D, a "Yes" button and a "No" button are displayed so that they can be pressed, along with a message "Do you want to read again?".

In step 150, as an example, the display control unit 46 determines whether or not rereading is necessary for the correction range R23 and the shadow D23 displayed on the correction range detection result screen 16D shown in FIG. 15 described above. If it is determined that rereading is unnecessary (in the case of affirmative determination), that is, if it is determined that the "No" button has been pressed on the correction range detection result screen 16D, the process proceeds to step 152. On the other hand, when it is determined that re-reading is necessary (in the case of a negative determination), that is, when it is determined that the "Yes" button has been pressed on the correction range detection result screen 16D, the process returns to step 142 and the process is repeated.

In step 152, as an example, the correction unit 48 corrects the correction range R23 and the shadow D23 displayed on the correction range detection result screen 16D shown in FIG. 15, that is, the correction range of the binding portion obtained from the second read data. The second read data is corrected with R23 and shadow D23 as correction targets.

In step 154, the correction unit 48 outputs the image corresponding to the corrected second read data to the subsequent stage, and ends a series of processing by the correction processing program 14A. The output destination may be, for example, the image forming unit 20 or the storage unit 14.

On the other hand, in step 156, as an example, the correction unit 48 corrects the correction range R21 and the shadow D21 displayed on the estimation result screen 16C shown in FIG. 14, that is, the correction range of the binding portion obtained from the first read data. The first read data is corrected with R21 and shadow D21 as correction targets.

In step 158, the correction unit 48 outputs the image corresponding to the corrected first read data to the subsequent stage, and ends a series of processing by the correction processing program 14A.

Next, a learning example of a reading method for reducing the correction range of the binding portion of the book will be described with reference to FIGS. 16A and 16B. Similar to the first embodiment, a normal reading method that does not use the jig 50A is referred to as a first reading method (see FIG. 7A), and a reading method that uses the jig 50A to reduce the correction range. Will be referred to as a second reading method (see FIG. 7B).

FIG. 16A is a diagram showing an example of the first image 58C obtained by the first reading method according to the second embodiment.
The first image 58C shown in FIG. 16A is an image showing the correction range R31 and the shadow D31 of the binding portion of the book 58. The correction range R31 and the shadow D31 are detected from the learning read image obtained by reading the spread page of the book 58 by the first reading method.

FIG. 16B is a diagram showing an example of the second image 58D obtained by the second reading method according to the second embodiment.
The second image 58D shown in FIG. 16B is an image showing the correction range R32 and the shadow D32 of the binding portion of the book 58. It can be seen that the correction range R32 and the shadow D32 of the second image 58D are smaller than the correction range R31 and the shadow D32 of the first image 58C.

Similar to the first embodiment, when learning the reading method, reading is performed for each spread page of the book 58 using the above two types of the first reading method and the second reading method, and each spread page of the book 58 is read. The pair of the first image 58C and the second image 58D is acquired. At this time, from the viewpoint of estimation accuracy, it is desirable to change the reading direction (for example, vertical direction, horizontal direction, etc.) of the book 58 and acquire a pair of the first image 58C and the second image 58D for each direction. Further, the floating part of the book 58 (for example, the front side part, the right side part, the left side part, the back side part, etc.) is changed, and a pair of the first image 58C and the second image 58D is acquired for each part. Is desirable.

Then, the position of the correction range R31 of the first image 58C and the position of the correction range R32 of the second image 58D are designated, and the second image 58D and the second reading method are associated with the first image 58C. The associated first image 58C, second image 58D, and second reading method are machine-learned as learning data to generate the learning data 14C shown in FIG. 11 above.

As described above, according to the present embodiment, the reading method for reducing the correction range of the binding portion and the reading method are carried out using the multi-gradation data and the binary data obtained from the first reading data of the book. An image showing the correction range of the binding portion and the shape of the shadow of the binding portion in the above is displayed. By implementing the scanning method displayed by the user, a scanned image in which the correction range of the binding portion and the shape of the shadow are reduced is acquired, and it is possible to prevent unnecessary correction from being performed on the scanned image.

The image forming apparatus has been described above as an example of the embodiment. The embodiment may be in the form of a program for causing a computer to execute the functions of each part included in the image forming apparatus. The embodiment may be in the form of a storage medium that can be read by a computer that stores this program.

In addition, the configuration of the image forming apparatus described in the above embodiment is an example, and may be changed depending on the situation within a range that does not deviate from the gist.

Further, the processing flow of the program described in the above embodiment is also an example, and even if unnecessary steps are deleted, new steps are added, or the processing order is changed within a range that does not deviate from the purpose. Good.

Further, in the above-described embodiment, the case where the processing according to the embodiment is realized by the software configuration by using the computer by executing the program has been described, but the present invention is not limited to this. The embodiment may be realized by, for example, a hardware configuration or a combination of a hardware configuration and a software configuration.

10A, 10B Image forming device 12 Control unit 12A CPU
12B ROM
12C RAM
12D I / O
14 Storage unit 14A Correction processing program 14B, 14C Learning data 16 Display unit 16A, 16C Estimation result screen 16B, 16D Correction range detection result screen 18 Operation unit 20 Image formation unit 22 Document reading unit 24 Communication unit 30, 40 Conversion unit 32, 42 Detection unit 34, 44 Estimating unit 36, 46 Display control unit 38, 48 Correction unit

Claims (12)

A conversion unit that converts the first reading data obtained by reading the spread page including the binding part of the book into multi-gradation data, and a conversion unit.
A detection unit that detects the correction range of the binding unit from the multi-gradation data,
A reading method for reducing the correction range of the binding portion from the correction range of the binding portion detected by the detection unit, and an estimation unit for estimating the correction range of the binding portion when the reading method is performed.
A control unit that controls to display an image showing the correction range of the binding unit estimated by the estimation unit together with the reading method.
An image forming apparatus equipped with. The image forming apparatus according to claim 1, wherein the estimation unit estimates the reading method and the correction range of the binding unit when the reading method is performed by using the learning data obtained by machine learning. The learning data is an image showing the correction range of the binding portion detected from the learning reading data obtained by reading the spread page including the binding portion of the book in a predetermined state, and the correction range of the binding portion is added to the image. The image forming apparatus according to claim 2, which is data learned by associating an image showing a correction range of the binding portion when the reading method is reduced and the reading method is performed. The image forming apparatus according to claim 3, wherein the learning data is further associated with an image showing a background portion of the spread page detected from the reading data for learning. The image forming apparatus according to any one of claims 1 to 4, wherein the estimation unit estimates a different reading method according to the shape of the correction range of the binding unit detected by the detection unit. The method according to claim 5, wherein the reading method is a method using a jig for pressing the book from above, and the installation position of the jig is different depending on the shape of the correction range of the binding portion. Image forming device. The conversion unit converts the second reading data obtained by re-reading the spread page into multi-gradation data by using the reading method displayed by the control unit.
The detection unit detects the correction range of the binding unit from the multi-gradation data, and then
The control unit controls to display an image showing the correction range of the binding portion obtained from the first read data and an image showing the correction range of the binding portion obtained from the second read data side by side. Item 2. The image forming apparatus according to any one of Items 1 to 6. The image forming apparatus according to claim 7, further comprising a correction unit for correcting the second read data, with the correction range of the binding portion obtained from the second read data as a correction target. The conversion unit further converts the first read data into binary data, and then converts the first read data into binary data.
The detection unit further detects the shape of the shadow of the binding portion from the binary data, and further detects the shape of the shadow of the binding portion.
The estimation unit is a reading method for reducing the correction range of the binding portion from the correction range of the binding portion and the shape of the shadow of the binding portion detected by the detection unit, and the reading method when the reading method is performed. Estimate the correction range of the binding part and the shape of the shadow of the binding part,
Any one of claims 1 to 8, wherein the control unit controls to display an image showing the correction range of the binding portion estimated by the estimation unit and the shape of the shadow of the binding portion together with the reading method. The image forming apparatus according to. 9. The estimation unit estimates the correction range of the binding portion and the shape of the shadow of the binding portion when the reading method and the reading method are performed by using the learning data obtained by machine learning. The image forming apparatus according to. The learning data is an image showing the correction range of the binding portion and the shape of the shadow of the binding portion detected from the learning reading data obtained by reading the spread page including the binding portion of the book in a predetermined state. The data is obtained by associating an image showing the correction range of the binding portion and the shadow shape of the binding portion when the reading method for reducing the correction range of the binding portion is performed and the reading method. Item 10. The image forming apparatus according to item 10. A program for causing a computer to function as each part included in the image forming apparatus according to any one of claims 1 to 11.