JP2021034813A - Image processing device and image processing program - Google Patents

Abstract

To provide an image processing device that, when correcting a punch hole in an image, can reduce an uncomfortable feeling of a correction result when there is a punch hole on a character, compared to a case where interpolation processing is uniformly performed regardless of whether there is a character around the punch hole.SOLUTION: An image processing device includes a processor. The processor corrects punch holes by using different methods to correct the punch holes depending on whether an area around the punch holes in an image is a character or a non-character.SELECTED DRAWING: Figure 1

Description

The present invention relates to an image processing apparatus and an image processing program.

Patent Document 1 is an object of providing an image processing device, an image processing method, and a program for repairing punch hole marks without discomfort in an image having punch hole marks, and the image processing device is included in an input image. An image processing device that removes punch hole marks, based on punch hole area detecting means for detecting punch hole areas from an input image and a pixel pattern of a peripheral area including an area in contact with the punch hole area. It is disclosed that it has an area type determining means for determining an area type and a punch hole repairing means for repairing punch hole marks based on the determined area type.

Japanese Unexamined Patent Publication No. 2010-154319

When a paper document having punch holes is read and digitized, punch holes remain on the electronic document. Therefore, when a punch hole is detected when reading a paper document, there is a function of correcting the punch hole and digitizing the paper document. As a method of correcting punch holes, there is a method of filling the punch holes with a color based on the pixels around the punch holes. However, when punch holes are opened on the characters of a paper document, if the punch holes are filled with colors based on the pixels around the punch holes, a feeling of strangeness will occur in the filled punch hole portions.
Therefore, in the present invention, when the punch holes in the image are corrected, the punch holes are opened on the characters as compared with the case where the interpolation processing is uniformly performed regardless of whether or not there are characters around the punch holes. It is an object of the present invention to provide an image processing apparatus and an image processing program capable of reducing the discomfort of the correction result.

The gist of the present invention for achieving such an object lies in the inventions of the following items.
The invention of claim 1 includes a processor, in which the punch holes are punched in different ways depending on whether the punch holes in the image are around characters or non-characters. It is an image processing device characterized by modifying.

According to the second aspect of the present invention, when the periphery of the punch hole is a character, the processor inquires the user about the character to be described in the punch hole, and the punch hole is the character input by the operation of the user. The image processing apparatus according to claim 1, wherein the image processing apparatus is modified.

The invention of claim 3 is based on the character color, size, or typeface input by the user's operation based on the character color, size, or typeface around the punch hole. 2. The image processing apparatus according to claim 2.

The invention of claim 4 is the image processing apparatus of claim 2, wherein the processor displays a virtual keyboard according to the type of characters around the punched holes.

The invention of claim 5 is the image processing apparatus according to claim 1, wherein the processor presents a candidate for an image in the periphery of the punch hole to the user when the periphery of the punch hole is not a character.

The invention according to claim 6 is the image processing apparatus according to claim 5, wherein the processor uses the same size or shape as the punched hole for the size or shape of the image candidate in the periphery.

The invention of claim 7 is as a correction means for correcting the punch holes by differently modifying the punch holes depending on whether the periphery of the punch holes in the image is characters or non-characters. It is an image processing program for making it work.

According to the image processing apparatus of claim 1, when the punch holes in the image are corrected, the punch holes are punched on the characters as compared with the case where the interpolation processing is uniformly performed regardless of whether or not there are characters around the punch holes. It is possible to reduce the discomfort of the correction result when is open.

According to the image processing apparatus of claim 2, when the periphery of the punched hole is a character, the user can input the character to be described in the punched hole.

According to the image processing device of claim 3, the color, size, or typeface of the characters input by the user's operation can be matched with the color, size, or typeface of the characters around the punched holes. it can.

According to the image processing apparatus of claim 4, the user can use a virtual keyboard according to the type of characters around the punched holes.

According to the image processing apparatus of claim 5, when the periphery of the punched hole is not a character, a candidate for an image around the punched hole can be presented to the user.

According to the image processing apparatus of claim 6, the user does not need to specify the same size or shape as the punched hole.

According to the image processing program of claim 7, when the punch holes in the image are corrected, the punch holes on the characters are compared with the case where the interpolation processing is uniformly performed regardless of whether or not there are characters around the punch holes. It is possible to reduce the discomfort of the correction result when is open.

It is a conceptual module configuration diagram about the configuration example of this embodiment. It is explanatory drawing which shows the system configuration example using this embodiment. It is explanatory drawing which shows the example of the function which corrects a punch hole. It is a flowchart which shows the processing example by this embodiment. It is a flowchart which shows the processing example by this embodiment. It is explanatory drawing which shows the processing example by this embodiment. It is a flowchart which shows the processing example by this embodiment. It is explanatory drawing which shows the processing example by this embodiment. It is a flowchart which shows the processing example by this embodiment. It is explanatory drawing which shows the processing example by this embodiment. It is explanatory drawing which shows the processing example by this embodiment. It is a flowchart which shows the processing example by this embodiment. It is a block diagram which shows the hardware configuration example of the computer which realizes this embodiment.

Hereinafter, an example of a preferred embodiment for realizing the present invention will be described with reference to the drawings.
FIG. 1 shows a conceptual module configuration diagram for a configuration example of the present embodiment.
Note that a module generally refers to parts such as software (including a computer program as an interpretation of "software") and hardware that are logically separable. Therefore, the module in this embodiment refers not only to the module in the computer program but also to the module in the hardware configuration. Therefore, in this embodiment, a computer program for functioning as those modules (for example, a program for causing the computer to perform each procedure, a program for causing the computer to function as each means, and a computer for each of them. It also serves as an explanation of the program), system, and method for realizing the functions of. However, for convenience of explanation, words equivalent to "remember" and "remember" are used, but these words are stored in a storage device or stored when the embodiment is a computer program. It means that it is controlled so that it is stored in the device. Further, the modules may have a one-to-one correspondence with the functions, but in the implementation, one module may be configured by one program, a plurality of modules may be configured by one program, and conversely, one module may be configured. May be composed of a plurality of programs. Further, the plurality of modules may be executed by one computer, or one module may be executed by a plurality of computers by a computer in a distributed or parallel environment. In addition, one module may include another module. In addition, hereinafter, "connection" is used not only for physical connection but also for logical connection (for example, data transfer, instruction, reference relationship between data, login, etc.). "Predetermined" means that it is determined before the target process, not only before the process according to the present embodiment starts, but also after the process according to the present embodiment starts. However, if it is before the target process, it is used with the intention that it is determined according to the situation / state at that time or according to the situation / state up to that point. When there are a plurality of "predetermined values", they may be different values, and two or more values ("two or more values" include, of course, all values). It may be the same. Further, the description "if A, do B" is used to mean "determine whether or not it is A, and if it is determined to be A, do B". However, this excludes cases where it is not necessary to determine whether or not it is A. Further, when a thing is listed such as "A, B, C", it is an example list unless otherwise specified, and includes a case where only one of them is selected (for example, only A).
In addition, a system or device is configured by connecting a plurality of computers, hardware, devices, etc. by communication means such as a network (the "network" includes a one-to-one correspondence communication connection), and one. It also includes cases where it is realized by computers, hardware, devices, and the like. "Device" and "system" are used as synonymous terms. Of course, the "system" does not include anything that is nothing more than a social "mechanism" (that is, a social system) that is an artificial arrangement.
In addition, for each process by each module or when multiple processes are performed in the module, the target information is read from the storage device, and after the processes are performed, the process results are written to the storage device. is there. Therefore, the description of reading from the storage device before processing and writing to the storage device after processing may be omitted. The storage device here includes a hard disk drive, a RAM (abbreviation of Random Access Memory), an external storage medium, a storage device via a communication line, a register in a CPU (abbreviation of Central Processing Unit), and the like. May be good.

The image processing device 100 of the present embodiment has an image processing function of correcting punch holes from an image in which a document is read, and as shown in the example of FIG. 1, the image data acquisition module 105 and punch holes It has a detection module 110, a peripheral information classification module 115, a user interface module 120, a punch hole correction module 125, and an image data output module 130.
Punched holes are holes created by punching (punching) paper such as documents. For example, a round hole drilled to bind a document using a file, clipboard, string, or the like. The punched holes include a copy of a document having punched holes.
The image processing device 100 detects the image of the punched hole from the image of the document in which the punched hole is read, and corrects the image of the punched hole. If the document information is missing in the partial image of the punched hole, the punched hole portion is interpolated from the information in the vicinity thereof to generate an image. The missing information includes characters, photographic images, etc. written on the punched holes.

The image data acquisition module 105 is connected to the punch hole detection module 110. The image data acquisition module 105 acquires an image of the document and passes the image to the punch hole detection module 110. For example, reading a paper document with a scanner, camera, etc., receiving an image from an external device via a communication line by fax, etc., a storage device (a hard disk built into a computer, etc., as well as via a network) It includes reading out images stored in (including connected servers, etc.). The image may be a binary image or a multi-value image (the multi-value image includes a color image). The image to be accepted may be a single image or an image composed of a plurality of pages. Further, the content of the image may be a document used for business, a pamphlet for advertising, or the like.

The punch hole detection module 110 is connected to the image data acquisition module 105 and the peripheral information classification module 115. The punch hole detection module 110 detects an image of punch holes from the image acquired by the image data acquisition module 105. As a method for detecting an image of punched holes, an existing technique may be used. For example, the image of punched holes may be detected by template matching. In addition, the edge in the image is detected, binarized, the circle in the image is searched by using the Hough transform, and if the circle has a predetermined size, the image of the punched hole is detected. It may be. As a predetermined size, for example, ISO838 defines a diameter of 5.5 mm to 6.5 mm, so a value in the vicinity thereof may be used.

The peripheral information classification module 115 is connected to the punch hole detection module 110 and the punch hole correction module 125. The peripheral information classification module 115 classifies the images around the punch holes detected by the punch hole detection module 110.
For example, the peripheral information classification module 115 may classify the periphery of the punched hole as if it is a character or a non-character. The judgment here may be made by using the conventional technique.
Further, the peripheral information classification module 115 may classify as if it is a photographic image or not a photographic image when the periphery of the punched hole is other than characters. The judgment here may be made by using the conventional technique.
Further, the peripheral information classification module 115 may classify as if it is a white background image or a solid image other than the white background when the periphery of the punched hole is other than characters and is not a photographic image. The judgment here may be made by using the conventional technique.
Of course, the peripheral information classification module 115 may classify the periphery of the punched hole as any of a character, a photographic image, a white background image, and a solid image other than the white background, without being hierarchical.

Further, when the peripheral information classification module 115 determines that the periphery of the punched hole is a character, the peripheral information classification module 115 may determine any one or more of the color, size, or typeface of the character around the character. The judgment here may be made by using the conventional technique.
Further, when the peripheral information classification module 115 determines that the periphery of the punched hole is a character, the peripheral information classification module 115 may determine the type of the character. The "character type" will be described later. The judgment here may be made by using the conventional technique. For example, a character recognition technique may be used to determine the type of character. Specifically, a plurality of character recognition applications such as a character recognition application for English characters and a character recognition application for Japanese may be used, and an application having a high accuracy of the character recognition result may be adopted. For example, when the accuracy of the character recognition result in the character recognition application for alphabetic characters is high, the user interface module 120 displays a virtual keyboard for alphabetic characters.

The user interface module 120 is connected to the punch hole correction module 125. The user interface module 120 provides a user interface for correcting punch holes to the user according to the classification result by the peripheral information classification module 115.
For example, the user interface module 120 controls a display device such as a liquid crystal display or an organic EL display that also serves as a touch panel, accepts a user's operation, and presents a message or the like to the user. In addition, the user's operation (including line of sight, gesture, voice, etc.) using a mouse, keyboard, camera, microphone, etc. may be accepted, or the user may use the voice output from the speaker or the tactile sensation using the tactile device. You may want to present a message to.
Specifically, it presents a virtual keyboard for character input and presents peripheral image candidates to the user in a selectable manner.

The punch hole correction module 125 is connected to the peripheral information classification module 115, the user interface module 120, and the image data output module 130. The punch hole correction module 125 corrects the punch holes by changing the method of correcting the punch holes depending on whether the periphery of the punch holes in the image is characters or non-characters.
Here, the "character" is a character in the image and has an image corresponding to the character code. The characters include alphanumeric characters, kanji, hiragana, katakana, characters corresponding to other languages, pictograms, and the like.
When the punch hole correction module 125 is a character around the punch hole, the punch hole correction module 125 asks the user for the character to be described in the punch hole, and corrects the punch hole with the character input by the user's operation. You may try to do it.
In that case, the punch hole correction module 125 determines the character color, size, or typeface input by the user's operation based on the character color, size, or typeface around the punched hole. You may do so.
Determine whether the character string of the document is horizontal or vertical, and if it is horizontal, refer to the characters on the right or left side of the punched hole as "characters around the punched hole" and vertically. In the case of writing, the characters on the upper side or the lower side of the punched hole may be referred to as "characters around the punched hole".
"Based on the color, size, or typeface of the text around the punched hole" means the color, size, or size of the text to be embedded in the punched hole so that it is the same as the color, size, or typeface. Or, to determine the typeface.
Further, the punch hole correction module 125 may display a virtual keyboard according to the type of characters around the punch holes.
The "type of character around the punched hole" is the type of language or number that uses the character. For example, if the "type of characters around the punch hole" is Japanese, the virtual keyboard for Japanese is displayed, if it is a number, the virtual keyboard for numbers is displayed, and if it is an alphabetic character, it is displayed. Display the virtual keyboard for alphabets, and if it is Chinese, display the virtual keyboard for Chinese.
The display destination is, of course, the display device provided in the image processing device 100 or the display device having a user interface function for using the image processing device 100. As an example of the latter, it is a mobile terminal possessed by a user, and the image processing device 100 and its mobile information terminal communicate (may be wireless communication or may be wired communication). , The virtual keyboard may be displayed on the display device of the mobile information terminal.

Further, when the punch hole correction module 125 is not a character around the punch hole, the punch hole correction module 125 may present a candidate for an image around the punch hole to the user.
The punched holes are corrected using the image selected by the user's operation. The modification in this case corresponds to embedding the selected image in the punched hole portion.
Further, as a candidate, an image having the same or similar hue as the image around the punched hole may be searched. Note that the punch hole correction module 125 only indicates a location as a candidate, and its size or shape may be left to the user's designation.
Further, the punch hole correction module 125 may use the same size or shape as the punch hole as the size or shape of the image candidate in the periphery.

The image data output module 130 is connected to the punch hole correction module 125. The image data output module 130 receives an image in which the punch hole image is corrected by the punch hole correction module 125, and outputs the image. To output an image means, for example, printing with a printing device such as a printer, displaying the image on a display device such as a display, transmitting an image with an image transmitting device such as FAX, or outputting an image to an image storage device such as an image database. This includes writing the image, storing the image in a storage medium such as a memory card, and passing the image to another information processing device. When the image is output as an electronic document, PDF (abbreviation of Portable Document Format), JPEG (abbreviation of Joint Photographic Experts Group), or the like may be used.

FIG. 2 is an explanatory diagram showing an example of a system configuration using the present embodiment.
The example shown in FIG. 2A is a stand-alone system in which the function of the image processing device 100 is realized in the image forming device 200. For example, the image processing apparatus 100 is used as a part of the copying function.
The image forming apparatus 200 includes an image processing apparatus 100, a scanning module 205, and a printing module 220.
The scan module 205 is connected to the image processing device 100. The scan module 205 reads a paper document having punch holes and passes the read image to the image processing device 100.
The image processing device 100 is connected to the scan module 205 and the print module 220. The image processing device 100 corrects the image of the punch hole with respect to the image received from the scan module 205, and passes the corrected image to the print module 220.
The print module 220 is connected to the image processing device 100. The printing module 220 receives an image in which the image of punch holes is corrected by the image processing device 100 and prints the image.

In the example shown in FIG. 2B, the function of the image processing device 100 is realized in the information processing device 230 such as a personal computer. For example, the image processing device 100 is used when a paper document is read by an image forming device 240 having a function as a scanner and the image is processed.
The image forming apparatus 240 is connected to the information processing apparatus 230. The image forming apparatus 240 reads a paper document having punch holes and passes the image to the information processing apparatus 230.
The information processing device 230 has an image processing device 100 and is connected to the image forming device 240. The image processing device 100 of the information processing device 230 corrects the image of the punch hole with respect to the image received from the image forming device 240, and stores the corrected image in the storage device of the information processing device 230.

The example shown in FIG. 2C is a network type system in which, for example, the function of the image processing device 100 is realized as a cloud service.
The image processing device 100, the image forming device 240a, the image forming device 240b, and the information processing device 250 are each connected via a communication line 290. The communication line 290 may be wireless, wired, or a combination thereof, and may be, for example, the Internet as a communication infrastructure, an intranet, or the like.
For example, the scanning function of the image forming apparatus 240 reads a paper document having punch holes, transmits the image to the image processing apparatus 100, receives an image obtained by modifying the punched hole image as a reply, and forms an image. Printing may be performed by the print function of the device 240.
Further, the image with punch holes stored in the information processing device 250 may be transmitted to the image processing device 100, and an image obtained by modifying the punched hole image may be received as a reply.

FIG. 3 is an explanatory diagram showing an example of a function of correcting punch holes.
As shown in the example of FIG. 3A, when a document 300 having punched holes 302a and punched holes 302b is copied or scanned, the punched holes 302a and punched holes 302b become conspicuous in the color of the paper holding portion. ..
For example, when a normal copy 310 is performed on the original 300, when the paper holding portion is black, the punched hole image 322a and the punched hole image 322b of the image data 320 are as shown in the example of FIG. 3 (b). It becomes black. Further, even if the paper holding portion is white, as shown in the example of FIG. 3C, around the punched holes as shown in the punched hole image 332a and the punched hole image 332b of the image data 330. The punch hole correction function in which a shadow remains is a function for removing the punch hole marks shown in the examples of FIGS. 3 (b) and 3 (c). That is, when the punch hole correction function 340 is used for the original 300, the punch hole image 322a, the punch hole image 322b or the punch hole image 332a, and the punch hole image 332b are not left from the image data 350. ..
In this example, the peripheral images of the punched holes 302a and the punched holes 302b are white background images, and the images of the punched holes 302a and the punched holes 302b are painted in white.
However, for example, when punch holes 302a and punch holes 302b are formed in the portion where the characters are written, if they are painted in white, a sense of incongruity remains. Therefore, in the image processing apparatus 100 of the present embodiment, the method of correcting the punch holes differs depending on whether the periphery of the punch holes in the image is characters or non-characters. Specifically, if the area around the punched hole is a character, the user can enter the character. Further, when the periphery of the punched hole is a photographic image, the user can select a candidate image in the periphery.

FIG. 4 is a flowchart showing a processing example according to the present embodiment.
In step S402, the punch hole detection module 110 detects punch holes in the image data.
In step S404, the punch hole detection module 110 determines whether or not there is a punch hole, proceeds to step S406 if there is a punch hole, and ends the process in other cases (step S499).

In step S406, the peripheral information classification module 115 determines the classification around the punched hole, proceeds to step S408 if it is not a character, and proceeds to step S410 if it is a character.
In step S408, the user interface module 120 and the punch hole correction module 125 perform processing for non-characters. The detailed processing of step S408 will be described later using the flowchart shown in the example of FIG.
In step S410, the user interface module 120 and the punch hole correction module 125 perform character processing. The detailed processing of step S410 will be described later using the flowchart shown in the example of FIG.

FIG. 5 is a flowchart showing a processing example according to the present embodiment. The processing example of step S410 of the flowchart shown in the example of FIG. 4 is shown.
In step S502, the missing or missing characters are counted from the surrounding character size. Specifically, the characters around the punched holes are extracted and the size of the characters is measured. Then, the size of the punch hole is divided by the size of the character to calculate. Further, a predetermined number may be added to the divided value. This is because if the punched holes are missing a part of the characters, it is expected that the punched holes will be missing more characters than the divided value.

In step S504, the user interface is displayed so that the user can input characters.
For example, the user interface shown in FIG. 6 is displayed. FIG. 6 is an explanatory diagram showing a processing example according to the present embodiment. The image display area 610, the comment display area 630, and the virtual keyboard display area 640 are displayed on the screen 600. The input area 620 is displayed in the image display area 610. The input area 620 is displayed so as to surround at least the punched holes. A description of character input is displayed in the comment display area 630. For example, "Touch □ to enter characters" is displayed. In step S508, the virtual keyboard is displayed in the virtual keyboard display area 640.

In step S506, peripheral character processing is performed. The detailed processing of step S506 will be described later using the flowchart shown in the example of FIG. 7.
In step S508, when the user touches the character input location, a virtual keyboard is displayed on the user interface and the user is made to input.
In step S510, an image after punch holes are removed is generated based on the content input by the user.

FIG. 7 is a flowchart showing a processing example according to the present embodiment. The processing example of step S506 of the flowchart shown in the example of FIG. 5 is shown.
In step S702, the peripheral image is extracted. For example, a rectangular area surrounding the image of the punch hole may be generated, and a peripheral image having a size obtained by multiplying the rectangular area by a predetermined magnification may be extracted, or characters at a distance close to the image of the punch hole may be extracted. At least one character may be extracted.
In step S704, the character type, character color, size, or typeface is extracted from the peripheral image. The characters in the peripheral image extracted in step S702 are targeted, and the type, color, size, or typeface of the characters is extracted by using the conventional technique.

In step S706, character properties are set. This property is the character type, color, size, or typeface extracted in step S704.
Depending on the type of this character, the virtual keyboard is displayed in the virtual keyboard display area 640 shown in the example of FIG. For example, if the character type is a number, a virtual keyboard dedicated to numbers is displayed, and if the character type is alphabetic, a virtual keyboard for alphabets is displayed, and the character type is Japanese (for example,). , Kanji, Hiragana, Katakana), you can display the virtual keyboard for Japanese.
Set the characters entered by the user according to the color, size, or typeface of the characters. That is, characters with the same attributes as the characters around the punch holes are entered by the user. Therefore, there is less discomfort between the conventional characters and the characters input by the user.

FIG. 8 is an explanatory diagram showing a processing example according to the present embodiment. An example of processing for generating an input area 620 to be displayed in the image display area 610 shown in the example of FIG. 6 is shown.
The punched hole area 810 and the input area 820 are displayed on the peripheral image 800. The input area 820 is generated as follows. A rectangle is generated so as to include at least the punched hole region 810 and further surround the black pixels in contact with the punched hole region 810 to be an input region 820. The user inputs characters in the input area 820. In the example of FIG. 8, "AB" is input to the input area 820. The punch holes are corrected by using the input area 820 in which characters are input. In the example of FIG. 8, the punched hole area 810 is deleted and becomes a character string of “ABCDE”.

FIG. 9 is a flowchart showing a processing example according to the present embodiment. The processing example of step S408 of the flowchart shown in the example of FIG. 4 is shown.
In step S902, a plurality of filled data are created. For example, in the target page image, an image similar to the image around the punched hole is searched. The area to be searched is the same size and shape as the punched hole. Specifically, as shown in the example of FIG. 10A, the hole filling candidate 1020a, the hole filling candidate 1020b, and the hole filling candidate 1020c are extracted as images similar to the periphery of the punched hole 1010.

In step S904, an image in which punch holes are filled with the created data is generated.
In step S906, all the generated images are displayed on the user interface and the user is allowed to select them. Specifically, as shown in the example of FIG. 10B, the candidate image 1030a, the candidate image 1030b, and the candidate image 1030c are generated. The original punched hole region 1040a of the candidate image 1030a is a punched hole 1010 fitted with the hole filling candidate 1020a. The original punched hole region 1040b of the candidate image 1030b is a punched hole 1010 fitted with a hole filling candidate 1020b. The original punched hole region 1040c of the candidate image 1030c is a punched hole 1010 fitted with the hole filling candidate 1020c.

In step S908, if the user wishes to process the image, the image is processed. Specifically, as shown in the example of FIG. 11A, it is assumed that the user selects the candidate image 1030b. Therefore, the user designates the area 1110 as the area to be deleted. After the region 1110 is deleted, the modified image 1150 shown in the example of FIG. 11B is obtained.
As a further correction method, although the load on the user increases, the color value may be input for each pixel.

FIG. 12 is a flowchart showing a processing example according to the present embodiment. Step S406 of the flowchart shown in the example of FIG. 4 was classified into two types, but step S1206 of the flowchart shown in the example of FIG. 12 was classified into four types.
In step S1202, punch holes are detected in the image data.
In step S1204, it is determined whether or not there is a punched hole, and if there is a punched hole, the process proceeds to step S1206, and if not, the process ends (step S1299).

In step S1206, the classification around the punch holes is determined, and if it is a white background, the process proceeds to step S1208, if it is solid, the process proceeds to step S1210, if it is a photograph, it proceeds to step S1212, and if it is a character, it proceeds to step S1214.

In step S1208, the punch holes are filled with white pixels.
In step S1210, the punch holes are filled with the average value of the peripheral pixels. In addition to the average value, other statistical values may be used, for example, the mode value and the median value.

In step S1212, photographic processing is performed. The detailed processing of step S1212 has been described above using the flowchart shown in the example of FIG.
In step S1214, character processing is performed. The detailed processing of step S1214 has been described above using the flowchart shown in the example of FIG.

A hardware configuration example of the image processing device 100 of the present embodiment will be described with reference to FIG. The configuration shown in FIG. 13 is configured by, for example, a personal computer or the like, and shows an example of a hardware configuration including a data reading unit 1317 such as a scanner and a data output unit 1318 such as a printer.

The CPU (abbreviation of Central Processing Unit) 1301 includes various modules described in the above-described embodiment, that is, an image data acquisition module 105, a punch hole detection module 110, a peripheral information classification module 115, a user interface module 120, and a punch hole. It is a control unit that executes processing according to a computer program that describes an execution sequence of each module such as a modification module 125 and an image data output module 130.

The ROM (abbreviation of Read Only Memory) 1302 stores programs, calculation parameters, and the like used by the CPU 1301. The RAM (abbreviation of Random Access Memory) 1303 stores a program used in the execution of the CPU 1301, parameters that are appropriately changed in the execution, and the like. These are connected to each other by a host bus 1304 composed of a CPU bus or the like.

The host bus 1304 is connected to an external bus 1306 such as a PCI (Peripheral Component Interconnect / Interface) bus via a bridge 1305.

The pointing device 1309 such as the keyboard 1308 and the mouse is a device operated by the operator. The display 1310 includes a liquid crystal display device, a CRT (abbreviation for Cathode Ray Tube), and the like, and displays various types of information as text or image information. Further, a touch screen or the like having the functions of both the pointing device 1309 and the display 1310 may be used. In that case, regarding the realization of the keyboard function, a keyboard (also called a so-called software keyboard, screen keyboard, etc.) is drawn by software on the screen (for example, touch screen) without physically connecting like the keyboard 1308. It may be possible to realize the function of the keyboard.

The HDD (abbreviation of Hard Disk Drive) 1311 has a built-in hard disk (a flash memory or the like may be used in addition to the hard disk), drives the hard disk, and records or reproduces programs and information executed by the CPU 1301. The HDD 1311 stores the acquired image data, the result data of processing by the punch hole detection module 110, the peripheral information classification module 115, the user interface module 120, the punch hole correction module 125, and the like. In addition, various other data, various computer programs, etc. are stored.

The drive 1312 reads out the data or program recorded on the removable recording medium 1313 such as the mounted magnetic disk, optical disk, magneto-optical disk, or semiconductor memory, and transfers the data or program to the interface 1307 and the external bus 1306. , Bridge 1305, and RAM 1303 connected via host bus 1304. The removable recording medium 1313 can also be used as a data recording area.

The connection port 1314 is a port for connecting the external connection device 1315, and has a connection portion such as USB or IEEE 1394. The connection port 1314 is connected to the CPU 1301 and the like via the interface 1307, the external bus 1306, the bridge 1305, the host bus 1304, and the like. The communication unit 1316 is connected to the communication line and executes data communication processing with the outside. The data reading unit 1317 is, for example, a scanner, and executes a document reading process. The data output unit 1318 is, for example, a printer, and executes a document data output process.

Among the above-described embodiments, in the case of a computer program, the system of the present hardware configuration is made to read the computer program which is software, and the software and the hardware resources cooperate with each other to carry out the above-described embodiment. Is realized.
The hardware configuration of the image processing device 100 shown in FIG. 13 shows one configuration example, and the present embodiment is not limited to the configuration shown in FIG. 13, and the modules described in the present embodiment are used. Any configuration may be feasible. For example, as a processor, a GPU (abbreviation of Graphics Processing Unit, including GPGPU (abbreviation of General-Purpose computing on Graphics Processing Units)) may be used, or some modules may be used with dedicated hardware (for example, for a specific application). It is composed of an integrated circuit (specific example, there is an ASIC (abbreviation of Application Specific Integrated System), etc.) and a reconfigurable integrated circuit (specific example, there is an FPGA (abbreviation of Field-Programmable Gate Array), etc.). Some modules may be in an external system and connected by a communication line, and a plurality of systems shown in FIG. 13 may be connected to each other by a communication line so as to cooperate with each other. In addition to personal computers, mobile information communication devices (including mobile phones, smartphones, mobile devices, wearable computers, etc. as mobile information communication devices), information appliances, robots, copying machines, fax machines, scanners, printers, etc. It may be incorporated in a compound machine (the compound machine is an image processing device having any two or more functions such as a scanner, a printer, a copying machine, and a fax machine).

In the above embodiment, the processor refers to a processor in a broad sense, and includes a general-purpose processor (for example, CPU, etc.) and a dedicated processor (for example, GPU, ASIC, FPGA, programmable logic device, etc.).
Further, the operation of the processor in the above embodiment may be performed by one processor, and not only by one processor, but also by a plurality of processors existing at physically separated positions in cooperation with each other. It may be made. Further, the order of each operation of the processor is not limited to the order described in the above embodiment, and may be changed as appropriate.

The described program may be stored in a recording medium and provided, or the program may be provided by a communication means. In that case, for example, the program described above may be regarded as an invention of "a computer-readable recording medium on which the program is recorded".
The "computer-readable recording medium on which a program is recorded" means a computer-readable recording medium on which a program is recorded, which is used for program installation, execution, program distribution, and the like.
The recording medium is, for example, a digital versatile disc (DVD), which is a standard established by the DVD Forum "DVD-R, DVD-RW, DVD-RAM, etc." and DVD + RW. Standards such as "DVD + R, DVD + RW, etc.", compact discs (CD), read-only memory (CD-ROM), CD recordable (CD-R), CD rewritable (CD-RW), etc., Blu-ray discs (CD-RW) Blu-ray (registered trademark) Disc), optical magnetic disk (MO), flexible disk (FD), magnetic tape, hard disk, read-only memory (ROM), electrically erasable and rewritable read-only memory (EEPROM (registered trademark)) )), Flash memory, random access memory (RAM), SD (abbreviation of Secure Digital) memory card and the like.
Then, the whole or a part of the program may be recorded on the recording medium and stored, distributed, or the like. Further, by communication, for example, a wired network used for a local area network (LAN), a metropolitan area network (MAN), a wide area network (WAN), the Internet, an intranet, an extranet, or wireless communication. It may be transmitted using a transmission medium such as a network or a combination thereof, or may be carried on a carrier.
Further, the program may be a part or all of other programs, or may be recorded on a recording medium together with a separate program. Further, the recording may be divided into a plurality of recording media. Further, it may be recorded in any mode as long as it can be restored, such as compression and encryption.

100 ... Image processing device 105 ... Image data acquisition module 110 ... Punch hole detection module 115 ... Peripheral information classification module 120 ... User interface module 125 ... Punch hole correction module 130 ... Image data output module 200 ... Image forming device 205 ... Scan module 220 … Printing module 230… Information processing device 240… Image forming device 250… Information processing device 290… Communication line

Claims (7)

Equipped with a processor
The processor
An image processing apparatus characterized in that the punch holes are corrected by different methods for correcting the punch holes depending on whether the periphery of the punch holes in the image is characters or other than characters. The processor
When the area around the punch hole is a character, the user is inquired about the character to be described in the punch hole, and the punch hole is corrected by the character input by the operation of the user.
The image processing apparatus according to claim 1. The processor
The color, size, or typeface of the character input by the user's operation is determined based on the color, size, or typeface of the character around the punched hole.
The image processing apparatus according to claim 2. The processor
Display the virtual keyboard according to the type of characters around the punched holes.
The image processing apparatus according to claim 2. The processor
If the area around the punched hole is not a character, the user is presented with a candidate image in the area.
The image processing apparatus according to claim 1. The processor
As the size or shape of the image candidate in the periphery, the same size or shape as the punched hole is used.
The image processing apparatus according to claim 5. Computer,
An image processing program for differentiating the method of correcting punch holes depending on whether the periphery of the punch holes in the image is characters or other than characters, and to function as a correction means for correcting the punch holes.