JP7342518B2 - Image processing device and image processing program - Google Patents

Description

The present invention relates to an image processing device and an image processing program.

Patent Document 1 has an object to provide an image processing device, an image processing method, and a program for restoring punch hole marks in an image having punch hole marks without any discomfort, and the image processing device is provided with An image processing device that removes punch hole marks, the device includes a punch hole area detection means that detects a punch hole area from an input image, and a punch hole area detection means that detects a punch hole area from an input image based on a pixel pattern of the surrounding area including an area that is in contact with the punch hole area. It is disclosed that the method includes an area type determining means for determining the area type, and a punch hole repairing means for repairing punch hole marks based on the determined area type.

Japanese Patent Application Publication No. 2010-154319

When a paper document with punched holes is read and digitized, the punched holes remain on the electronic document. Therefore, if a punch hole is detected when reading a paper document, there is a function to correct the punch hole and digitize the paper document. As a method for correcting punch holes, there is a method of filling the punch hole with a color based on pixels around the punch hole. However, when a punch hole is formed on a character in a paper document, if the punch hole is filled with a color based on pixels around the punch hole, the filled punch hole portion will look strange.
Therefore, when correcting a punch hole in an image, the present invention improves the accuracy of the punch hole when there is a punch hole on the character, compared to the case where interpolation processing is uniformly performed regardless of whether there is a character around the punch hole. It is an object of the present invention to provide an image processing device and an image processing program that can reduce discomfort caused by correction results.

The gist of the present invention to achieve this object resides in the following inventions. The "claims" below are the claims originally filed.
The invention according to claim 1 includes a processor, and the processor corrects the punch hole by changing the method of correcting the punch hole depending on whether the surrounding area of the punch hole in the image is text or non-text. This is an image processing device characterized by correction.

In the invention according to claim 2, when the area surrounding the punch hole is a character, the processor inquires the user about the character to be written in the punch hole, and writes the character input in the punch hole by the user's operation. The image processing apparatus according to claim 1, wherein the image processing apparatus corrects the image processing apparatus.

The invention according to claim 3 is characterized in that the processor determines the color, size, or font of the characters input by the user's operation based on the color, size, or font of the characters around the punch hole. The image processing device according to claim 2 , wherein the image processing device determines the image by using the image processing method.

A fourth aspect of the present invention is the image processing apparatus according to the second aspect, wherein the processor displays a virtual keyboard according to the type of characters around the punch hole.

The invention according to claim 5 is the image processing apparatus according to claim 1, wherein when the area around the punch hole is not a character, the processor presents image candidates around the punch hole to the user.

A sixth aspect of the invention is the image processing apparatus according to the fifth aspect, wherein the processor uses the same size or shape as the punch hole as the size or shape of the image candidate in the periphery.

The invention according to claim 7 uses a computer as a correction means for correcting the punch hole by changing the correction method for the punch hole depending on whether the surrounding area of the punch hole in the image is text or non-text. This is an image processing program to make it work.

According to the image processing device of claim 1, when correcting a punch hole in an image, compared to a case where interpolation processing is uniformly performed regardless of whether or not there is a character around the punch hole, the punch hole on the character is corrected. It is possible to reduce the discomfort of the correction results when the is open.

According to the image processing apparatus of the second aspect, when the area around the punch hole is a character, the user can input the character to be written in the punch hole.

According to the image processing device of claim 3, it is possible to match the color, size, or font of the characters input by the user's operation to the color, size, or font of the characters around the punch hole. can.

According to the image processing apparatus of the fourth aspect, the user can use a virtual keyboard according to the type of characters around the punch hole.

According to the image processing apparatus of the fifth aspect, if the area around the punch hole is not a character, image candidates around the punch hole can be presented to the user.

According to the image processing apparatus of the sixth aspect, the user does not need to specify the same size or shape as the punch hole.

According to the image processing program of claim 7, when correcting a punch hole in an image, compared to a case where interpolation processing is uniformly performed regardless of whether there are characters around the punch hole, punch holes on the character are corrected. It is possible to reduce the discomfort of the correction results when the is open.

FIG. 2 is a conceptual module configuration diagram regarding a configuration example of the present embodiment. FIG. 1 is an explanatory diagram showing an example of a system configuration using this embodiment. FIG. 3 is an explanatory diagram showing an example of a function for correcting punch holes. 3 is a flowchart illustrating an example of processing according to the present embodiment. 3 is a flowchart illustrating an example of processing according to the present embodiment. FIG. 2 is an explanatory diagram showing an example of processing according to the present embodiment. 3 is a flowchart illustrating an example of processing according to the present embodiment. FIG. 2 is an explanatory diagram showing an example of processing according to the present embodiment. 3 is a flowchart illustrating an example of processing according to the present embodiment. FIG. 2 is an explanatory diagram showing an example of processing according to the present embodiment. FIG. 2 is an explanatory diagram showing an example of processing according to the present embodiment. 3 is a flowchart illustrating an example of processing according to the present embodiment. 1 is a block diagram showing an example of a hardware configuration of a computer that implements the present embodiment. FIG.

Hereinafter, an example of a preferred embodiment for realizing the present invention will be described based on the drawings.
FIG. 1 shows a conceptual module configuration diagram of a configuration example of this embodiment.
Note that a module generally refers to logically separable components such as software (computer programs are included in the interpretation of "software"), hardware, and the like. Therefore, the module in this embodiment refers not only to a module in a computer program but also to a module in a hardware configuration. Therefore, this embodiment provides a computer program for making the computer function as those modules (for example, a program for making the computer execute each procedure, a program for making the computer function as each means, a program for making the computer It also serves as an explanation of programs, systems, and methods for realizing the functions of However, for convenience of explanation, words such as "storing", "storing", and equivalent words are used; however, when the embodiment is a computer program, these words mean "storing in a storage device" or "storing". This means controlling the device so that it stores information. In addition, modules may have one-to-one correspondence with functions, but in implementation, one module may be composed of one program, multiple modules may be composed of one program, or conversely, one module may be composed of one program. may be composed of multiple programs. Also, multiple modules may be executed by one computer, or one module may be executed by multiple computers by computers in a distributed or parallel environment. Note that one module may include other modules. Further, hereinafter, the term "connection" is used not only for physical connection but also for logical connection (for example, data exchange, instructions, reference relationships between data, login, etc.). "Predetermined" refers to something that has been determined before the target process, not only before the process according to this embodiment starts, but also after the process according to this embodiment starts. Also, before the target process, it is used to include the meaning that it is determined according to the situation/state at that time or according to the situation/state up to that point. If there are multiple "predetermined values", they may be different values, or two or more values ("two or more values" includes all values, of course). They may be the same. Furthermore, the statement "If A, do B" is used to mean "determine whether A is the case, and if it is A, do B." However, this excludes cases where it is not necessary to determine whether or not it is A. Furthermore, when things are listed like "A, B, C", etc., unless otherwise specified, this is an exemplary list, and includes cases where only one of them is selected (for example, only A).
In addition, a system or device is composed of multiple computers, hardware, devices, etc. connected by communication means such as a network (a "network" includes a one-to-one communication connection), or a This also includes cases where it is realized by a computer, hardware, device, etc. The terms "device" and "system" are used interchangeably. Of course, the term "system" does not include anything that is nothing more than a social "mechanism" (in other words, a social system) that is an artificial arrangement.
In addition, for each process performed by each module, or for each process when multiple processes are performed within a module, the target information is read from the storage device, and after the processing is performed, the processing results are written to the storage device. be. Therefore, a description of reading from the storage device before processing and writing to the storage device after processing may be omitted. Note that storage devices here include hard disk drives, RAM (abbreviation for Random Access Memory), external storage media, storage devices via communication lines, registers in CPU (abbreviation for Central Processing Unit), etc. Good too.

The image processing apparatus 100 according to the present embodiment has an image processing function for correcting punch holes from an image read from a document, and as shown in the example of FIG. It has a detection module 110, a peripheral information classification module 115, a user interface module 120, a punch hole correction module 125, and an image data output module 130.
A punch hole is a hole created by punching a hole in paper such as a document. For example, a round hole punched to bind documents using files, clipboards, strings, etc. Note that punch holes include copies of documents with punch holes.
The image processing apparatus 100 detects an image of a punch hole from a read image of a document having punch holes, and corrects the image of the punch hole. Note that if document information is missing in the punch hole partial image, the punch hole portion is interpolated from information in the vicinity to generate an image. Missing information includes characters written in the punched holes, photographic images, etc.

The image data acquisition module 105 is connected to the punch hole detection module 110. Image data acquisition module 105 acquires an image of the document and passes the image to punch hole detection module 110 . For example, reading paper documents with a scanner or camera, receiving images from an external device via a communication line such as by fax, or using a storage device (such as a hard disk built into a computer, or via a network) This includes reading out images stored in a connected server (including a connected server, etc.). The image may be a binary image or a multivalued image (multivalued images include color images). The image to be accepted may be a single image or may be an image consisting of multiple pages. Furthermore, the content of the image may be a document used for business, a pamphlet for advertising, or the like.

The punch hole detection module 110 is connected to the image data acquisition module 105 and the peripheral information classification module 115. The punch hole detection module 110 detects an image of a punch hole from the image acquired by the image data acquisition module 105. Existing techniques may be used as a method for detecting images of punched holes. For example, images of punched holes may be detected by template matching. It also detects edges in the image, binarizes them, searches for a circle in the image using Hough transform, etc., and if the circle has a predetermined size, detects an image of a punch hole. You may also do so. The predetermined size is, for example, defined in ISO 838 as a diameter of 5.5 mm to 6.5 mm, so a value around this range may be used.

The peripheral information classification module 115 is connected to the punch hole detection module 110 and the punch hole correction module 125. The peripheral information classification module 115 classifies images around the punched hole detected by the punched hole detection module 110.
For example, the surrounding information classification module 115 may classify whether the surroundings of a punch hole are text or non-text. The determination here can be made using conventional techniques.
Further, if the area around the punch hole is anything other than text, the surrounding information classification module 115 may classify it as a photographic image or a non-photographic image. The determination here can be made using conventional techniques.
Further, if the area around the punch hole is not a text and is not a photographic image, the peripheral information classification module 115 may classify the punch hole as either a white background image or a solid image other than a white background. The determination here can be made using conventional techniques.
Of course, the surrounding information classification module 115 may classify the surroundings of a punch hole into text, a photographic image, a white background image, or a solid image other than a white background, instead of hierarchically.

Further, if the peripheral information classification module 115 determines that the area around the punch hole is a character, it may determine one or more of the color, size, or font of the character around the punch hole. The determination here can be made using conventional techniques.
Further, if the peripheral information classification module 115 determines that the area around the punch hole is a character, it may determine the type of the character. Note that the "character type" will be described later. The determination here can be made using conventional techniques. For example, character recognition technology may be used to determine the type of character. Specifically, a plurality of character recognition applications may be used, such as a character recognition application for English characters, a character recognition application for Japanese characters, etc., and one with a high accuracy of character recognition results may be adopted. For example, if the accuracy of the character recognition result in the character recognition application for alphabetic characters is high, the user interface module 120 displays a virtual keyboard for alphabetic characters.

User interface module 120 is connected to punch hole correction module 125 . The user interface module 120 provides the user with a user interface for modifying punch holes according to the classification result by the peripheral information classification module 115.
For example, the user interface module 120 controls a display device such as a liquid crystal display or an organic EL display that also serves as a touch panel, receives user operations, and presents messages and the like to the user. In addition, it may be possible to accept user operations using a mouse, keyboard, camera, microphone, etc. (including line of sight, gestures, voice, etc.), or to provide audio output from speakers and tactile sensations using haptic devices. It may also be possible to present a message to the user.
Specifically, it presents a virtual keyboard for character input and presents nearby image candidates for selection to the user.

The punch hole correction module 125 is connected to the peripheral information classification module 115, the user interface module 120, and the image data output module 130. The punch hole correction module 125 corrects the punch hole using different correction methods depending on whether the area around the punch hole in the image is text or non-text.
Here, a "character" is a character in an image, and has an image corresponding to a character code. Note that the characters include alphanumeric characters, kanji, hiragana, katakana, characters corresponding to other languages, pictograms, etc.
In addition, if the area surrounding the punch hole is a character, the punch hole correction module 125 inquires of the user about the character that should be written in the punch hole, and corrects the punch hole using the character input by the user's operation. You may also do so.
In that case, the punch hole correction module 125 determines the color, size, or font of the characters input by the user's operation based on the color, size, or font of the characters around the punch hole. You can do it like this.
Determine whether the character string in the document is written horizontally or vertically, and if it is written horizontally, refer to the characters on the right or left side of the punch hole as "characters around the punch hole", and In the case of writing, characters above or below the punch hole may be referred to as "characters around the punch hole."
"Based on the color, size, or font of the text surrounding the punch hole" refers to the color and size of the text embedded in the punch hole so that it matches the color, size, or font of the text surrounding the punch hole. , or deciding on a typeface.
Further, the punch hole correction module 125 may display a virtual keyboard according to the type of characters around the punch hole.
"The type of characters around the punch hole" refers to the type of language in which the characters are used, or numbers. For example, if the "type of characters around the punch hole" is Japanese, a virtual keyboard for Japanese is displayed, if it is a number, a virtual keyboard for numbers is displayed, and if it is an alphabetic character, a virtual keyboard for numbers is displayed. Display the virtual keyboard for English characters, and if it is Chinese, display the virtual keyboard for Chinese characters.
The display device is, of course, a display device installed in the image processing device 100 or a display device equipped with a user interface function for using the image processing device 100. An example of the latter is a mobile terminal owned by a user, and the image processing apparatus 100 and the mobile information terminal communicate with each other (which may be wireless communication or wired communication). , a virtual keyboard may be displayed on the display device of the portable information terminal.

Further, if the area around the punch hole is not a character, the punch hole correction module 125 may present image candidates around the punch hole to the user.
The punch hole is corrected using the image selected by the user's operation. In this case, the modification corresponds to embedding the selected image in the punched hole.
Furthermore, it is sufficient to search for an image that has the same or similar hue as an image around the punch hole as a candidate. Note that the punch hole correction module 125 may only indicate a location as a candidate, and the size or shape of the location may be left to the user's designation.
Further, the punch hole correction module 125 may use the same size or shape as the punch hole as the size or shape of the image candidates in the periphery.

The image data output module 130 is connected to the punch hole correction module 125. The image data output module 130 receives the image in which the punch hole image has been corrected by the punch hole correction module 125, and outputs the image. Outputting an image means, for example, printing it with a printing device such as a printer, displaying it on a display device such as a display, transmitting the image with an image transmission device such as a fax, or transmitting the image to an image storage device such as an image database. This includes writing the image, storing the image on a storage medium such as a memory card, and passing the image to another information processing device. Note that when outputting an image as an electronic document, PDF (abbreviation for Portable Document Format), JPEG (abbreviation for Joint Photographic Experts Group), etc. may be used.

FIG. 2 is an explanatory diagram showing an example of a system configuration using this embodiment.
The example shown in FIG. 2A is a stand-alone system in which the functions of the image processing apparatus 100 are realized within the image forming apparatus 200. For example, the image processing device 100 is used as part of the copying function.
The image forming apparatus 200 includes an image processing apparatus 100, a scan module 205, and a print module 220.
The scan module 205 is connected to the image processing device 100. The scan module 205 reads a paper document with punched holes and passes the read image to the image processing apparatus 100.
The image processing apparatus 100 is connected to a scan module 205 and a print module 220. The image processing apparatus 100 corrects the punch hole image in the image received from the scan module 205 and passes the corrected image to the print module 220.
The print module 220 is connected to the image processing device 100. The printing module 220 receives the image in which the punch hole image has been corrected by the image processing device 100, and prints the image.

In the example shown in FIG. 2(b), the functions of the image processing apparatus 100 are realized in an information processing apparatus 230 such as a personal computer. For example, the image processing apparatus 100 is used when an image forming apparatus 240 having a function as a scanner reads a paper document and processes the image.
Image forming device 240 is connected to information processing device 230. Image forming device 240 reads a paper document with punched holes and passes the image to information processing device 230 .
The information processing device 230 includes the image processing device 100 and is connected to the image forming device 240. The image processing device 100 of the information processing device 230 corrects the punch hole image in the image received from the image forming device 240, and stores the corrected image in the storage device of the information processing device 230.

The example shown in FIG. 2C is a network-type system in which, for example, the functions of the image processing apparatus 100 are realized as a cloud service.
The image processing device 100, the image forming device 240a, the image forming device 240b, and the information processing device 250 are each connected via a communication line 290. The communication line 290 may be wireless, wired, or a combination thereof, and may be, for example, the Internet, an intranet, or the like as a communication infrastructure.
For example, a paper document with punched holes is read by the scan function of the image forming device 240, the image is sent to the image processing device 100, an image with the punched holes corrected is received as a reply, and the image is formed. Printing may also be performed using the print function of the device 240.
Alternatively, an image with a punched hole stored in the information processing device 250 may be transmitted to the image processing device 100, and an image with the punched hole corrected may be received as a reply.

FIG. 3 is an explanatory diagram showing an example of a function for correcting punch holes.
As shown in the example of FIG. 3(a), when copying or scanning a document 300 with punch holes 302a and 302b, the punch holes 302a and 302b become conspicuous because they become the color of the paper holding part. .
For example, when a normal copy 310 is performed on a document 300 and the paper holding part is black, the punch hole images 322a and 322b of the image data 320 are It becomes black. Furthermore, even if the paper holding part is white, as shown in the example of FIG. Shadows remain The punch hole correction function is a function for removing the punch hole marks shown in the examples of FIGS. 3(b) and 3(c). In other words, when the punch hole correction function 340 is used for the document 300, the punch hole image 322a, the punch hole image 322b, or the punch hole image 332a, the punch hole image 332b are not left in the image data 350. .
In this example, the surrounding images of the punch holes 302a and 302b are white background images, and the images of the punch holes 302a and 302b are filled with white.
However, for example, if punch holes 302a and punch holes 302b are formed in a portion where characters are written, and the punch holes 302a and 302b are filled in with white, an unnatural feeling will remain. Therefore, in the image processing apparatus 100 according to the present embodiment, the punch hole correction method is different depending on whether the area surrounding the punch hole in the image is text or non-text. Specifically, if there are characters around the punch hole, the user is allowed to enter the characters. Furthermore, if the area surrounding the punch hole is a photographic image, the user is allowed to select image candidates in the area.

FIG. 4 is a flowchart showing an example of processing according to this embodiment.
In step S402, the punch hole detection module 110 detects punch holes from the image data.
In step S404, the punch hole detection module 110 determines whether or not there is a punch hole. If there is a punch hole, the process proceeds to step S406; otherwise, the process ends (step S499).

In step S406, the peripheral information classification module 115 determines the classification of the vicinity of the punch hole, and if the punch hole is not a character, the process proceeds to step S408, and if it is a character, the process proceeds to step S410.
In step S408, the user interface module 120 and punch hole correction module 125 perform processing for characters other than characters. The detailed process of step S408 will be described later using a flowchart shown in the example of FIG.
In step S410, the user interface module 120 and punch hole correction module 125 perform character processing. The detailed process of step S410 will be described later using the flowchart shown in the example of FIG.

FIG. 5 is a flowchart showing an example of processing according to this embodiment. This is an example of processing in step S410 of the flowchart shown in the example of FIG. 4.
In step S502, missing or missing characters are counted from the surrounding character sizes. Specifically, the characters around the punch hole are extracted and the size of the characters is measured. Then, the size of the punch hole is divided by the size of the character. Furthermore, a predetermined number may be added to the divided value. This is because if a punch hole causes part of a character to be missing, it is expected that more characters than the divided value will be missing due to the punch hole.

In step S504, a user interface is displayed so that the user can input characters.
For example, the user interface shown in FIG. 6 is displayed. FIG. 6 is an explanatory diagram showing an example of processing according to this embodiment. The screen 600 displays an image display area 610, a comment display area 630, and a virtual keyboard display area 640. An input area 620 is displayed in the image display area 610. The input area 620 is displayed so as to surround at least the punch hole. Comment display area 630 displays an explanation of character input. For example, the message "Touch □ to enter characters" is displayed. In step S508, a virtual keyboard is displayed in the virtual keyboard display area 640.

In step S506, peripheral character processing is performed. The detailed process of step S506 will be described later using a flowchart shown in the example of FIG.
In step S508, when the user touches a character input location, a virtual keyboard is displayed on the user interface and the user is prompted to input.
In step S510, an image after punch holes are removed is generated based on the content input by the user.

FIG. 7 is a flowchart showing an example of processing according to this embodiment. This shows an example of the process in step S506 of the flowchart shown in the example of FIG.
In step S702, peripheral images are extracted. For example, you can generate a rectangular area surrounding the punched hole image and extract a surrounding image with a size that is obtained by multiplying that rectangular area by a predetermined magnification, or you can extract characters that are close to the punched hole image. At least one character or more may be extracted.
In step S704, the type of character, color, size, or font of the character is extracted from the surrounding image. Targeting the characters in the surrounding image extracted in step S702, the type, color, size, or font of the characters is extracted using conventional techniques.

In step S706, character properties are set. This property is the type, color, size, or font of the character extracted in step S704.
Depending on the type of character, a virtual keyboard is displayed in the virtual keyboard display area 640 shown in the example of FIG. For example, if the character type is numeric, a virtual keyboard for numbers is displayed, if the character type is alphabetic, a virtual keyboard for alphabets is displayed, and if the character type is Japanese (e.g. , kanji, hiragana, and katakana), you can display a virtual keyboard for Japanese.
Set the characters input by the user according to the color, size, or font of the characters. In other words, characters with the same attributes as the characters surrounding the punched hole are input by the user. Therefore, the sense of incongruity between conventional characters and characters input by the user is reduced.

FIG. 8 is an explanatory diagram showing an example of processing according to this embodiment. This is an example of processing for generating an input area 620 to be displayed in the image display area 610 shown in the example of FIG. 6 .
A punch hole area 810 and an input area 820 are displayed in the peripheral image 800. Input area 820 is generated as follows. A rectangle is generated so as to include at least the punch hole area 810 and further surround black pixels that are in contact with the punch hole area 810, and is set as an input area 820. The user inputs characters into the input area 820. In the example of FIG. 8, "AB" is input into the input area 820. The punch hole is corrected using the input area 820 in which characters have been input. In the example of FIG. 8, the punch hole area 810 is deleted, resulting in the character string "ABCDE".

FIG. 9 is a flowchart showing an example of processing according to this embodiment. This shows an example of the process in step S408 of the flowchart shown in the example of FIG. 4.
In step S902, a plurality of padding data are created. For example, an image similar to the image surrounding the punch hole is searched within the target page image. The area to be searched has the same size and shape as the punched hole. Specifically, as shown in the example of FIG. 10A, a filling candidate 1020a, a filling candidate 1020b, and a filling candidate 1020c are extracted as images similar to the surroundings of the punched hole 1010.

In step S904, an image is generated in which the punch holes are filled with the created data.
In step S906, all generated images are displayed on the user interface for the user to select. Specifically, as shown in the example of FIG. 10(b), a candidate image 1030a, a candidate image 1030b, and a candidate image 1030c are generated. The original punch hole area 1040a of the candidate image 1030a is obtained by applying the hole filling candidate 1020a to the punch hole 1010. The original punch hole area 1040b of the candidate image 1030b is obtained by applying the hole filling candidate 1020b to the punch hole 1010. The original punch hole area 1040c of the candidate image 1030c is obtained by applying the hole filling candidate 1020c to the punch hole 1010.

In step S908, if the user desires to process the image, the image is processed. Specifically, as shown in the example of FIG. 11(a), assume that the user selects candidate image 1030b. Therefore, the user specifies area 1110 as the area to be deleted. After the area 1110 is deleted, the image becomes a modified image 1150 shown in the example of FIG. 11(b).
Note that as a further correction method, a color value may be input for each pixel, although this increases the burden on the user.

FIG. 12 is a flowchart showing an example of processing according to this embodiment. Step S406 of the flowchart shown in the example of FIG. 4 classifies into two types, but step S1206 of the flowchart shown in the example of FIG. 12 classifies into four types.
In step S1202, punch holes are detected from the image data.
In step S1204, it is determined whether or not there is a punched hole. If there is, the process advances to step S1206; otherwise, the process ends (step S1299).

In step S1206, the classification of the vicinity of the punch hole is determined. If it is a white background, the process advances to step S1208, if it is a solid background, the process advances to step S1210, if it is a photograph, the process advances to step S1212, and if it is a text, the process advances to step S1214.

In step S1208, the punch holes are filled with white pixels.
In step S1210, the punch hole is filled with the average value of surrounding pixels. Note that other than the average value, other statistical values may be used, such as the mode or median value.

In step S1212, photo processing is performed. The detailed process of step S1212 has been described above using the flowchart shown in the example of FIG.
In step S1214, character processing is performed. The detailed process of step S1214 has been described above using the flowchart shown in the example of FIG.

An example of the hardware configuration of the image processing apparatus 100 according to the present embodiment will be described with reference to FIG. 13. The configuration shown in FIG. 13 is configured by, for example, a personal computer, and shows an example of a hardware configuration including a data reading section 1317 such as a scanner, and a data output section 1318 such as a printer.

The CPU (abbreviation for Central Processing Unit) 1301 includes the various modules described in the above embodiments, namely, the image data acquisition module 105, the punch hole detection module 110, the peripheral information classification module 115, the user interface module 120, and the punch hole detection module 110. This is a control unit that executes processing according to a computer program that describes the execution sequence of each module such as the correction module 125 and the image data output module 130.

A ROM (abbreviation for Read Only Memory) 1302 stores programs used by the CPU 1301, calculation parameters, and the like. A RAM (abbreviation for Random Access Memory) 1303 stores programs used in the execution of the CPU 1301, parameters that change as appropriate during the execution, and the like. These are interconnected by a host bus 1304 composed of a CPU bus and the like.

The host bus 1304 is connected via a bridge 1305 to an external bus 1306 such as a PCI (abbreviation for Peripheral Component Interconnect/Interface) bus.

A keyboard 1308 and a pointing device 1309 such as a mouse are devices operated by an operator. The display 1310 is a liquid crystal display device, CRT (abbreviation for cathode ray tube), or the like, and displays various information as text or image information. Alternatively, a touch screen or the like having the functions of both the pointing device 1309 and the display 1310 may be used. In that case, the keyboard function can be realized by drawing a keyboard (also called a software keyboard, screen keyboard, etc.) on the screen (for example, a touch screen) using software, without having to physically connect it like the keyboard 1308. The function of the keyboard may also be realized by using the keyboard function.

The HDD (abbreviation for Hard Disk Drive) 1311 includes a built-in hard disk (in addition to the hard disk, a flash memory or the like may be used), drives the hard disk, and records or reproduces programs and information to be executed by the CPU 1301. The HDD 1311 stores acquired image data, data resulting from processing by the punch hole detection module 110, peripheral information classification module 115, user interface module 120, punch hole correction module 125, and the like. Furthermore, various other data, various computer programs, etc. are stored.

The drive 1312 reads data or programs recorded on the attached removable recording medium 1313 such as a magnetic disk, optical disk, magneto-optical disk, or semiconductor memory, and transfers the data or programs to the interface 1307 and the external bus 1306. , a bridge 1305 , and a RAM 1303 connected via a host bus 1304 . Note that the removable recording medium 1313 can also be used as a data recording area.

The connection port 1314 is a port for connecting an external connection device 1315, and has connections such as USB and IEEE1394. The connection port 1314 is connected to the CPU 1301 and the like via an interface 1307, an external bus 1306, a bridge 1305, a host bus 1304, and the like. The communication unit 1316 is connected to a communication line and executes data communication processing with the outside. The data reading unit 1317 is, for example, a scanner, and executes document reading processing. The data output unit 1318 is, for example, a printer, and executes output processing of document data.

Among the embodiments described above, those based on computer programs are implemented by loading the computer program, which is software, into the system with this hardware configuration, and the software and hardware resources cooperate to implement the embodiments described above. is realized.
Note that the hardware configuration of image processing apparatus 100 shown in FIG. 13 shows one example of the configuration, and this embodiment is not limited to the configuration shown in FIG. 13. Any configuration that is executable is sufficient. For example, as a processor, a GPU (abbreviation for Graphics Processing Unit, including GPGPU (abbreviation for General-Purpose computing on Graphics Processing Units)) may be used, or some modules may be configured as dedicated hardware (for example, for a specific purpose). It consists of integrated circuits (specific examples include ASICs (application specific integrated circuits)) and reconfigurable integrated circuits (specific examples include FPGAs (field-programmable gate arrays)). Alternatively, some of the modules may be located in an external system and connected via communication lines, or a plurality of the systems shown in FIG. 13 may be connected to each other via communication lines and operate cooperatively with each other. In addition to personal computers, in particular, mobile information communication equipment (mobile information communication equipment includes mobile phones, smartphones, mobile devices, wearable computers, etc.), information appliances, robots, copiers, faxes, scanners, printers, It may be incorporated into a multifunction device (a multifunction device is an image processing device having two or more functions such as a scanner, a printer, a copying machine, and a fax).

Note that in the above embodiments, the processor refers to a processor in a broad sense, and includes a general-purpose processor (eg, CPU, etc.) and a dedicated processor (eg, GPU, ASIC, FPGA, programmable logic device, etc.).
Further, the operations of the processor in the above embodiments may be performed by one processor, or may be performed not only by one processor, but also by multiple processors located at physically distant locations working together. It may be something that is done. Further, the order of each operation of the processor is not limited to the order described in the above embodiments, and may be changed as appropriate.

Note that the program described above may be provided by being stored in a recording medium, or may be provided by means of communication. In that case, for example, the program described above may be regarded as an invention of "a computer-readable recording medium on which a program is recorded."
A "computer-readable recording medium on which a program is recorded" refers to a computer-readable recording medium on which a program is recorded, which is used for program installation, execution, program distribution, etc.
Examples of recording media include digital versatile discs (DVDs), such as "DVD-R, DVD-RW, DVD-RAM, etc.", which are the standards established by the DVD Forum, and DVD+RW, which is the standard established by the DVD Forum. standard "DVD+R, DVD+RW, etc.", compact disc (CD), read-only memory (CD-ROM), CD recordable (CD-R), CD rewritable (CD-RW), etc., Blu-ray disc ( Blu-ray (registered trademark) Disc), magneto-optical disk (MO), flexible disk (FD), magnetic tape, hard disk, read-only memory (ROM), electrically erasable and rewritable read-only memory (EEPROM (registered trademark) )), flash memory, random access memory (RAM), SD (abbreviation for Secure Digital) memory card, etc.
The entire program or a portion thereof may be recorded on the recording medium and stored or distributed. Also, by communication, for example, a local area network (LAN), a metropolitan area network (MAN), a wide area network (WAN), a wired network used for the Internet, an intranet, an extranet, etc., or wireless communication. It may be transmitted using a transmission medium such as a network or a combination thereof, or it may be carried on a carrier wave.
Further, the above program may be part or all of another program, or may be recorded on a recording medium together with a separate program. Further, the information may be divided and recorded on a plurality of recording media. Further, the data may be recorded in any format, such as compression or encryption, as long as it can be restored.

100... Image processing device 105... Image data acquisition module 110... Punch hole detection module 115... Peripheral information classification module 120... User interface module 125... Punch hole correction module 130... Image data output module 200... Image forming device 205... Scan module 220 ...Printing module 230...Information processing device 240...Image forming device 250...Information processing device 290...Communication line

Claims (2)

Equipped with a processor,
The processor includes:
If the area around the punch hole in the image is text , the user is allowed to enter text into the punch hole in the image, and the punch hole is corrected using the text;
If the area around the punch hole in the image is other than text, the punch hole is corrected by allowing the user to select from image candidates, and using the selected image candidate ,
Extraction of the area around the punch hole is performed by extracting a peripheral image having a size obtained by multiplying a rectangular area surrounding the image of the punch hole by a predetermined magnification,
Extraction of the image candidates involves searching for images around the punched hole extracted from within the image, and extracting image candidates having the same size and shape as the punched hole.
Image processing device. computer,
If the area around the punch hole in the image is text , the user is allowed to enter text into the punch hole in the image, and the punch hole is corrected using the text;
If the area around the punch hole in the image is other than text, the user is allowed to select from image candidates, and the selected image candidate functions as a correction means for correcting the punch hole ;
Extraction of the area around the punch hole is performed by extracting a peripheral image having a size obtained by multiplying a rectangular area surrounding the image of the punch hole by a predetermined magnification,
Extraction of the image candidates involves searching for images around the punched hole extracted from within the image, and extracting image candidates having the same size and shape as the punched hole.
Image processing program.

Images