JP2012190357A - Image processing device, image processing method and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an image processing device that hardly causes incomplete correction and incomplete checks when a person corrects or checks the result of OCR processing on handwritten characters.SOLUTION: An image processing device includes: a display unit; a character determination unit for recognizing a character candidate area in image data, determining whether or not a character in the character candidate area is a handwritten character, and outputting the determination result as a handwritten character determination result when determining that the character is a handwritten character; a character data processing unit for associating the handwritten character determined by the character determination unit with a character pattern of a character recognition candidate, and outputting the association result as a handwritten character recognition result; a display control unit for displaying the image data in a first screen area of the display unit, displaying the handwritten character determination result in a second screen area of the display unit, and displaying the character pattern of the handwritten character recognition result in a third screen area of the display unit; and a correction unit for changing the character pattern in response to correction of the character pattern displayed in the third screen area on the basis of correction instruction by a user.

Description

  The present invention relates to an image processing apparatus, an image processing method, and a program for recognizing handwritten characters and printed characters.

  2. Description of the Related Art Conventionally, there is image processing for recognizing handwritten characters and type characters from image data, so-called optical character recognition (OCR: Optical Character Recognition) processing. The accuracy of OCR processing for handwritten characters is significantly worse than for printed characters because the handwriting varies from person to person. Therefore, in the technique of Patent Literature 1, the handwritten character search accuracy is improved by distinguishing and recognizing handwritten characters and printed characters.

  However, if a process different from the OCR process of type is performed as the OCR process of handwritten characters as in Patent Document 1, it is necessary to store a plurality of OCR process results as candidates for one handwritten character. The amount of OCR processing results stored for each handwritten character is enormous. Moreover, if the recognition accuracy of handwritten characters is improved by storing many patterns as character patterns of handwritten characters, it is necessary to store many character patterns of handwritten characters. Become enormous. Thus, when trying to improve the OCR processing accuracy for handwritten characters, the load on the OCR processing becomes enormous and the processing time becomes long. In addition, when an image processing apparatus that takes less time for OCR processing is to be manufactured, it is necessary to employ a central processing unit (CPU) having a high calculation speed or a memory having a large capacity. For this reason, there is a problem that the manufacturing cost is high to realize this.

  In addition, when a character recognition process is performed on a manuscript with a large proportion of handwritten characters, a large processing load is applied. For this manuscript, an OCR processing device is introduced to improve efficiency. But it has little effect. For example, in a manuscript with a high percentage of handwritten characters, a correction instruction (such as proofreading of the manuscript) may be written by hand on the original type document. When an OCR processing apparatus is introduced for a document having a large proportion of handwritten characters as described above, it is effective to check or correct the OCR processing result of handwritten characters by humans, and there is a need for it.

JP 2002-312398 A

  However, when a person checks and corrects the OCR processing result of a handwritten character, there arises a problem that an omission of confirmation or correction is more likely to occur as the amount of handwritten characters increases.

  Therefore, the present invention has been made in view of such a situation, and provides an image processing apparatus that is less likely to cause omission of correction and omission of confirmation when a person corrects or confirms an OCR processing result of a handwritten character. The purpose is to do.

  In order to achieve the above object, an image processing apparatus according to an aspect of the present invention is an image processing apparatus that performs processing on image data obtained by optically reading a document, the display unit, and the image The character candidate area is recognized from the data, it is determined whether or not the character included in the character candidate area is a handwritten character, and when it is determined to be a handwritten character, the determination result is output as a handwritten character determination result. A character determination unit, a character data processing unit that associates the character determined to be a handwritten character by the character determination unit with a character pattern that is a candidate for character recognition, and outputs the associated result as a handwritten character recognition result; The image data is displayed in the first screen area of the display unit, and the handwritten character determination result output by the character determination unit is different from the first screen area in the second of the display unit. A character pattern associated with the character data processing unit among the handwritten character recognition results corresponding to the handwritten character determination result selected by the user in the second screen region. Based on the display control unit to be displayed in the three-screen area and the correction instruction from the user, the correction of the character pattern displayed in the third screen area is accepted, and the character data among the handwritten character recognition results A correction unit that changes the character pattern associated by the processing unit.

  According to this, the recognized handwritten character is extracted from the image data obtained by optically reading the document, and the recognized handwritten character determination result is different from the first screen area where the image data is displayed. It is displayed in the second screen area. And a correction part changes the character pattern (what is called OCR process result) linked | related by the character data processing part displayed on a 3rd screen area | region based on the instruction | indication of a correction from a user.

  For this reason, the user can compare and correct the handwritten character recognition result and the handwritten character in the original image data. Further, since only handwritten characters that are likely to be erroneously recognized are extracted and displayed in the second screen area, it is possible to prevent omission of confirmation or omission of erroneously recognized OCR processing results. Thereby, the user can work efficiently when correcting the OCR processing result.

  Preferably, the character determination unit further determines whether or not the character included in the character candidate area is a type, and determines that the character is a type, and the result of the determination is a type determination result. The character data processing unit further associates the type determined by the character determination unit with the character pattern, outputs the associated result as a type recognition result, and the display control unit further includes: The type determination result output by the character determination unit is displayed in the second screen area, and the character pattern associated by the character data processing unit among the type recognition results is displayed in the third screen area.

  According to this, the character determination unit outputs the type determination result also for the character candidate area of the type, and the character data processing unit outputs the type recognition result. In other words, the image processing apparatus performs OCR processing on type characters.

  For this reason, the extracted character determination result can be displayed in a selectable state with the handwritten character determination result. Further, for example, the correction unit may correct the OCR processing result of the type in the type recognition result based on the correction instruction from the user. In this way, the user can check or correct the OCR processing result even for the type.

  Preferably, the image processing apparatus further includes a first type recognition in which the first character determination result determined by the character determination unit based on the first image data and the character pattern are associated by the character data processing unit. A result and a second type recognition result obtained by associating the second character determination result determined by the character determination unit based on the second image data and the character pattern by the character data processing unit are a predetermined ratio or more. When the type match determination unit determines that the first type recognition result and the second type recognition result match at the predetermined ratio or more, A first handwritten character recognition in which the first handwritten character determined by the character determining unit based on the first image data and the character pattern are associated by the character data processing unit. Integrated handwriting by integrating the result and the second handwritten character recognition result in which the second handwritten character determined by the character determining unit based on the second image data and the character pattern are associated by the character data processing unit A data integration unit that outputs as a character recognition result, wherein the display control unit displays the integrated handwritten character recognition result in the second screen region, and the integrated handwriting selected by the user in the second screen region Among the character recognition results, the character pattern associated with the character data processing unit is displayed in the third screen area, and the correction unit is configured to display the first handwritten character recognition result based on a correction instruction from a user. Accepting correction of the first character pattern associated by the character data processing unit in the integrated handwritten character recognition result Change the first character pattern associated by the data processing unit, or accept the correction of the second character pattern associated by the character data processing unit in the second handwritten character recognition result, In the integrated handwritten character recognition result, the second character pattern associated with the character data processing unit is changed.

  Among the correctors who write handwritten characters in the document, there are correctors who correct only the content of the text, and there are correctors who correct only typographical errors. In this way, when there are multiple correctors, the same document is printed multiple times, and when multiple correctors modify each document by handwriting, some of the documents are corrected by multiple correctors. It is more efficient than reusing and correcting.

  When a handwritten correction portion (character string of handwritten characters) is extracted for each of a plurality of documents, the user confirms an OCR processing result based on a result processed for each of the plurality of documents even though they are the same document. It must be corrected and work efficiency is poor. In addition, when there are a plurality of correctors, image data is also created by the number of correctors, and therefore the storage destination of the plurality of image data may be different for each image data. There is a risk that all corrections by the corrector cannot be reflected. Furthermore, since a plurality of image data are created for the same document, a large amount of capacity for storing the data after image processing is consumed.

  In order to solve this problem, when a plurality of type recognition results for a plurality of image data match at a predetermined ratio or more, a plurality of handwritings for a plurality of image data output by the character data processing unit Integrate character recognition results. In other words, the image processing apparatus regards that a plurality of image data has been corrected for the same document when the OCR processing result of the type matches at a predetermined ratio or more, and the OCR of the handwritten character is considered. Integrate processing results. And a display control part displays the integrated integrated handwritten character recognition result on the 2nd screen area | region of a display part.

  As a result, even when there are a plurality of correctors, the user can compare and correct the integrated handwritten character recognition result with the handwritten characters in the original image data. Further, since only handwritten characters that are likely to be erroneously recognized are extracted and displayed in the second screen area, it is possible to prevent omission of confirmation or omission of erroneously recognized OCR processing results. Thereby, the user can work efficiently when correcting the OCR processing result. In addition, since handwritten writing by a plurality of correctors can be displayed in an integrated manner, work efficiency for one document can be improved.

  Preferably, the third screen area is a screen area different from the first screen area and the second screen area.

  According to this, as a screen area for displaying the correction status of the OCR processing result corresponding to the handwritten character determination result displayed in the second screen area, it is displayed in a third screen area, which is another screen area. Thereby, the user can confirm or correct the OCR processing result of the handwritten character while referring to the handwritten character based on the original image data.

  Preferably, the display control unit displays the first handwritten character recognition result on the second screen area in a first display form, and the second handwritten character recognition result is different from the first display form. It is displayed in the second screen area in the two display mode.

  According to this, the display control unit displays the first handwritten character determination result and the second handwritten character determination result, which are handwritten characters written by a plurality of correctors, for example, in the second screen area in different display forms. For this reason, for example, when a plurality of correctors are assigned roles for each type of correction content, such as correction of text content and correction of typographical errors as described above, correction by handwritten characters for each role Can be grasped by the user. Thereby, for example, when there is a restriction such as a time limit for correcting the OCR processing result of the handwritten character, the user selects correction by the handwritten character recognized as a role having a high priority and performs OCR of the handwritten character. The processing result can be corrected.

  Preferably, the character determination unit further outputs a display position and a display range of a character candidate area determined to include a handwritten character as the handwritten character determination result.

  According to this, a character determination part outputs the display position and display range of a character candidate area | region as a handwritten character determination result. For this reason, for example, when the user selects a character determination result displayed in the second screen area, it is used for processing such as changing the display form of the character candidate area corresponding to the selected character determination result. Can do.

  Preferably, the character determination unit further determines whether or not the character included in the character candidate area is a type, and determines that the character is a type, the result of the determination is the image data. Are output as a type determination result together with the display position of the type and the display range of the type, and the display control unit is determined by the character determination unit to include a type based on the handwritten character determination result and the type determination result The character candidate area is displayed on the first screen area in a third display form, and the character candidate area determined by the character determination unit to include a handwritten character is displayed in a fourth display form different from the third display form. The first screen area is displayed.

  According to this, a display control part displays the display form of the character candidate area | region of the displayed type character, and the display form of the character candidate area | region of a handwritten character on a 1st screen area | region with a different display form. For this reason, the user can instantly distinguish between a character candidate area for printed characters and a handwritten character candidate area in the image data in the first screen area.

  In addition, preferably, the display control unit, when a user makes a first selection with respect to a handwritten character determination result displayed in the second screen area, the handwriting that is the target of the first selection A character candidate area corresponding to the character determination result is displayed in the first screen area in the fifth display form.

  According to this, when the user makes the first selection with respect to the handwritten character determination result displayed in the second screen area, the character candidate corresponding to the handwritten character determination result in which the first selection has been made. The area is displayed on the first screen area in the fifth display mode. Specifically, the user checks, for example, a check box corresponding to the handwritten character determination result as the first selection by the correction unit from the handwritten character determination results displayed in the second screen area. In this case, a display control part changes the display form of the character candidate area | region corresponding to the checked handwritten character determination result, and displays it on a 1st screen area | region.

  For this reason, for example, the user changes the display of the character candidate area in which the first selection has been performed by performing the first selection on the handwritten character determination result for which the confirmation or correction of the OCR processing result has been completed. be able to. Thereby, the user can instantly identify the character candidate region corresponding to the handwritten character determination result for which the confirmation or correction of the OCR processing result has been completed, and prevent the erroneous recognition or the omission of correction of the erroneously recognized OCR processing result. Can do. For this reason, the user can work efficiently when correcting the OCR processing result.

  Preferably, the display control unit further sets the second selection target when the user makes a second selection on the handwritten character determination result displayed in the second screen area. The candidate character area corresponding to the handwritten character determination result is displayed on the first screen area in a sixth display form different from the fifth display form.

  According to this, when the user performs the second selection on the handwritten character determination result displayed in the second screen area, the character candidate corresponding to the handwritten character determination result on which the second selection has been performed. The area is displayed on the first screen area in a sixth display form different from the fifth display form. Specifically, as a second selection by the correction unit from among the handwritten character determination results displayed in the second screen area, for example, the user clicks and highlights the handwritten character determination result with a pointing device such as a mouse. Let In this case, the display control unit further changes the display of the character candidate area corresponding to the highlighted handwritten character determination result to a display different from the fifth display form, and displays it in the first display area.

  For this reason, for example, the user selects a character candidate area corresponding to the selected handwritten character determination result in the first screen area by, for example, selecting a handwritten character determination result by clicking a pointing device such as a mouse. Can be identified instantly. Thereby, the user can work efficiently when correcting the OCR processing result.

  Preferably, the image processing apparatus further includes an image acquisition unit that optically acquires the image data from a document.

  According to this, since the image acquisition unit is further provided, the image data can be acquired independently from the document without separately preparing an image acquisition device such as a scanner or a digital camera. Can be processed.

  Preferably, the image processing apparatus further includes a storage unit that stores the character pattern.

  According to this, since the storage unit is further provided, the OCR processing can be performed independently without preparing a separate storage device such as a hard disk or a memory, and the processing corresponding to claims 1 to 10 can be performed. it can.

  In addition, the present invention can be realized not only as such an image processing apparatus but also as a method using each operation of a characteristic processing unit included in the image processing apparatus as a step. It can also be realized as an integrated circuit that performs processing of the processing unit. Furthermore, the present invention can also be realized as a program for causing a computer to execute the above steps. Such a program can also be distributed via a recording medium such as a CD-ROM or a transmission medium such as the Internet. In addition, the present invention can be realized as a network complex machine incorporating such an image processing apparatus.

  According to the image processing apparatus of the present invention, the user can prevent omission of confirmation or omission of erroneously recognized OCR processing results, and can efficiently work when correcting the OCR processing results.

It is a figure which shows the structure of the workflow system containing the image processing apparatus in one Embodiment of this invention. 1 is a block diagram illustrating a hardware configuration of a network multifunction peripheral according to an embodiment of the present invention. FIG. It is a block diagram which shows the hardware constitutions of the terminal device in one Embodiment of this invention. It is a block diagram which shows the function structure which concerns on the image processing of the image processing apparatus in one Embodiment of this invention. It is a figure which shows an example of the character recognition result data in one Embodiment of this invention. It is a flowchart which shows the process sequence in the image process of the image data by the workflow system in one Embodiment of this invention. It is a figure which shows an example of the correction work screen in one Embodiment of this invention. It is a flowchart which shows an example of the work procedure of the correction operation | work when the user corrects character recognition result data in one Embodiment of this invention. It is a block diagram which shows the function structure which concerns on the integration process of the image processing apparatus in one Embodiment of this invention. It is a flowchart which shows the process sequence in the integration process of the character recognition result data in the workflow system in one Embodiment of this embodiment. It is a block diagram which shows the function structure which concerns on the image processing of the image processing apparatus in other embodiment of this invention.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

  First, an outline of a configuration of a workflow system including an image processing apparatus according to an embodiment of the present invention will be described with reference to FIGS. 1 and 2.

  This workflow system includes a creator, a corrector, and an assistant who assists the creator and the corrector. The creator creates a document printed by a PC or a word processor (hereinafter referred to as a print document). The corrector manually corrects the print document created by the creator. At this time, the document corrected by the corrector is set as the corrected document. The assistant uses the scanner function of the network multifunction peripherals 1 and 2 and the digital camera 7 to acquire an image file including a handwritten character created by the corrector, and performs an OCR process with the image processing apparatus of the present application. Do. The assistant checks whether the recognition processing result of handwritten characters that are easily misrecognized matches the content written in the original document by performing OCR processing on the document. Correct the recognition processing result. That is, since the workflow system of the present embodiment is mainly used by an assistant, the assistant will be described as a user from the following.

  FIG. 1 is a diagram illustrating an example of a configuration of a workflow system including an image processing apparatus according to the present embodiment.

  As shown in the figure, this workflow system includes network multifunction peripherals 1 and 2, terminal devices 3 and 4, PSTN (Public Switched Telephone Networks) 5, and LAN (Local Area Network) 6. Is done. This workflow system is a system capable of performing OCR processing of document image data 53 acquired from the network multifunction peripherals 1 and 2 or the digital camera 7.

  The terminal devices 3 and 4 are computers that function as image processing devices in the present embodiment. In the present embodiment, the terminal devices 3 and 4 are personal computers used by users (mainly assistants) who use the network multifunction peripherals 1 and 2.

  The network multifunction device 1 transmits, for example, a document read by the scanner by facsimile to the network multifunction device 2 via the PSTN 5, transmits it to the terminal devices 3 and 4 via the LAN 6, and prints out with a built-in printer. You can do it.

  FIG. 2 is a block diagram illustrating a hardware configuration of the network multifunction peripheral 1 according to the present embodiment.

  As shown in the figure, the network MFP 1 includes a CPU (Central Processing Unit) 10, a ROM (Read Only Memory) 11, a RAM (Random Access Memory) 12, a modem 13, an NCU (Network Control Unit) 14, and an operation panel. 15, a display 16, a scanner 17, a printer 18, and a LAN I / F (LAN Interface) 19.

  The CPU 10 controls the entire network multifunction peripheral 1 by executing a control program 50 stored in the ROM 11.

  The ROM 11 is a read-only memory that holds a control program 50 executed by the CPU 10.

  The RAM 12 is a readable / writable memory that holds work data used when the CPU 10 executes the control program 50, image data obtained from the scanner 17, and the like.

  The modem 13 modulates the image data and the like held in the RAM 12 into a facsimile signal and transmits it, and demodulates the facsimile signal received from the outside into line data. The modem 13 is a fax modem compliant with the G3 standard, for example.

  The NCU 14 is a network control device that controls the connection between the modem 13 and the PSTN 5.

  The operation panel 15 is a touch panel that receives an operation from the user.

  The display 16 is a display device that displays an operation guide to the user or an operation state of the network multifunction peripheral 1, and is, for example, an LCD (Liquid Crystal Display).

  The scanner 17 is an image reading device, and generates image data by optically reading a document using a CCD under the control of the CPU 10.

  The printer 18 is a printing device, and prints out an image image represented by, for example, image data held in the RAM 12 under the control of the CPU 10.

  The LAN I / F 19 is a communication adapter that connects the network multifunction device 1 and the LAN 6, and transmits, for example, image data held in the RAM 12 to the terminal device 3 or the like under the control of the CPU 10.

  FIG. 3 is a block diagram showing a hardware configuration of terminal devices 3 and 4 in the present embodiment.

  As shown in FIG. 3, the terminal device 3 mainly includes a CPU 20, a hard disk 21, a RAM 22, an input device 23, a display 24, a LAN I / F 25, and a USB I / F (Universal Serial Bus Interface) 26. Although only the terminal device 3 is described here, the terminal device 4 has the same configuration.

  The CPU 20 controls the entire terminal device 3 by executing an application program stored in the hard disk 21.

  The hard disk 21 is a program such as an image processing program 51, character pattern data 52, image data 53 obtained from a scanner or digital camera, character determination result data 54, character recognition result data 55, integrated character recognition result data 60, integrated character recognition. Result data 60 and the like are stored.

  The “image processing program 51” is an application program that performs image processing to be described later on image data. The “character pattern data 52” is data of character patterns that are candidates for character recognition for performing OCR processing of handwritten characters or printed characters. The “character pattern data 52” includes a handwritten character pattern for performing OCR processing of handwritten characters and a type pattern for performing OCR processing of printed characters, although not shown. In other words, the OCR processing for type characters and the OCR processing for handwritten characters are performed using different character patterns. The “image data 53” is image data acquired from a document by an image acquisition device such as the scanner 17 or the digital camera 7 of the network multifunction peripheral 1. The “character recognition result data 55” is data created during image processing by the character data processing unit 32 described later. The “integrated character recognition result data 60” is data created during the integration process by the data integration unit 38 described later.

  The RAM 22 is a readable / writable memory that temporarily holds the character determination result data 54 used when the CPU 20 executes a program stored in the hard disk 21, the matching data table 59, and the image data 53 obtained from the scanner 17. Memory. The “character determination result data 54” is data created during image processing by the character determination unit 31 described later. The “match data table 59” is data created by a type match determination unit 37 described later.

  The input device 23 is a device that receives an operation from a user, and is a keyboard or a pointing device such as a mouse or a touch panel.

  The display 24 is a CRT display, a liquid crystal display, a plasma display, an organic EL display, or the like for displaying data transmitted from the RAM 22 as characters or images.

  The LAN I / F 25 is a communication adapter that connects the terminal device 3 and the LAN 6. The LAN I / F 25 transmits, for example, image data 53, character recognition result data 55, or integrated character recognition result data 60 held in the hard disk 21 of the terminal device to the terminal device 4 under the control of the CPU 20. The LAN I / F 25 also receives image data 53, character recognition result data 55, and the like transmitted from the terminal device 4 and the network multifunction device 1 to the terminal device 3.

  The USB I / F 26 is a communication adapter that connects the terminal device 3 to the digital camera 7 or the like. Through the USB I / F 26, for example, the image data 53 acquired by the digital camera 7 is received by the terminal device 3 and stored in the hard disk 21.

  FIG. 4 is a block diagram illustrating a functional configuration related to image processing of the image processing apparatus 100 according to the present embodiment. That is, of the functions of the network multifunction peripheral 1 and the terminal devices 3 and 4 that are exhibited by the hardware configuration shown in FIGS. 2 and 3, the functions mainly related to the image processing of the image processing apparatus 100 of the present embodiment. It is a block diagram which shows a structure.

  Functionally, the image processing apparatus 100 includes a storage unit 30, a character determination unit 31, a character data processing unit 32, a display control unit 33, a correction unit 34, an input unit 35, and a display unit 36. And is connected in a communicable state (for example, LAN, USB, etc.) with the network multifunction device 1 or the digital camera 7 as the image acquisition device 110.

  The storage unit 30 is realized by the hard disk 21, the RAM 22, and the like in FIG. The storage unit 30 stores character pattern data 52. The storage unit 30 also includes image data 53 transmitted from the image acquisition device 110, character determination result data 54 output by a character determination unit 31 described later, character recognition result data 55 output by a character data processing unit 32, and the like. Remember.

  The character determination unit 31 is a determination unit realized by the CPU 20, the program, and the like in FIG. The character determination unit 31 performs a partial region extraction process and a hand activity determination process based on the image data 53 stored in the storage unit 30. The partial area extraction process is a process for recognizing a character candidate area from image data. For example, the character candidate area of the image data 53 is decomposed for each partial area that is a lump of characters such as one line, and a part of the image data This is processing for recognizing the position and range of an area. The hand activity determination process determines whether the character included in the partial area decomposed by the partial area extraction process is a handwritten character or a printed character, and outputs it as handwritten character determination result data when it is determined as a handwritten character. , A process of outputting as character determination result data when it is determined that the character is a character. Hereinafter, a character displayed as an image before the OCR process, such as a character included in the partial area decomposed by the partial area cutting process, is referred to as a “character image”.

  The handwritten character determination result data and the type determination result data output by the character determination unit 31 are output as character determination result data 54. That is, in the character determination result data 54, although not illustrated, for the handwritten character determination result data output by the character determining unit 31 for the partial region including the handwritten character image and the partial region including the printed character image. Type determination result data output by the character determination unit 31. The character determination unit 31 adds handwritten character attribute data or type attribute data to the determined partial area, and outputs the result as character determination result data 54. It should be noted that the character determination result data 54 output here is information on the position and range of the partial area derived by the partial area extraction process and the attribute data or type of handwritten characters with respect to the original image data 53. Is added to the attribute data.

  The character data processing unit 32 is a processing unit realized by the CPU 20, the program, etc. in FIG. The character data processing unit 32 performs character pattern association processing based on the character pattern data 52 stored in the storage unit 30 and the character determination result data 54 output by the character determination unit 31. The character pattern association process is a process of associating the character pattern data 52 with the character image included in each partial area of the character determination result data 54. In the character pattern association processing, processing is performed for each partial area, and a character string corresponding to the character image included in each partial area is extracted from the character pattern data, and is associated with the character image included in each partial area as character string data. It is done. That is, the character string data is a so-called OCR process. The character data processing unit 32 outputs the result of the character pattern association process as character recognition result data 55. The character recognition result data 55 output here is data obtained by adding character string data associated by the character pattern association process to the character determination result data 54.

  As shown in FIG. 5, the character recognition result data 55 is associated with the partial area ID, the partial area data size, the partial area position information, the partial area attribute data, the number of characters in the partial area, and the partial area. Holds string data. The “partial area ID” is an identifier for identifying the partial area distinguished in S10. The “partial area ID” is a number determined in ascending order of values calculated by a predetermined calculation based on the numerical value of the position information. In the present embodiment, for example, the start coordinates of the position information are determined in order from the smallest Y coordinate, and the numbers having the same Y coordinate are numbered in descending order from the smallest X coordinate. “Position information” is data indicating the position and range where the partial area is displayed in the image data, and is, for example, coordinate data as shown in FIG. In the coordinate data, a region surrounded by the start coordinate and the end coordinate is digitized by the X coordinate and the Y coordinate and expressed as position information. The coordinate data is, for example, a numerical value derived in units of pixels. The “character string data” is a character string associated with a character image included in the partial area based on the character pattern data. Although not shown, the character recognition result data 55 includes handwritten character recognition result data obtained by performing character pattern association processing on handwritten characters and type recognition result data obtained by performing character pattern association processing on characters. Character recognition result data 55 shown in FIG. 5 is obtained by integrating handwritten character recognition result data and type recognition result data.

  The display control unit 33 is a processing unit realized by the CPU 20, the program, and the like in FIG. The display control unit 33 displays the image data in the first screen area 40 of the display unit 36. Further, the display control unit 33 displays the character recognition result data 55 output by the character data processing unit 32 in the second screen area 41 of the display unit 36. Further, the display control unit 33 displays character string data corresponding to the character recognition result data 55 selected by the user operating the input unit 35 in the second screen region 41 in the third screen region 42 of the display unit 36. . The first screen area 40, the second screen area 41, and the third screen area 42 are screen areas that are displayed on the display unit 36, and are displayed on the screen of the display unit 36 as shown in FIG. It is a separate area.

  The correction unit 34 receives correction of character string data displayed in the third screen area 42 based on an input signal transmitted by the user operating the input unit 35, and each part of the character recognition result data 55. The character string data associated with the area is changed. For example, the character string data associated with the character image included in each partial area of the character candidate area extracted from the image data may become character string data different from the originally intended character due to erroneous recognition. is there. In such a case, the correction unit 34 can read the character recognition result data 55 by the user operating the input unit 35, and can correct the erroneously associated character pattern to the correct character pattern. The corrected character recognition result data 55 is overwritten and stored in the storage unit 30.

  The input unit 35 is realized by the input device 23 in FIG. As described above, the input unit 35 can be operated by the user to transmit an input signal to the display control unit 33 or the correction unit 34 to correct the character string data of the character recognition result data 55. it can.

  The display unit 36 is realized by the display 24 in FIG. The display unit 36 displays the character recognition result data 55 via the display control unit 33. That is, the display unit 36 displays the character recognition result data 55 before being corrected by the input unit 35, the character recognition result data 55 being corrected by the input unit 35, and the like, thereby correcting the character string data by the user. Support. As described above, the display unit 36 also displays the character image and the character string data of the character recognition result data 55 separately in the first screen area 40, the second screen area 41, and the third screen area 42. To do.

  Next, a processing procedure of image processing in the present embodiment configured as described above will be described.

  FIG. 6 is a flowchart showing a processing procedure in image processing of the image data 53 by the workflow system according to the present embodiment.

  As illustrated in FIG. 6, the character determination unit 31 starts image processing when image data 53 stored in the storage unit 30 is input to the character determination unit 31. The character determination unit 31 performs a partial area extraction process for each page of the input image data 53 (S10).

  Next, the character determination part 31 adds partial area ID and position information for every decomposed partial area (S11).

  And the character determination part 31 performs a hand activity determination process with respect to the character image contained in a partial area (S12).

  The attribute data of the handwritten character is added to the partial area determined to be a handwritten character by the character determining unit 31 in the hand activity determining process (S13). The character determination unit 31 outputs a partial region to which handwritten character attribute data is added as handwritten character determination result data. The handwritten character determination result data output by the character determination unit 31 is buffered in the storage unit 30.

  And the character data processing part 32 is based on the handwritten character pattern data of the character pattern data 52 stored in the memory | storage part 30, and the handwritten character determination result data output by the character determination part 31. Pattern association processing is performed (S14). The handwritten character pattern association process is a so-called OCR process for a partial region of handwritten characters. The handwritten character recognition result data output by the character data processing unit 32 is buffered in the storage unit 30.

  On the other hand, type attribute data is added to the partial area determined to be a type by the character determination unit 31 in the hand type determination process (S15). The character determination unit 31 outputs a partial area to which type attribute data is added as type determination result data. The character determination result data output by the character determination unit 31 is buffered in the storage unit 30.

  The character data processing unit 32 then performs type pattern association processing based on the type pattern data in the character pattern data 52 stored in the storage unit 30 and the type determination result data output by the character determination unit 31. (S16). This type pattern association process is a so-called OCR process for a partial region of type. The type recognition result data output by the character data processing unit 32 is buffered in the storage unit 30.

  The character determination unit 31 determines whether or not there is an unprocessed partial area that has not been subjected to the handwritten character association process or the type association process (S17). If there is an unprocessed partial area, image processing (S11 to S16) is performed again on the unprocessed partial area.

  When there is no unprocessed partial area, the character data processing unit 32 integrates handwritten character recognition result data and type recognition result data corresponding to all partial areas belonging to the image data of the page for which the character pattern association process has been completed. (S18).

  The character data processing unit 32 checks whether or not there is a page for which the character pattern association process has not been completed (S19). If there is a page of unprocessed image data, image processing (S10 to S18) is performed again.

  If there is no page of unprocessed image data, the character data processing unit 32 determines that the processing has been completed for all pages of the image data, and performs image processing (S10 to S18) performed on all pages. ) Is output as character recognition result data 55 (S20). The output character recognition result data 55 ends the image processing stored in the storage unit 30.

  The character recognition result data 55 stored in the storage unit 30 is read by the correction unit 34 and displayed on the display unit 36 when the correction unit 34 is operated by operating the input unit 35 by the user. Based on the character recognition result data 55, the correction unit 34 causes the display unit 36 to separately display the character image included in the partial region of the original image data 53 and the character string data associated with the character image. The correction unit 34 also receives correction of character string data associated with each partial area based on an input signal from the input unit 35.

  Below, based on FIG. 7, the screen displayed on the display part 36 by the process of the specific correction part 34 is demonstrated.

  The screen displayed on the display unit 36 is mainly divided into three screen areas: a first screen area 40, a second screen area 41, and a third screen area 42.

  The first screen area 40 is a main screen area, and is an image data display area that performs display based mainly on the original image data 53. In the first screen area 40, in addition to the original image data 53, a ruled line indicating each partial area and a partial area ID of each partial area are displayed. That is, the partial area decomposed by the character determination unit 31 is displayed in the first screen area 40 in a visualized state. Further, in the first screen area 40, the partial area to which the handwritten character attribute data is added and the partial area to which the type attribute data is added are displayed with different display forms. In the present embodiment, as shown in FIG. 7, the character candidate area of the handwritten attribute displayed in the first screen area 40 is red (displayed in gray in FIG. 7), and the type attribute displayed in the first screen area 40 The character candidate area is displayed in black.

  The second screen area 41 is a screen area that is displayed vertically (side-displayed) on the right side of the first screen area 40, and character string data of the character recognition result data 55 is displayed. In the second screen area 41, character string data associated with each partial area included in the image data of the page displayed in the first screen area 40 is displayed together with the partial area ID of the partial area.

  In the second screen area 41, character string data associated with all partial areas included in the image data of the page displayed in the first screen area 40 is displayed, and these character string data are displayed in the partial area. Displayed for each attribute data. For example, in FIG. 6, “all character strings”, “only handwriting”, and “only type” can be selected in the pull-down list 43. When the user selects “all character strings” from the pull-down list 43, character string data having attribute data of either handwritten characters or type characters is displayed among the plurality of character string data. All character patterns are displayed in the two-screen area 41. When the user selects “handwriting only” from the pull-down list 43, only character string data having handwritten character attribute data among a plurality of character string data is displayed in the second screen area 41. In addition, when the user selects “type only” from the pull-down list 43, only character string data having type attribute data among a plurality of character string data is displayed in the second screen area 41.

  In the second screen area 41, a check box 44 is displayed on the left side of the displayed character string data. The user can input a check mark in the check box 44 by selecting it with the input device 23 such as a pointing device. The partial area corresponding to the character string data in which a check mark is input in the check box 44 is displayed with a strikethrough in the display in the first screen area 40. Specifically, as shown in FIG. 7, when a check mark is input to the check box 44 of “2: Add!” In the second screen area 41, “2: Add! Strikethrough will be displayed in the display part of “!”.

  In the second screen area 41, when a character portion of a plurality of character string data to be displayed is selected by the input unit 35 (specifically, a pointing device such as a mouse), black and white are inverted as shown in FIG. Is displayed. Thus, the character string data corresponding to the selected character string data is displayed in the third screen area 42 in a modifiable state. Specifically, as shown in FIG. 7, when “6: this part...” In the second screen area 41 is selected and highlighted, the third character string data can be corrected in a third state. It is displayed in the screen area 42. Further, in the first screen area 40, the border of the ruled line indicating the partial area displayed as “6: This part drawing,” corresponding to the character string data selected and highlighted in the second screen area 41 is thick. It will be displayed in a frame.

  As described above, in the third screen area 42, the character string data selected and highlighted in the second screen area 41 by the input unit 35 (specifically, a pointing device such as a mouse) is displayed. In the third screen area 42, correction of the character string data selected in the second screen area 41 by the input unit 35 (specifically, a character input device such as a keyboard) is accepted. That is, when the character image included in the actual partial area and the associated character string data are different from each other due to misrecognition, the user replaces the character string data including the error in the third screen area 42 with the correct character. Can be modified to column data.

  FIG. 8 is a flowchart showing an example of the work procedure of the correction work when the user corrects the character recognition result data.

  First, when the user reads the character recognition result data 55 by the terminal devices 3 and 4, an image as shown in FIG. 7 is displayed on the display (S30).

  The user operates the input unit 35 to select “handwriting only” from the pull-down list 43 to display only character string data to which handwritten character attribute data is added in the second screen area 41 (S31). .

  The user operates the input unit 35 to select one of a plurality of character string data extracted in the second screen area 41 from the read character recognition result data 55 and to reverse-display it (S32).

  Then, the character string data selected and highlighted in the second screen area 41 is displayed in the third screen area 42 by the correction unit 34 in a state where the correction by the operation of the user input unit 35 is accepted. At the same time, in the first screen area 40, the border lines of the partial areas corresponding to the character string data selected and highlighted in the second screen area 41 are displayed as thick frames (S33).

  While viewing the character image displayed in the partial area in which the border of the ruled line is displayed as a thick frame in the first screen area 40, the user determines whether the character string data associated with the partial area is correct. The determination is made by comparing the character string data displayed in the screen area 42 with the character image (S34).

  When the user determines that the character string data associated with the partial area is correct, the user inputs a check mark in the check box 44 without performing correction input in the third screen area 42 (S35).

  When determining that the character string data associated with the partial area is incorrect, the user performs correction input in the third screen area 42 to correct the character string data, and then puts a check mark in the check box 44. Input (S36).

  The user determines whether or not check marks have been input to all the check boxes 44 corresponding to the character string data displayed in the second screen area 41 (S37). Here, the user repeats the character string data correction operation (S31 to S36) corresponding to the check box 44 in which no check mark is input when no check mark is input in all.

  When all the check boxes 44 are checked, the user determines that the correction work for the page has been completed, and confirms whether the correction work has been performed on all pages (S38). . When there is an uncorrected page, the user repeats the correction work (S31 to S37) for the uncorrected page.

  If the correction work has been performed on all pages, the user ends the correction work.

  The image processing apparatus 100 according to the present embodiment further performs integration processing as the image processing described above. The integration processing is performed on a plurality of image data read when a plurality of correctors modify the same document by handwriting, and the OCR processing is performed when the type parts of the documents are common. It is a process of integrating the results.

  FIG. 9 is a block diagram illustrating a functional configuration related to the integration processing of the image processing apparatus 100 according to the present embodiment. That is, FIG. 4 is a block diagram illustrating a configuration of functions related to integration processing of the image processing apparatus 100 among the functions of the network multifunction peripheral 1 and the terminal devices 3 and 4 that are exhibited by the hardware configuration illustrated in FIGS. 2 and 3. .

  Functionally, the image processing apparatus 100 includes a storage unit 30, a character determination unit 31, a character data processing unit 32, a display control unit 33, a correction unit 34, an input unit 35, a display unit 36, A type match determination unit 37 and a data integration unit 38 are provided. Hereinafter, only the type match determination unit 37 and the data integration unit 38, which are processing units different from the description of the functional configuration related to image processing, will be described.

  The type match determination unit 37 is a determination unit realized by a CPU, a program, and the like in FIG. The type match determination unit 37 determines whether or not the first type recognition result data and the second type recognition result data match at a predetermined ratio (for example, 90%) or more. The first character determination result data is data in which the character data processing unit 32 associates the first character determination result data determined by the character determination unit 31 with the character pattern based on the first image data. The second type recognition result data is data in which the second type determination result determined by the character determination unit 31 based on the second image data is associated with the character pattern by the character data processing unit 32. Specifically, the type match determination unit 37 is based on type recognition result data (hereinafter referred to as new type recognition result data) of newly stored character recognition result data 55 (hereinafter referred to as new character recognition result data 56). Thus, all type recognition result data included in a plurality of character recognition result data 55 (hereinafter referred to as character recognition result data group 57) already stored in the storage unit 30 are collated. That is, the new type recognition result data corresponds to the first type recognition result data described above. Then, the type match determination unit 37 determines whether there is character recognition result data 55 including type recognition result data that matches the new type recognition result data from the character recognition result data group 57. I do. That is, each type recognition result data of the character recognition result data group 57 corresponds to the above-described second type recognition result data.

  In the type match determination process, when the type recognition result data match at a predetermined ratio or more, it is determined that they match. For example, in the type match determination process, determination is made based on at least two elements: the position information of each partial area having type attribute data matches, and the character string data associated with each partial area match. . Thereby, it is determined that the character recognition result data of the two character recognition result data match.

  The type match determination unit 37 then matches the character recognition result data 55 determined to match in the type match determination process (hereinafter referred to as matched character recognition result data 58) and the new character recognition result data 56. The data table 59 is output. In this case, when there are a plurality of matched character recognition result data 58, a match data table 59 for associating all the matched character recognition result data 58 with the new character recognition result data 56 is output. .

  The data integration unit 38 is a processing unit realized by the CPU 20, the program, and the like in FIG. The data integration unit 38, when the type match determination unit determines that the first type recognition result and the second type recognition result match at a predetermined ratio or more, the first handwritten character recognition result data and the second handwritten character recognition result. Data is integrated and output as integrated handwritten character recognition result data. The first handwritten character recognition result data is data in which the first handwritten character determined by the character determination unit 31 based on the first image data is associated with the character pattern by the character data processing unit 32. The second handwritten character recognition result data is data in which the character data processing unit 32 associates the second handwritten character determined by the character determining unit 31 with the character pattern based on the second image data. Specifically, the data integration unit 38, based on the matching data table 59 stored in the storage unit 30, the new character recognition result data 56 and the matched character recognition result data 58 determined that the type match determination unit matches. Are extracted from the storage unit 30. Then, the data integration unit 38 integrates and integrates the handwritten character recognition result data included in the extracted new character recognition result data 56 and the handwritten character recognition result data included in the matched character recognition result data 58. Is output as integrated character recognition result data 60. Here, the handwritten character recognition result data included in the new character recognition result data 56 corresponds to the first handwritten character recognition result data described above. The handwritten character recognition result data included in the matched character recognition result data 58 corresponds to the above-described second handwritten character recognition result. The integrated character recognition result data 60 is basically the same as the character recognition result data 55 shown in FIG. 5, but the attribute data of the partial area is different. In the partial region attribute data of the integrated character recognition result data 60, the partial region attribute data is classified according to the handwritten character recognition result data before integration. This will be described, for example, when the first character recognition result data 55 and the second character recognition result data 55 are integrated. The attribute data of the partial area corresponding to the first handwritten character recognition result data belonging to the first character recognition result data 55 is classified as “first handwritten character” and the second handwritten character belonging to the second character recognition result data 55. The attribute data of the partial area corresponding to the recognition result data is classified as “second handwritten character”. Thus, in the integrated character recognition result data 60, the attribute data of the partial region of the handwritten character recognition result data is classified for each original character recognition result data 55.

  The storage unit 30 is the same as that described in the functional configuration relating to image processing, and further includes a match data table 59 output by the type match determination unit 37 and an integrated character recognition output by the data integration unit 38. The result data 60 is stored.

  The display control unit 33 is the same as that described in the functional configuration related to image processing, and includes a character image included in the partial area of the integrated character recognition result data 60, and character string data associated with the character image. Are displayed on the display unit 36 separately.

  The correction unit 34 receives correction of the character string data displayed in the third screen area 42 based on an input signal transmitted by the user operating the input unit 35, and each part of the integrated character recognition result data 60. The character string data associated with the area is changed.

  Next, a processing procedure of integration processing in the present embodiment configured as described above will be described.

  FIG. 10 is a flowchart showing a processing procedure in the integration processing of the character recognition result data 55 in the workflow system according to this embodiment.

  As shown in FIG. 10, the integration process is started when the character data processing unit 32 outputs the new character recognition result data 56 (at the end of image processing).

  When the new character recognition result data 56 is input, the type match determination unit 37 checks whether the character recognition result data group 57 is stored in the storage unit 30 (S40).

  When there is a character recognition result data group 57, the type match determination unit 37 reads the new character recognition result data 56 and the character recognition result data group 57 from the storage unit 30, and performs a type match determination process (S41).

  On the other hand, if the character recognition result data group 57 is not stored in the storage unit 30, the type match determination unit 37 adds information for setting the read new character recognition result data 56 as the character recognition result data group 57. . The character recognition result data 55 to which the information is added is stored in the storage unit 30 as a character recognition result data group 57 (S42). And this integration process is complete | finished.

  When the type match determination unit 37 determines that there is type recognition result data that matches the new type recognition result data in the type match determination processing (S41), the matched character recognition result data 58 and the new character recognition result data 56 (S43). The type match determination unit 37 outputs the result of the association processing as a match data table 59. The match data table 59 output by the type match determination unit 37 is buffered in the storage unit 30. The new character recognition result data 56 subjected to the association process is added with information for making the character recognition result data group 57, and the character recognition result data 55 to which the information is added is the character recognition result data group. 57 is stored in the storage unit 30.

  On the other hand, when it is determined that there is no type recognition result data that matches the new type recognition result data in the type match determination process (S41), the type match determination unit 37 uses the read new character recognition result data 56 as the character recognition result data. Information for group 57 is added. The character recognition result data 55 to which the information is added is stored in the storage unit 30 as a character recognition result data group 57 (S44). And this integration process is complete | finished.

  Then, the data integration unit 38 extracts a plurality of character recognition result data 55 subjected to the association process from the storage unit 30 based on the match data table 59 output by the type match determination unit 37. The data integration unit 38 integrates the handwritten character recognition result data of the plurality of extracted character recognition result data 55 (that is, the new character recognition result data 56 and the matched character recognition result data 58), and integrates the integrated results. It outputs as the character recognition result data 60 (S45). The integrated character recognition result data 60 output to the data integration unit 38 is stored in the storage unit, and the integration process ends.

  The display control unit 33 controls the display unit 36 as follows when displaying the integrated character recognition result data 60 on the display unit 36. In the integrated character recognition result data 60, there are two types of attribute data of the partial area, “handwritten characters” and “printed characters” in the character recognition result data 55, “first handwritten characters” and “second handwritten characters”. Three or more of “character” and “type”. When there are more than three types of attribute data in the partial area, the number varies depending on the number of documents for which the type data is determined to be the same, and the number of the document and the number of handwritten character attributes increase. For example, if the number of documents determined to have the same type data is three, the attribute data of the partial area is “first handwritten character”, “second handwritten character”, “third handwritten character”, “type”. There are four types.

  When there are three types of attribute data of the partial area, the character recognition result data pull-down list 43 is displayed in the integrated character recognition result data 60 as “all character strings”, “handwriting only”, and “handwriting 1 only”. , “Handwriting 2 only” and “Type only”. Here, when “handwriting only” is selected, all handwritten characters are displayed in the second screen region 41 regardless of whether the attribute data of the partial region is the first handwritten character or the second handwritten character. When “only handwriting 1” is selected, character string data in which the partial region attribute data is only the first handwritten character is displayed in the second screen region 41. When “only handwritten 2” is selected, character string data in which the partial region attribute data is only the second handwritten character is displayed in the second screen region 41.

  Further, regarding the color of the character displayed in the first screen area 40, the first handwritten character is displayed in red, the second handwritten character is displayed in blue, and the printed character is displayed in black. The same applies to the color of characters displayed in the second screen area. However, the present invention is not limited to this, and the whole handwritten character may be displayed in red regardless of the first handwritten character and the second handwritten character.

  As described above, according to the image processing apparatus 100 of the present embodiment, the handwritten character recognition result data recognized from the image data 53 obtained by optically reading a document is extracted, and the characters of the handwritten character recognition result data are extracted. The column data is displayed in a second screen area 41 different from the first screen area 40 in which the original image data 53 is displayed. Then, based on a correction instruction from the correction unit 34 user, the character string data associated with the character data processing unit 32 displayed in the third screen area 42 is changed.

  For this reason, the user can compare and correct the handwritten character recognition result data and the character image having the handwritten character attribute data in the original image data 53. In addition, since only the handwritten character recognition result data that is likely to be erroneously recognized is extracted and displayed in the second screen area 41, it is possible to prevent omission of confirmation or omission of the OCR processing result of the handwritten character. Thereby, the user can work efficiently when correcting the OCR processing result.

  Further, according to the image processing apparatus 100 according to the present embodiment, the OCR process is performed also on the character candidate area of the type. For this reason, the display control part 33 can extract a character and can display it in the state which can be selected as a handwritten character. In addition, the correction unit 34 accepts correction for the OCR processing result of the type based on the correction instruction from the user, and changes the character string data. For this reason, the user can correct the OCR processing result for the type characters.

  Also, according to the image processing apparatus 100 according to the present embodiment, when the type recognition result data for a plurality of image data 53 match at a predetermined ratio or more, the plurality of image data 53 is corrected for the same document. It is considered to have been done. Then, a plurality of handwritten character recognition result data related to the plurality of image data 53 are integrated. The integrated integrated handwritten character recognition result is displayed in the second screen area 41.

  As a result, even if there are a plurality of correctors, the user can add the handwritten character recognition result data displayed in the second screen area 41 and the original image data 53 displayed in the first screen area 40. The character image included in the partial area can be compared and corrected. Further, since only handwritten characters that are likely to be erroneously recognized are extracted and displayed in the second screen area 41, it is possible to prevent omission of confirmation or omission of erroneously recognized OCR processing results. Thereby, the user can work efficiently when correcting the OCR processing result. Further, since handwritten writing by a plurality of correctors can be displayed in an integrated manner, the working efficiency for one original can be improved.

  Further, according to the image processing apparatus 100 according to the present embodiment, the field that can correct the character string data of the handwritten character recognition result data displayed in the second screen area 41 (that is, the OCR processing result of the handwritten character) is further separated. Are displayed in the third screen area 42. Thereby, the user can confirm or correct the OCR processing result of the handwritten character while referring to the handwritten character by the original image data 53.

  Further, according to the image processing apparatus 100 according to the present embodiment, in the first screen area 40, the character image corresponding to the first handwritten character recognition result data is displayed in red, and corresponds to the second handwritten character recognition result data. The character image is displayed in blue. The same applies to the color of the character displayed in the second screen area 41.

  For this reason, when a plurality of correctors are divided for each role such as correction of sentence content, correction of typographical error, etc., the user can grasp correction by handwritten characters for each role. Thereby, for example, when there is a restriction such as the time for the user to correct the OCR processing result of the handwritten character is limited, the correction by the handwritten character recognized as the role with high priority is selected and the OCR processing result of the handwritten character Corrections can be made.

  Further, according to the image processing apparatus 100 according to the present embodiment, the character determination unit 31 outputs the display position and display range of each partial area as the character determination result data 54. For this reason, when the user selects the character string data extracted in the second screen area 41, for example, the display of the partial area corresponding to the selected character string data can be changed.

  In addition, according to the image processing apparatus 100 according to the present embodiment, in the first screen area 40, the character image of the partial area having the displayed print attribute data is displayed in black, and the part having the handwritten character attribute data The character image of the area is displayed in red. Therefore, the user can instantly distinguish between a printed character image and a handwritten character image in the image data 53 in the first screen area 40.

  Further, according to the image processing apparatus 100 according to the present embodiment, when the user checks the check box corresponding to the character string data displayed in the second screen area 41, it corresponds to the checked character string data. A strikethrough is displayed on the character image in the partial area.

  For this reason, for example, the user can change the display of the character image in the partial area corresponding to the character string data by checking a check box corresponding to the character string data for which the confirmation or correction of the OCR processing result is completed. Can do. Thereby, the user can instantly identify the character image of the partial area corresponding to the character string data for which the confirmation or correction of the OCR processing result has been completed, thereby preventing an erroneously recognized OCR processing result from being checked or corrected. be able to. Thereby, the user can work efficiently when correcting the OCR processing result.

  Further, according to the image processing apparatus 100 according to the present embodiment, when the user clicks the character string data displayed in the second screen area 41 with a pointing device such as a mouse to highlight it, the highlighted character is displayed. The character image of the partial area corresponding to the column data is displayed in reverse video.

  For this reason, the user performs the above-described selection on the character string data in the second screen area 41, so that the character image of the partial area corresponding to the selected character string data is displayed in the first screen area 40. Can be identified instantly. Thereby, the user can work efficiently when correcting the OCR processing result.

  In addition, the present invention can be realized not only as such an image processing apparatus but also as a method using each operation of a characteristic processing unit included in the image processing apparatus as a step. It can also be realized as an integrated circuit that performs processing of the processing unit. Furthermore, the present invention can also be realized as a program for causing a computer to execute the above steps. Such a program can also be distributed via a recording medium such as a CD-ROM or a transmission medium such as the Internet. In addition, the present invention can be realized as a network complex machine incorporating such an image processing apparatus.

  Although the image processing apparatus according to the present invention has been described based on the embodiment, the present invention is not limited to this embodiment.

  That is, the embodiment disclosed this time should be considered as illustrative in all points and not restrictive. The scope of the present invention is defined by the terms of the claims, rather than the description above, and is intended to include any modifications within the scope and meaning equivalent to the terms of the claims.

  For example, in the above-described embodiment, the color of the partial area corresponding to the handwritten character attribute data displayed in the first screen area is displayed in red, and the part corresponding to the type attribute data displayed in the first screen area The color of the area is displayed in black. Thus, although the display form of the partial area | region corresponding to the attribute data of a handwritten character differs from the partial area | region corresponding to the attribute data of a type character, it is not restricted to this. For example, character candidates such that a ruled line frame surrounding a partial area corresponding to handwritten character attribute data is displayed as a bold solid line, and a ruled line frame surrounding a partial area corresponding to type attribute data is displayed as a dotted line The display form of the frame of the ruled line surrounding the area may be different. Further, for example, the character font is changed so that the characters in the partial area corresponding to the attribute data of the handwritten character are displayed in italics and the characters in the partial area corresponding to the attribute data of the type are displayed as the original image data 53. It may be different.

  In the above embodiment, the image processing apparatus 100 does not include the image acquisition unit 39. However, the present invention is not limited to this, and the image processing apparatus 200 includes the image acquisition unit 39 as shown in FIG. It may be. Since the image processing apparatus 200 further includes the image acquisition unit 39, the above-described image processing can be performed without separately preparing an image acquisition apparatus such as a scanner or a digital camera.

  In the present embodiment, the character determination unit 31 performs the character candidate region extraction process and the hand activity determination process, and the character data processing unit 32 performs the character pattern association process. Absent. In the character candidate region extraction process, the hand activity determination process, and the character pattern association process, these three processes may be performed independently by three different processing units, or one processing unit may collectively perform the processing. Alternatively, two of the three processes may be performed by one processing unit, and the remaining one process may be performed by another one processing unit.

  In the above embodiment, the hardware configuring the terminal devices 3 and 4 functions as the image processing apparatus 100. However, the present invention is not limited to this, and the network multifunction device 1 functions as the image processing apparatus 100. good. In this case, the storage unit 30 is realized by the RAM 12 or the like in FIG. The character determination unit 31, the character data processing unit 32, the display control unit 33, and the correction unit 34 are realized by the CPU 10, the control program 50, and the like in FIG. The input unit 35 is realized by the operation panel 15 in FIG. The display unit 36 is realized by the display 16 in FIG. Alternatively, the network multifunction device 1 may perform processing until the character recognition result data 55 is created, and the terminal devices 3 and 4 may perform subsequent processing by the correction unit 34.

  In the above embodiment, the user selects “only handwriting” from the pull-down list 43 in step S21 of the user's workflow processing. "May be selected, or" only type "may be selected.

  In the above embodiment, the integration process is automatically performed when new character recognition result data 56 is generated in the integration process. However, the present invention is not limited to this, and a plurality of character recognition result data 55 selected by the user is used. Only the integration process may be performed. Further, the data to be integrated is not limited to character recognition result data, and may be image data. In this case, the user selects a plurality of image data and outputs them as a result of image processing (S10 to S20). The character recognition result data to be used is used.

  As a result, the image processing apparatus in this case performs the type match determination process only for the character recognition result data 55 for which the type match determination unit 37 has been selected. For this reason, the image processing apparatus in this case does not have to perform the type match determination process for all the character recognition result data 55 already stored in the storage unit 30, and can reduce the load on the system. it can.

  Moreover, in the said embodiment, although the data produced in each process part are memorize | stored in the memory | storage part 30 (namely, hard disk), it is not restricted to this, For example, what is buffered by the memory | storage part 30 May be stored in the RAM 22.

  In the above-described embodiment, the correction unit 34 selects the attribute data classification from the pull-down list 43 so that the character string data of the partial area that selectively has either the handwritten character or the typed attribute data. Is displayed on the display unit 36. However, the present invention is not limited to this, and the pull-down list 43 may not be provided, and character string data of a partial area having handwritten character attribute data may be automatically displayed. Thus, since the character string data of the partial area having at least the attribute data of the handwritten character is displayed, the OCR processing result of the handwritten character with low recognition accuracy of the OCR processing can be corrected. An effect of preventing omission of confirmation of results or omission of correction can be obtained.

  In the above embodiment, the display control unit 33 extracts the character string data of the partial area and displays it on the second screen area 41. However, what is displayed on the second screen area 41 is the character string data. Not exclusively. For example, the character image of the partial area decomposed | disassembled from the image data 53 may be displayed by the character determination part 31 performing a partial area extraction process. That is, based on the character determination result data 54 processed by the character determination unit 31, the character information (character image) relating to the partial region of the handwritten character may be extracted and displayed in the second screen region 41. Based on the character recognition result data 55 obtained by further processing the character determination result data 54 by the character data processing unit 32 as described above, character information (that is, a character image) relating to the partial region of the handwritten character is extracted in the second screen region 41. The character string data may be extracted and displayed.

  Moreover, in the said embodiment, the correction | amendment part 34 receives the correction of the character string data of a partial area in the 3rd screen area | region 42 based on the correction instruction | indication from a user, and changes the character string data. However, the screen area on which this display is performed is not limited to the third screen area 42. For example, the screen area for displaying the correction status of the character string data from the user may be the first screen area 40 or the second screen area 41.

  INDUSTRIAL APPLICABILITY The present invention can be used as an image processing apparatus that can prevent a misrecognized OCR processing result from being omitted from being confirmed or corrected, or as a network complex machine that incorporates the image processing apparatus.

1, 2 Network MFP 3, 4 Terminal device 5 PSTN
6 LAN
7 Digital camera 10 CPU
11 ROM
12 RAM
13 Modem 14 NCU
15 Operation Panel 16 Display 17 Scanner 18 Printer 19 LAN I / F
20 CPU
21 Hard disk 22 RAM
23 Input device 24 Display 25 LAN I / F
26 USB I / F
DESCRIPTION OF SYMBOLS 30 Memory | storage part 31 Character determination part 32 Character data processing part 33 Display control part 34 Correction part 35 Input part 36 Display part 37 Type match determination part 38 Data integration part 39 Image acquisition part 40 1st screen area 41 2nd screen area 42 2nd 3-screen area 43 pull-down list 44 check box 50 control program 51 image processing program 52 character pattern data 53 image data 54 character determination result data 55 character recognition result data 56 new character recognition result data 57 character recognition result data group 58 matched character recognition Result data 59 Match data table 60 Integrated character recognition result data 100 Image processing device 110 Image acquisition device 200 Image processing device

Claims (13)

An image processing apparatus that processes image data obtained by optically reading a document,
A display unit;
When the character candidate area is recognized from the image data, it is determined whether or not the character included in the character candidate area is a handwritten character, and when the handwritten character is determined, the determination result is used as the handwritten character determination result. A character determination unit to output;
A character data processing unit that associates a character that is determined to be a handwritten character by the character determination unit with a character pattern that is a candidate for character recognition, and outputs the associated result as a handwritten character recognition result;
The image data is displayed on the first screen area of the display unit, and the handwritten character determination result output by the character determination unit is displayed on the second screen area of the display unit different from the first screen area, The display which displays the character pattern linked | related by the said character data process part among the said handwritten character recognition results corresponding to the said handwritten character determination result selected by the user in the 2nd screen area on the 3rd screen area of the said display part A control unit;
Based on the correction instruction from the user, the correction of the character pattern displayed in the third screen area is accepted, and the character pattern associated with the character data processing unit in the handwritten character recognition result is changed. An image processing apparatus comprising: a correction unit that performs the processing. The character determination unit further determines whether or not the character included in the character candidate area is a type, and outputs the result of the determination as a type determination result when determining that the character is a type.
The character data processing unit further associates the type determined by the character determination unit with the character pattern, and outputs the associated result as a type recognition result,
The display control unit further displays the character determination result output by the character determination unit in the second screen area, and the character pattern associated by the character data processing unit among the character recognition results is displayed in the second screen area. The image processing apparatus according to claim 1, wherein the image processing apparatus is displayed in a three-screen area. further,
The first type recognition result determined by the character determining unit based on the first image data and the character pattern are associated with each other by the character data processing unit, and the second image data based on the second image data. Type match for determining whether the second type recognition result determined by the character determination unit matches the second type recognition result associated with the character pattern by the character data processing unit at a predetermined ratio or more. A determination unit;
When the type match determination unit determines that the first type recognition result and the second type recognition result match at the predetermined ratio or more, the character determination unit determines based on the first image data The first handwritten character and the character pattern determined by the character determining unit based on the first handwritten character recognition result in which the first handwritten character and the character pattern are associated by the character data processing unit, and the second image data. And a data integration unit that integrates the second handwritten character recognition result associated by the character data processing unit and outputs the result as an integrated handwritten character recognition result,
The display control unit displays the integrated handwritten character recognition result in the second screen region, and is associated with the character data processing unit among the integrated handwritten character recognition results selected by the user in the second screen region. The displayed character pattern in the third screen area,
The correction unit receives correction of the first character pattern associated by the character data processing unit among the first handwritten character recognition results based on a correction instruction from a user, and The first character pattern associated with the character data processing unit is changed, or the modification of the second character pattern associated with the character data processing unit among the second handwritten character recognition results is accepted. The image processing apparatus according to claim 2, wherein the second character pattern associated with the character data processing unit in the integrated handwritten character recognition result is changed. The image processing apparatus according to any one of claims 1 to 3, wherein the third screen area is a screen area different from the first screen area and the second screen area. The display control unit displays the first handwritten character recognition result on the second screen area in a first display form, and displays the second handwritten character recognition result in a second display form different from the first display form. The image processing apparatus according to claim 4, wherein the image processing apparatus is displayed in the second screen area. The image processing according to any one of claims 1 to 5, wherein the character determination unit further outputs a display position and a display range of a character candidate area determined to include a handwritten character as the handwritten character determination result. apparatus. The character determination unit further determines whether or not the character included in the character candidate area is a type, and when determining that the character is a type, the result of the determination is the display position of the type in the image data. And output as a result of type judgment together with the display range of type,
The display control unit displays, in the first screen area, a character candidate area determined to include a type by the character determination unit based on the handwritten character determination result and the type determination result. The image processing apparatus according to claim 6, wherein the character candidate area determined by the character determination unit to include a handwritten character is displayed on the first screen area by a fourth display form different from the third display form. The display control unit corresponds to a handwritten character determination result that is a target of the first selection when the user makes a first selection with respect to the handwritten character determination result displayed in the second screen area. The image processing apparatus according to claim 6, wherein a character candidate area to be displayed is displayed on the first screen area in a fifth display form. The display control unit further determines the handwritten character that is the target of the second selection when the user makes a second selection with respect to the handwritten character determination result displayed in the second screen area. The image processing apparatus according to claim 8, wherein a character candidate area corresponding to the result is displayed on the first screen area in a sixth display form different from the fifth display form. further,
The image processing apparatus according to claim 1, further comprising an image acquisition unit that optically acquires the image data from a document. further,
The image processing apparatus according to claim 1, further comprising a storage unit that stores the character pattern. An image processing method for processing image data obtained by optically reading a document,
When the character candidate area is recognized from the image data, it is determined whether or not the character included in the character candidate area is a handwritten character, and when the handwritten character is determined, the determination result is used as the handwritten character determination result. A character determination step to output;
A character data processing step of associating the handwritten character determined in the character determining step with a character pattern as a candidate for character recognition, and outputting the associated result as a handwritten character recognition result;
The image data is displayed in a first screen area of a display unit, the handwritten character determination result output in the character determination step is displayed in a second screen area of the display unit different from the first screen area, and the first Display control for displaying the character pattern associated by the character data processing unit in the handwritten character recognition result corresponding to the handwritten character determination result selected by the user in the two-screen region in the third screen region of the display unit Steps,
Based on the correction instruction from the user, the correction of the character pattern displayed in the third screen area is accepted, and the character pattern associated with the character data processing unit in the handwritten character recognition result is changed. An image processing method comprising: a correction step to be performed.   A program for causing a computer to execute the steps included in the image processing method according to claim 12.

Images