JP2019067146A - Information processing device, processing method of the same, and program - Google Patents

Abstract

To provide a mechanism for suitably performing re-imaging after entry error check.SOLUTION: An information processing device is an information processing device including an imaging apparatus and includes mode switching means for switching a first mode for imaging the whole of a business form and a second mode for imaging the part of the business form by using the imaging apparatus, acquisition means for acquiring the whole or part of the imaged business form as electronic data by imaging by the imaging apparatus, error determination means for determining the error of an entry content included in the electronic data acquired by the acquisition means, and control means for performing control for switching to the second mode, when the number pf places determined as the error by the error determination means is a predetermined number or more.SELECTED DRAWING: Figure 10

Description

  The present invention relates to an information processing apparatus for checking entry content, a processing method thereof, and a program.

  Conventionally, there has been a mechanism for checking whether there is a defect in the entry content by using the completed form data and outputting the check result.

  Patent Document 1 describes the technology of a system in which the server checks the entry content of a form transmitted from a user terminal.

Japanese Patent Laid-Open No. 6-28275

  However, in the system configuration of Patent Document 1, when the form sent from the user terminal results in an error due to the entry content, it is necessary to retransmit the form data in which the entry content has been corrected.

  On the other hand, when performing an entry defect check on a captured image obtained by capturing a form using an imaging device, the characters and symbols may be blurred due to unclearness of the captured image, crushing of characters and symbols, blurring, overlapping, paper stains, dust, etc. It can not read correctly, and "reading error" may occur. This "reading error" is difficult to eliminate even if the entire form is re-imaged, and sometimes it is resolved when the error portion is enlarged and re-imaged. However, if there are multiple error locations, if you magnify and re-image, all the error locations can not be included in the image, so the operation does not end with one re-imaging, and the operation to enlarge only the number of error locations and re-image There is a problem that you have to repeat

  Therefore, an object of the present invention is to provide a mechanism for suitably performing re-imaging after a check for incomplete writing.

  An information processing apparatus including an imaging device, wherein the imaging device switches mode switching means for switching between a first mode for imaging the entire form and a second mode for imaging the portion of the form; Acquisition means for acquiring the entire form or part of the imaged form as electronic data by imaging in step (d), error determination means for determining an error in the entry content included in the electronic data acquired by the acquisition means, and the error determination An information processing apparatus comprising: control means for performing control to switch to the second mode when there are a predetermined number or more of places determined as errors by means.

  According to the present invention, it is possible to provide a mechanism for suitably performing re-imaging after a check for incomplete writing.

A figure showing an example of composition of form system 100 of the present invention A diagram showing an example of a hardware configuration of the information processing apparatus 101 Block diagram showing an example of the functional configuration of the information processing apparatus 101 Flowchart showing an example of the entry defect check process in the information processing apparatus 101 A flowchart showing an example of the entire entry defect check process in the information processing apparatus 101 A flowchart showing an example of partial entry defect check processing in the information processing apparatus 101 A diagram showing an example of an entire captured image of a form captured by the information processing apparatus 101 A diagram showing an example of the entire entry defect check result screen (with an entry defect) displayed on the information processing apparatus 101 A diagram showing an example of a whole entry defect check result screen (without entry defect) displayed on the information processing apparatus 101 A diagram showing an example of a partial entry defect check screen displayed on the information processing apparatus 101 A diagram showing an example of a table for managing form forms in the information processing apparatus 101 The figure which shows an example of the table which manages the entry defect check recognition area for every form in the information processing apparatus 101, and the check method. A diagram showing an example of a result list of an entry defect check process in the information processing apparatus 101

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

  As shown in FIG. 1, in the present embodiment, the form system 100 has a configuration in which the information processing apparatus 101 and the server 102 are communicably connected by the network 103.

  The information processing apparatus 101 uses a camera or the like to capture an image of a form (hereinafter referred to as a captured image) and performs an entry defect check. The information processing apparatus 101 manages, for each form form identification information (hereinafter referred to as a form ID), a recognition area of an entry content and an entry type, which are necessary for checking the incompleteness of a captured image.

  In this embodiment, although the entry defect check is performed in the information processing apparatus 101, the present invention is not limited to this. The information processing apparatus 101 transmits a captured image to the server 102, and the entry defect check in the server 102 You may

The information processing apparatus 101 is a portable terminal such as a mobile phone, a smartphone, a tablet terminal, a notebook PC, and a PDA terminal, and an application of a browser or a form system is installed, and wireless communication is performed via the network 103. , And can communicate with the server 102.

  Hereinafter, an example of a hardware configuration applicable to the information processing apparatus 101 illustrated in FIG. 1 will be described with reference to FIG.

  In FIG. 2, reference numeral 201 denotes a CPU, which centrally controls devices and controllers connected to the system bus 204. Further, the ROM 203 or the external memory 212 is required to realize a BIOS (Basic Input / Output System) which is a control program of the CPU 201, an operating system program (hereinafter referred to as OS), and functions executed by each server or each PC. Programs are stored.

  A RAM 202 functions as a main memory, a work area, and the like of the CPU 201. The CPU 201 loads programs necessary for execution of processing from the ROM 203 or the external memory 212 to the RAM 202, and executes the loaded programs to realize various operations.

  An input controller 205 controls an input from the touch sensor 210. A video controller 206 controls display on a display such as the display 211. Although the display 211 is described in FIG. 2, the display may be not only a CRT but also another display such as a liquid crystal display. These are used by the user of the information processing apparatus 101 as needed.

  A memory controller 207 is an external storage device (hard disk (HD)) for storing a boot program, various applications, font data, user files, editing files, various data, etc., a flexible disk (FD), or a PCMCIA card slot. It controls access to an external memory 212 such as a Compact Flash (registered trademark) memory connected via an adapter.

  A communication I / F controller 208 connects and communicates with an external device through the network 103, and executes communication control processing in the network. For example, communication using TCP / IP is possible.

  Note that the CPU 201 enables display on the display 211 by executing, for example, outline font rasterization processing on a display information area in the RAM 202. The CPU 201 can also receive a user's touch operation instruction on the touch sensor 210 on the display 211.

  Various programs and various tables to be described later for realizing the present invention are stored in the external memory 212, loaded into the RAM 202 as necessary, and executed by the CPU 201.

Reference numeral 209 denotes a captured image input controller, which controls an input from the camera 215 via the captured image input processing unit 214. The captured image input from the camera 215 can be recognized and captured.

  FIG. 3 is a block diagram showing an example of a functional configuration of the information processing apparatus 101. As shown in FIG.

  The mode switching unit 301 is a functional unit that switches between a first mode for imaging the entire form and a second mode for imaging a portion of the form using the imaging device.

  The acquisition unit 302 is a functional unit that acquires the whole or part of a captured form as electronic data by imaging with an imaging device.

  The error determination unit 303 is a functional unit that determines an error in the entry content included in the electronic data acquired by the acquisition unit 302.

  The control unit 304 is a functional unit that performs control to switch to the second mode when there are a predetermined number or more of portions determined as errors by the error determination unit 303.

  The error display unit 305 is a functional unit that identifies and displays a portion determined as an error by the error determination unit 303 in the image captured in the second mode.

  When the part determined as an error by the error determination part 303 is not included in the image captured in the second mode, the out-of-image error display part 306 indicates that the part determined as the error exists outside the image. It is a functional unit that identifies and displays.

  The definition information acquisition unit 307 is a functional unit that acquires form form definition information including a recognition area of a form and an entry type.

The entry defect determination unit 308 is a functional unit that makes a determination using the recognition area acquired by the definition information acquisition unit 307 and the entry type.

  FIG. 4 is a flowchart showing an example of the entry defect check process in the information processing apparatus 101.

  FIG. 11 is a diagram illustrating an example of a table for managing form forms in the information processing apparatus 101. The form table 1100 is stored in the external memory 212 of the server 102, and manages “form ID”, “form name”, and “file name of form image” for form forms managed by the server 102.

  The records of the form table 1100 are registered by the system administrator of the form system 100, and the system administrator performs maintenance using a maintenance screen (not shown) and stores them in the server 102, and the information processing apparatus 101 periodically At the time of start of the form application, a record included in the maintained form table 1100 is acquired according to a request from the user of the information processing apparatus 101 or the like, and stored in the external memory 212 of the information processing apparatus 101.

Note that maintenance of the form table 1100 refers to, for example, registration of a new record in the form table 1100 when a form form is newly created, "form name", "file name of form image", etc. by changing form layout or wording. Means that the form image value of the form table 1100 is updated to a new form image file, etc., when it is changed.

  The entry defect check process of FIG. 4 will be described.

  In step S401, the information processing apparatus 101 accepts login to the form system performed by the user of the form system. In the entry defect check process, the form application serving as the user interface may be an application installed in the information processing apparatus 101 or an application such as a Web application displayed on a browser.

  In step S402, the information processing apparatus 101 receives a selection of a mode for imaging a completed form (not shown). The imaging mode in the present embodiment is “whole imaging (still image)” and “partial imaging (moving image)”. That is, step S402 is a step showing an example of processing for switching between a first mode for imaging the entire form and a second mode for imaging a portion of the form using the imaging device. When the selection of the "whole imaging (still image)" mode is received, the process proceeds to step S403, and when the selection of the "partial imaging (moving image)" mode is received, the process proceeds to step S411. Here, the imaging mode in the present embodiment will be described.

  The "whole imaging (still image)" is a mode in which the entire entry form is imaged as a still image. Since a still image is captured, resolution is high, character recognition and check processing can be accurately performed, and many character recognition and check processing can be collectively performed in one imaging.

  The “partial imaging (moving image)” is a mode in which the captured image is subjected to character recognition and check processing in real time while imaging a part of the completed form. In this mode, there is no need to fix the information processing apparatus 101 so as not to shake at the time of imaging as in "whole imaging (still image)" or to adjust the entire form within the frame, and character recognition and check processing Normal / abnormal can be grasped immediately, and character recognition and check can be performed more easily and more efficiently than “whole imaging (still image)”. In particular, when character recognition and check processing of the entry contents of a plurality of items are to be performed continuously in real time, since normal / abnormal can be grasped immediately and continuously, it has merit compared to "whole imaging (still image)" large. In addition, since a part of the form is enlarged for imaging, the disadvantage that the resolution is lower than that of a still image does not pose a major problem. However, in order to capture an image while moving the information processing apparatus 101, character recognition and check processing can not be accurately performed depending on the blurring or moving speed of the image, so it is necessary to move the information processing apparatus 101 at an appropriate enlargement and an appropriate speed. is there.

  In this embodiment, “partial imaging (moving image)” is character recognition and check processing by moving image capturing, but storing the captured moving image is not essential, for example, the camera 215 It is determined that the captured image is not stored in the external memory 212, and it is determined based on the image that the recognition area falls within the guide frame, the recognition of the item name of the recognition item or the identification information, etc. The image may be analyzed in real time to perform character recognition and check processing. Alternatively, the image captured by the camera 215 may be automatically captured according to the trigger, a still image may be analyzed, and character recognition and check processing may be performed.

In the present embodiment, the imaging mode is limited to two of “whole imaging (still image)” and “partial imaging (moving image)”, but is not limited to these, “partial imaging (still image) There may be other modes such as “full imaging → partial imaging (moving image)” and the like.

  In step S403, the information processing apparatus 101 captures the entire filled-in form to be checked for incomplete writing using the camera 215, and stores the captured still image (hereinafter, captured image) (FIG. 7) in the external memory 212. Do. That is, step S403 is a step showing an example of processing for acquiring the whole or part of the imaged form as electronic data by imaging with the imaging device. In this embodiment, although a captured image is obtained using the camera 215 of the information processing apparatus 101, an image captured using a digital camera or the like, an image scanned using a scanner or the like, or the like is processed by the information processing apparatus A method such as loading in 101 may be used.

  In step S404, the information processing apparatus 101 identifies the form ID of the form captured from the captured image. Specifically, all records in the form table 1100 are searched, and the form image and the captured image are compared with the form image extracted from the external memory 212 using the file path stored in the "form image" column. Identify the form ID of the form form to be checked. The form image extracted from the external memory 212 is compared with the captured image, and when the images match at a ratio equal to or more than a predetermined value, the captured image is determined to be a form corresponding to the form image, and the form ID of the captured form Is identified as the form ID corresponding to the matched form image. In the present embodiment, it is assumed that the “shopping credit application form” of form ID = FORM01 is specified from the captured image.

  The two images used for this image comparison may be compared without processing, or in order to increase the accuracy of the image comparison, image comparison is performed after image processing such as extraction of outlines is performed. It is also good.

  In this embodiment, the form form to be subjected to the entry defect check is specified by comparing the captured image with the form image managed by the server 102. However, the present invention is not limited to this method, and the form ID Alternatively, a barcode or a number indicating the form may be described in the form, and the form ID of the form may be specified from the reading at the time of imaging or the analysis of the captured image.

In step S405, the information processing apparatus 101 performs an entire entry defect check process (FIG. 5). That is, step S405 is a step showing an example of the process of determining an error in the entry content included in the acquired electronic data. Step S405 is a step showing an example of processing for determining an error of the entry content using the recognition area and the entry type.

  Here, FIG. 5 will be described.

  FIG. 5 is a flow chart showing an example of the process of checking the entire entry defect in the information processing apparatus 101.

FIG. 12 is a view showing an example of a form definition table for managing an incomplete entry check recognition area for each form and a check method in the information processing apparatus 101. The form definition table 1200 is stored in the external memory 212 of the server 102, and checks “form ID” “group ID identifying a group of plural recognition areas” “identifying a recognition area in the entry defect check” It manages ID's, "form name", "fill-in type for (fill-in defect check)", "start point coordinates (of recognition area) (X, Y)" and "end point coordinates (X, Y)". That is, the form definition table 1200 is an example of a means for managing form form definition information including a recognition area of a form and an entry type.
The records of the form definition table 1200 are registered by the system administrator of the form system 100, and the system administrator performs maintenance using a maintenance screen (not shown) and stores them in the server 102, and the information processing apparatus 101 Periodically, when the form application is activated, a record included in the maintained form definition table 1200 is acquired according to a request from the user of the information processing apparatus 101 or the like, and stored in the external memory 212 of the information processing apparatus 101.
Note that maintenance of the form definition table 1200 refers to, for example, registering a new record in the form definition table 1200 when creating a form form newly, changing the form layout of the form, changing the form requirement, "form name", "entry type" This means that, for example, the value of each record in the form definition table 1200 is updated when the recognition area coordinate or the like is changed.

In this embodiment, the server 102 manages the form table 1100 and the form definition table 1200, acquires the data from the server 102, and manages the form table 1100 and the form definition table 1200 in the information processing apparatus 101. Although the information processing apparatus 101 performs the entry defect check using the table, the present invention is not limited to this method, and the information processing apparatus 101 transmits the captured image to the server 102, and the server 102 manages the form table 1100 and The server 102 may perform an entry defect check using the form definition table 1200, transmit the check result to the information processing apparatus 101, and the information processing apparatus 101 may display the check result.

  The entire entry defect check process of FIG. 5 will be described.

  In step S501, the information processing apparatus 101 searches the form definition table 1200 using the “form ID” identified in step S404 as a search key, and acquires a form definition information list.

  In step S 502, the information processing apparatus 101 substitutes 1 into the variable i and stores the variable i in the RAM 202.

  In step S503, the information processing apparatus 101 repeats steps S504 to S509 for the number of records in the form definition information list acquired in step S501.

  In step S504, the information processing apparatus 101 acquires the start point (X coordinate, Y coordinate) and end point (X coordinate, Y coordinate) of the recognition area and the essential requirement / entry type from the i-th record of the form definition information list. .

  In step S505, the information processing apparatus 101 proceeds to step S506 in the case where the essential requirement = “○ (essential)” acquired in step S504, and proceeds to step S507 in the case where the essential requirement = “NULL (not essential)”. .

  In step S506, the information processing apparatus 101 determines whether a character or a symbol is entered in the recognition area, since the recognition area of the captured image corresponds to the entry required item. If a character or a symbol is entered, it is determined as "OK (OK)", and the process proceeds to step S507. If a character or a symbol is not entered, it is determined as "absent (NG)" and the process proceeds to step S508. In addition, although the analysis technique by OCR etc. is used for this determination process, since it is a known technique, the description is abbreviate | omitted.

  In step S507, the information processing apparatus 101 determines whether the entry content of the recognition area of the captured image matches (= “OK”) or does not match (= “NG”) the entry type acquired in step S504. . In the case of "OK", the process proceeds to step S509, and in the case of "NG", the process proceeds to step S508. That is, step S506 and step S507 are steps which show an example of the process which determines the error of the entry content contained in the acquired electronic data.

  For example, in the case of the entry type = “circled”, it is determined whether the entry content is only a circle (= OK) or not (= NG). If a circle is entered, it is determined as "OK" and the process proceeds to step S509. If a character / symbol other than a circle is entered, it is determined as "NG" and the process proceeds to step S508. In addition, although this determination processing uses an analysis technique by OMR or the like, since it is a known technique, the description thereof will be omitted. In the present embodiment, the circle is taken as an example, but the present invention is not limited to the circle, and determination of marks or other figures / symbols may be made. Further, although the number of circles is not particularly limited in the above, the form definition table 1200 is provided with a column such as “upper limit number of checks” or “number of checks” and the number of circles is “upper limit number of checks”. In the case where the number of circles is other than the "number of checks", it may be determined as "NG".

  In the case of the entry type = “number”, it is determined whether the entry content is only a number (= OK) or not (= NG). Similarly, in the case of the entry type = “katakana”, it is determined whether only katakana (= OK) or not (= NG).

  Further, in the case of the entry type = “date”, it is determined whether the number in the date format is correctly entered (= OK) or not (= NG). Therefore, even when a number in the date format is entered as in "February 31, 2017", such a date does not exist, so it is determined as "NG".

  Similarly, in the case of the entry type = “zip code”, it is determined whether or not the number in the zip code format is entered correctly (= OK) or not (= NG). Therefore, even if an incorrect zip code is entered as "140-9999" in 704 in FIG. 7, since such a zip code does not exist, it is determined as "NG". In this case, the postal code database is stored in the information processing apparatus 101 or the external service or the like is used through the network 103 to determine whether the postal code is normal or abnormal. Further, it may be determined in combination with another entry area (e.g., an address) to determine whether the combination with the zip code is normal or abnormal.

  Moreover, although not mentioned in this example, it is a case of reading a slip like a "health check result slip", and the number of the entry area of entry type = "total cholesterol" is "150 to 219 (mg / dl) If it corresponds to “)”, it is judged as “OK”, and if not, it is judged as “NG”. In that case, by reading the appropriate value range of “total cholesterol” described in the form, storing a database of medical diagnosis appropriate numerical values in the information processing apparatus 101 or using external services through the network 103 , Determine the normality / abnormality of the measured value of health checkup Further, it may be calculated using numerical values of two items such as height and weight, and normality / abnormality may be determined for the calculation result.

  In addition to “circled”, “number”, “katakana”, “date”, “zip code” as the entry type in the form definition table 1200, “kanji”, “phone number”, “city There is a prefecture, etc., but there is no limitation to these, and there may be other types such as “number of digits”, “user-defined” and the like. For example, in the case of "number of digits", "number of digits" column is provided in the form definition table 1200, and if the entered character is the defined number of digits, "OK", if it is other than the defined number of digits " It is judged as "NG". In the case of "user-defined", the "user-defined" column is provided in the form definition table 1200, "OK" if the entered characters are in the format defined by the user, "NG" if it is other than the defined format It is determined that

  Also, if the entered character can not be recognized in the analysis result by the OCR or the like, it may be determined as “NG”.

  In addition, the determination result of whether the recognition area conforms to the entry type is not limited to two of “OK” and “NG”, and values such as “unrecognizable”, “unclear”, “inconsistency”, etc. You may have it.

  In step S508, the information processing apparatus 101 inserts the ith record of the form definition information list into the entry defect check result list 1300 (FIG. 13). The incomplete entry check result list 1300 in FIG. 13 is a list having, as elements, form definition information records in which the result of the incomplete entry check is “NG”.

  As shown in FIG. 13, the entry defect check result list 1300 has an “error content” item, “mandatory”, “circled”, “zip code (= illegal zip code)”, “not recognized”, Record the reasons for the "inconsistency", "inconsistencies", etc. as a result of the incompleteness check = "NG".

  In step S509, the information processing apparatus 101 increments the value of the variable i.

  In step S510, if there is a record of the form definition information list acquired in step S501, steps S504 to S509 are repeated.

This is the end of the description of the entire entry defect check process of FIG. By the entire entry defect check process of FIG. 5, the form definition information record in which the entry defect check result = “NG” is registered in the entry defect check list 1300.

  It returns to the description of the entry defect check process of FIG.

  In step S406, the information processing apparatus 101 determines the number of deficiencies detected in the entry deficiency check in step S405 or step S413. Specifically, it is determined by the number of records existing in the entry defect check result list 1300. If it is determined that there is no defect, the process proceeds to step S407. If it is determined that the number of defects is one or more and less than the predetermined number, the process proceeds to step S408. If it is determined that the number of deficiencies is more than the predetermined number, the process proceeds to step S410. Here, the predetermined number is one or more. That is, step S406 is a step showing an example of processing for controlling switching to the second mode when there are a predetermined number or more of places determined to be in error.

  In step S407, the information processing apparatus 101 displays an entry defect check result screen (without entry defect) 900 (FIG. 9).

  Specifically, in the case of imaging in the "whole imaging (still image)" mode, the captured image 901 captured in step S403 is displayed on the display 211, and a message such as "There was no entry defect." To complete the incompleteness check process of FIG.

  On the other hand, in the case of imaging in the “partial enlargement imaging (still image)” mode, an image such as the captured image 901 is not imaged, so a message such as “description was not complete.” Display (not shown), and the entry defect check process of FIG. 4 ends. In addition, after the imaging in the “whole imaging (still image)” mode, if the correction contents of the defective portion are corrected and imaging in the “partial enlargement imaging (still image)” mode disappears, at least Once the captured image 901 is acquired in the “whole imaging (still image)” mode, the moving image when it is determined that the defect has been eliminated in the “partially enlarged imaging (still image)” mode in the captured image 901 The capture image may be synthesized, and the entry defect check result screen (without entry defect) 900 (FIG. 9) may be displayed.

  In step S408, the information processing apparatus 101 displays an incomplete entry check result screen (with incomplete entry) 800 (FIG. 8).

  Specifically, in the case of imaging in the "whole imaging (still image)" mode, the captured image 801 captured in step S403 is displayed on the display 211, and there is a defect such as "incomplete entry in ○ part". A message is displayed to identify and display an incomplete entry area (802 to 806). Note that the identification display of the incomplete entry region is the start point coordinates (X, Y) and the end point coordinates (X, Y) of the “recognition area” from the records (1301 to 1305) of the entry failure check result list 1300 generated in step S405. ) Is drawn, and a rectangle with the two points as a diagonal is drawn on the captured image and surrounded by a red frame. This makes it possible to distinguishably display the area where the entry was incomplete.

  In addition, the display of the incomplete part 802-805 is not limited to a red frame, and the display by which identification by a user is possible, such as enclosure by a frame other than red, filling, a mark, blinking, enlargement, reverse display etc. If it is Further, the present invention is not limited to the identification method by display, and may be an identification method by other methods such as reading out an incomplete portion by voice.

  On the other hand, in the case of imaging in the "partial enlargement imaging (still image)" mode, since an image such as the captured image 901 is not imaged, it is determined that the message "There is an incomplete entry in ○". The captured image of the moving image at the time of performing is displayed on the display 211 only for the amount of the defect (not shown). As a result, even in the “partial enlargement imaging (still image)” mode, it is possible to distinguishably display the area where the entry was incomplete.

  In addition, on the entry defect check result screen (with an entry defect) 800, a “Recapture whole image (still image) button 804” and a “Remagnify an incomplete portion and recapture (moving image) button 805” are displayed, The content of the entry defect check allows the user to select the reimaging method.

  In step S409, the information processing apparatus 101 receives the selection of the re-imaging mode by the user. If the depression of the “re-image the whole (still image) button 804” is received, the process proceeds to step S403, and the depression of the “re-image (removing image) button 805 which enlarges the insufficient portion is received, the step S411 Go to That is, step S409 is a step showing an example of processing for switching between the first mode for imaging the entire form and the second mode for imaging a portion of the form.

  Specifically, when the user determines that an error has occurred due to an entry in the form, such as an error or omission, the user corrects the entry, and then the user clicks the “Reimage All button (still image) "804" is pressed. In this way, even if there is a plurality of incomplete entries, rechecking can be performed by one reimaging. On the other hand, when the user determines that an error has occurred due to blurring of the captured image, crushing of characters and symbols, blurring, blurring, overlapping, paper stains, dust, etc. In other words, the error is resolved if re-imaging is performed again. The user presses the “magnify the defective portion and re-image (moving image) button 805”. As a result, when reimaging the form, the user can not freely select whether to reimage the entire form as described above by enlarging the error part as described above or reentering the error part. It can be solved.

  In step S409 of this embodiment, the user selects “Re-image the entire image” or “Imagnifies the defective portion and re-image”. However, the present invention is not limited to this, and the entry defect check result If the value of "error content" in list 1300 is "required", "circled", "inconsistency", etc., it is regarded as an error due to the entry content such as an error, and a message is displayed to correct the form and the entire image is recaptured If the value of “error content” is “unclear”, “unrecognizable”, etc., it is regarded as an error due to the captured image, and only the defective portion is enlarged and re-imaged, etc. According to the value, it may be determined whether the information processing apparatus 101 executes “re-image the whole” or “magnify the defective portion and re-image”.

  In step S410, the information processing apparatus 101 displays an incomplete entry check result screen (with incomplete entry) 800 (FIG. 8). The details are the same as step S408, but the “incomplete remark (still image) button 804” is not displayed (or displayed) on the incomplete entry check result screen (incomplete entry) 800 displayed in step S410. But deactivate it so that you can not press it). Thereafter, when the depression of the “remagnify the defective portion and re-imaging (moving image) button 805” is received (or a predetermined time elapses), the process proceeds to step S411. As a result, when the number of defects is determined to be equal to or more than a predetermined number and the form is re-imaged, the error portion is enlarged and re-imaged to continue character recognition and check processing of a plurality of error portions in real time. Since it is possible to continuously and immediately grasp normal / abnormal, character recognition and check can be performed more easily and more efficiently.

  In step S411, the information processing apparatus 101 starts partial imaging of a completed form. That is, steps S411 to S413 are steps showing an example of processing for acquiring the whole or part of the imaged form as electronic data by imaging with the imaging device. The image captured by the camera 215 is displayed on the display 211 in real time (1001 in FIG. 10). A message such as "Please move slowly so that characters can be read." (1002 in Fig. 10) or "moving image capturing" (1003 in Fig. 10) is displayed on the screen, and the camera slowly Be careful to move the

  In step S412, the information processing apparatus 101 identifies the form ID of the form of the captured image. The details are the same as step S404.

In step S413, the information processing apparatus 101 performs a partial entry defect check process (FIG. 6). That is, step S413 is a step which shows an example of the process which determines the error of the entry content contained in the acquired electronic data. Step S413 is a step showing an example of processing for determining an error in the entry content using the recognition area and the entry type. Here, FIG. 6 will be described.

  FIG. 6 is a flowchart showing an example of the partial entry defect check process in the information processing apparatus 101. The flowchart of the whole entry defect check process of FIG. 5 is replaced with a part entry defect check process, and the same process as the step of FIG. By the process of FIG. 6, the incompleteness check of the captured partial form is performed.

  In step S601, the information processing apparatus 101 searches the form definition table 1200 using the “form ID” identified in step S412 as a search key, and acquires the form definition information list.

  In step S602, the information processing apparatus 101 identifies coordinate identification items from the captured image. For example, if the captured image is an image such as 1001 in FIG. 10, black paint such as “(1) application date”, “(2) applicant”, and “(3) work destination” from the captured image Identify the heading portion of as a coordinate identification item. In the present embodiment, although the blackened heading portion is used as the coordinate identification item, the present invention is not limited to this. For example, each item of the form, a barcode or number indicating the form ID of the form, a frame of the form A dot pattern or the like for identifying the end of the form sheet or the coordinates of the form sheet may be used as the coordinate identification item. That is, the information may be any information that can specify which position of the entire form is imaged from the partially captured image.

  In step S603, the information processing apparatus 101 calculates imaging coordinates from the coordinate identification item identified in step S602. For example, in the captured image, the coordinate identification item "(1) application date" has two points of origin coordinates (10, 30) -end coordinates (150, 110) with the upper left corner of the captured image as the origin. "(1) Application date" when not expanding from the form definition record 1201 of the form definition table 1200 when it is located in the rectangle (horizontal: 140 pixels, vertical: 80 pixels) is the origin coordinates (50, 50) -Because it is a rectangle (horizontal: 70 pixels, vertical: 40 pixels) with two points at the end point coordinates (120, 90) as diagonals, it is enlarged by a factor of 2 compared to when capturing a form within the entire document It is understood that imaging is performed. From these, it can be calculated at which position of the entire form (in this case, the start point coordinates (45, 35) −end point coordinates (455, 520)) of the image currently being imaged is imaged.

  In step S604, the information processing apparatus 101 calculates the number of characters in the captured image in order to eliminate the problem that the character recognition and check processing can not catch up with the moving speed of the camera and the character recognition and check processing can not be accurately performed. If it is equal to or more than the predetermined value, the process proceeds to step S605, and if it is less than the predetermined value, the process proceeds to step S606. Here, it is assumed that the calculated number of characters is displayed (1005 in FIG. 10), and the “predetermined value” is set to 200. Note that even if this calculation method is a method of counting the number of characters in a captured image, the form definition information list acquired in step S601 is limited using the captured coordinates calculated in step S603, The number of items may be calculated from the number of items included.

  In step S605, the information processing apparatus 101 automatically zooms the camera 215, reduces the number of characters in the captured image by enlarging the captured image, and enables character recognition and check processing in real time. Here, when auto-zooming, a message indicating that auto-zooming is displayed (1004 in FIG. 10) so that the user is not surprised by the auto-zooming. Further, on the screen (1010 in FIG. 10) enlarged by the automatic zoom, a message indicating that the zoom is finished is displayed (1011 in FIG. 10), and the number of characters in the captured image after zooming is displayed (1012 in FIG. 10). Thus, the problem that the character recognition and check processing can not catch up with the moving speed of the camera and the character recognition and check processing can not be accurately performed can be solved, and the information processing apparatus 101 can be moved at an appropriate enlargement and an appropriate speed. It will be.

In step S606, the information processing apparatus 101 limits the form definition information list acquired in step S601 using the imaging coordinates calculated in step S603. In particular,
The form definition information list acquired in step S601 includes all items of “shopping credit application form”, but by using the imaging coordinates calculated in step S603, limitation is performed in the captured image. Only included items can be listed for check. This makes it possible to speed up character recognition and check processing.

  The subsequent steps S502 (FIG. 6) to S510 (FIG. 6) are the same as the description of FIG.

  However, the recognition area in steps S506 and S507 is determined based on the distance (number of pixels) from the coordinate identification item identified in step S602. For example, in the captured image, the coordinate identification item "(1) application date" has two points of origin coordinates (10, 30) -end coordinates (150, 110) with the upper left corner of the captured image as the origin. If it is located in a rectangular shape (horizontal: 140 pixels, vertical: 80 pixels), it is known from the description of step S603 that the image is doubled and imaged, so the form definition record 1202 of the form definition table 1200 "Date of application" is the start point coordinates (150, 30)-end point coordinates in the captured image based on the coordinates in the captured image of the coordinate identification item "(1) application date" and the coordinates of the form definition record 1202. It can be seen that it exists in the (410, 110) rectangle. As a result, it is possible to perform an incompleteness check using an image captured in an enlarged manner.

  In this embodiment, the recognition area is determined according to the distance from the coordinate identification item included in the enlarged and re-captured captured image, but the present invention is not limited to this. For example, coordinates and distances may be used. Instead, the recognition area of each record item of the form definition table 1200 is determined directly by recognizing the frame of the form included in the captured image, the print character, the dot pattern identifying the coordinates of the form sheet, etc. It may be.

  In step S 607, the information processing apparatus 101 superimposes the captured image so that “recognition NG” is recognized in the recognition area of the item determined as “NG” in step S 506 (FIG. 6) or step S 507 (FIG. 6). And display (1014 in FIG. 10). Further, in the vicinity of “recognition NG”, the detailed explanation of recognition NG is displayed by a balloon or the like (1015 in FIG. 10). That is, step S 607 is a step showing an example of processing for identifying and displaying a portion determined as an error in the image captured in the second mode. As described above, character recognition and check processing can be performed on a captured image in real time, so that it is possible to provide a mechanism for suitably checking for incomplete writing.

  In step S608, the information processing apparatus 101 displays the superimposed image on the captured image so that "recognition OK" can be recognized in or near the recognition area of the item determined as "OK" in step S507 (FIG. 6) (FIG. 10). 1013). As described above, character recognition and check processing can be performed on a captured image in real time, so that it is possible to provide a mechanism for suitably checking for incomplete writing.

  In step S609, when there is a form definition information record in which the result of the entry defect check = “OK” in step S608 exists in the entry defect check result list 1300 (FIG. 13), the information processing apparatus 101 performs the entry defect check Delete the record from the result list 1300. As a result, since the form definition information record in which the result of the entry defect check = “NG” → “OK” is deleted from the entry defect check result list 1300 in real time, the entry defect check result list 1300 again includes the entry defect check Only the records that need to be

  In step S610, the information processing apparatus 101 determines the presence / absence of a record in the entry defect check result list. If there is a record, the process proceeds to step S611. If there is no record, the process proceeds to step S509 (FIG. 6).

  In step S611, the information processing apparatus 101 displays the position indicated by the record remaining in the entry defect check result list by using an arrow. That is, in the process of step S611, when the portion determined as an error is not included in the image captured in the second mode, the portion determined as the error is identified and displayed outside the image and displayed. It is a step showing an example. Specifically, when the record 1305 remains in the entry defect check result list, the coordinates of the record 1305 (form name: year-end income) are compared with the imaging coordinates of the currently displayed captured image calculated in step S603. And a message such as “There is an error in this direction. Please move the camera slowly.” Is displayed along with the downward arrow, since it exists below the captured image (1099 in FIG. 10). . This makes it possible to guide the camera to an item that needs to be checked for incomplete writing, so character recognition and checking can be performed more efficiently. That is, it is possible to solve the problem that the user does not know which part of the form should be imaged when the image is enlarged and imaged.

This is the end of the description of the partial entry defect check process of FIG. By the process of FIG. 6, the incompleteness check of the imaged partial form is performed, and the result of the incompleteness check of the imaged partial form in the incompleteness check result list 1300 is form definition information for which the result = "NG" is not eliminated. Records will be left. It returns to the description of the entry defect check process of FIG.

  In step S414, the information processing apparatus 101 ends partial imaging of the form.

This is the end of the description of the entry defect check process of FIG.

As described above, when there are a plurality of error locations, if the image is enlarged and recaptured, all the error locations can not be included in the image, so one reimaging does not end, and only the number of error locations is magnified and recaptured. It is possible to solve the problem that the operation has to be repeated, and to provide a mechanism for suitably performing re-imaging after the entry defect check.

  As described above, the recording medium recording the program for realizing the functions of the above-described embodiments is supplied to the system or apparatus, and the computer (or CPU or MPU) of the system or apparatus stores the program stored in the recording medium. It goes without saying that the object of the present invention can also be achieved by reading and executing.

  In this case, the program itself read out from the recording medium realizes the novel function of the present invention, and the recording medium recording the program constitutes the present invention.

  As a recording medium for supplying the program, for example, a flexible disk, hard disk, optical disk, magneto-optical disk, CD-ROM, CD-R, DVD-ROM, magnetic tape, non-volatile memory card, ROM, EEPROM, silicon A disk etc. can be used.

  Further, by executing the program read by the computer, not only the functions of the above-described embodiment are realized, but also an operating system (OS) or the like running on the computer is actually executed based on the instructions of the program. It goes without saying that a part or all of the processing is performed and the processing realizes the functions of the above-described embodiments.

  Furthermore, after the program read from the recording medium is written to the memory provided to the function expansion board inserted into the computer or the function expansion unit connected to the computer, the function expansion board is read based on the instruction of the program code. It goes without saying that the case where the CPU or the like provided in the function expansion unit performs part or all of the actual processing and the functions of the above-described embodiment are realized by the processing.

  Further, the present invention may be applied to a system constituted by a plurality of devices or to an apparatus comprising a single device. It goes without saying that the present invention can also be applied to the case where it is achieved by supplying a program to a system or apparatus. In this case, by reading a recording medium storing a program for achieving the present invention into the system or apparatus, the system or apparatus can receive the effects of the present invention.

  The form of the program may be an object code, a program code executed by an interpreter, script data supplied to an operating system (OS), and the like.

100 information processing system 101 information processing apparatus 102 server 103 network

Claims (6)

An information processing apparatus comprising an imaging device, wherein
Mode switching means for switching between a first mode for imaging the entire form using the imaging device, and a second mode for imaging a portion of the form;
An acquisition unit configured to acquire the whole or a part of the captured form as electronic data by imaging with the imaging device;
An error determination unit that determines an error of the entry content included in the electronic data acquired by the acquisition unit;
An information processing apparatus comprising: control means for performing control to switch to the second mode when there are a predetermined number or more of places determined as errors by the error determination means. 2. The information processing apparatus according to claim 1, further comprising error display means for identifying and displaying a part determined as an error by the error determination means in the image captured in the second mode. When the part determined to be an error by the error determination means is not included in the image captured in the second mode, the outside of the image is identified and displayed that a part determined to be the error is present outside the image. The information processing apparatus according to claim 1, further comprising an error display unit. The information processing apparatus further comprises definition information acquisition means for acquiring form form definition information including a recognition area of the form and an entry type.
The information processing apparatus according to any one of claims 1 to 3, wherein the error determination unit determines using the recognition area and the entry type acquired by the definition information acquisition unit. A processing method in an information processing apparatus provided with an imaging device, comprising:
The information processing apparatus
Mode switching means for switching between a first mode for imaging the entire form using the imaging device, and a second mode for imaging a portion of the form;
An acquisition unit configured to acquire the whole or a part of the captured form as electronic data by imaging with the imaging device;
An error determination unit that determines an error of the entry content included in the electronic data acquired by the acquisition unit;
And a control step of performing control to switch to the second mode when there are a predetermined number or more of places determined as errors by the error determination means. A program executable by an information processing apparatus including an imaging apparatus,
The information processing apparatus
Mode switching means for switching between a first mode for imaging the entire form using the imaging device, and a second mode for imaging a portion of the form;
An acquisition unit configured to acquire the whole or a part of the captured form as electronic data by imaging with the imaging device;
An error determination unit that determines an error of the entry content included in the electronic data acquired by the acquisition unit;
A program for functioning as control means for controlling to switch to the second mode when there are a predetermined number or more of places determined as errors by the error determination means.

Images