JP2010136221A - Image processing system and image processing method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To perform various kinds of slip processing by using a camera function of a general-purpose terminal device such as a cellular phone. <P>SOLUTION: A slip image is photographed by a terminal device, a photographed image is transmitted to a center and in the center, it is investigated whether or not quality of the image is satisfactory. If the quality is not satisfactory, the terminal device is notified about a quality defect of the image and retransmission of the image is instructed. The instructed terminal device photographs the slip image again and retransmits it. If the quality of the transmitted slip image is satisfactory, character recognition processing is performed, and the read slip image is registered so as to be retrieved together with a character recognition result. When processing a slip, the registered slip image is displayed visibly, the corresponding character recognition result is displayed visibly, and the character recognition result is updated as needed and registered again in association with the slip image. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to an image processing system and an image processing method for registering image information having satisfactory quality with a unique identification number in association with a pattern recognition result for the image information.

  With recent developments in electronic technology, devices have become smaller and higher in performance, and mobile phones and personal digital assistants incorporating camera functions in addition to communication functions with other devices have become widespread. Furthermore, these functions are becoming more sophisticated year by year. For this reason, many applications that use the camera function of this terminal device have also appeared.

  For example, in Patent Document 1, a general-purpose mobile phone having a camera function is used for meter reading of a gas meter that measures the amount of gas used, the counter display of the gas meter is photographed by the camera, and the photographed data is managed via a public communication line. It is described that it is visually checked at the management center or directly read as numerical data by character recognition processing software.

  Further, Patent Document 2 describes a system in which a document is photographed using a camera function of a cellular phone, a photographed image is sent to a server, character recognition is performed by the server, and text data is attached to a cellular phone as a mail and sent back. Has been. At that time, the server rotates the received image by a predetermined angle, performs preliminary character recognition processing at each rotation position, and performs normal character recognition processing with the rotation state having the maximum character recognition rate as the appropriate rotation state. I was going.

JP 2007-11665 A JP 2004-272658 A

  However, in Patent Document 1, no check is made on the image data taken by the mobile phone, and if the image data is incomplete, the meter reading cannot be performed, and the meter reading must be performed again. There was a problem.

  Furthermore, Patent Document 2 only describes that a captured image is received as it is, and only correction of tilt is performed on the received image, and it is assumed that the captured image is used as it is. When the image quality was poor, almost no characters could be recognized.

  As described above, this type of conventional apparatus is premised on that there is no problem with the photographed image, and even though it can be used theoretically, there is a problem when handling information actually operated by a general user. There was something.

  The present invention has been made for the purpose of solving the above-described problems, and an image image generated by a terminal device having a camera function such as a mobile phone is suitable for character recognition and the like. This is to ensure the quality of the captured image so as to be of quality.

For example, the following configuration is provided as a means for achieving the object. That is,
Image generating means for generating an image, and pattern recognition of a predetermined area in the image generated by the image generating means, and confirming whether the image information generated by the image generating means is satisfactory quality based on the pattern recognition result The quality confirmation means, the regeneration evaluation means for instructing the image generation means to regenerate the image information when the quality evaluation means is not satisfactory image information, and the quality confirmation means is satisfactory. A registration means for registering an image confirmed with quality with a unique identification number; and a pattern recognition means for performing pattern recognition processing on the image confirmed with the quality confirmation means. The means relates to the registered image information the pattern recognition result for the registered image information recognized by the pattern recognition means. Characterized in that an image processing system for registering wearing.

  For example, the quality confirmation unit performs pattern identification on a pattern of a predetermined area of the image information generated by the image generation unit, and confirms the quality of the image information based on the pattern identification result.

Further, for example, the pattern information recognized by the pattern recognition means is at least character pattern information, and the character pattern information can be converted into character data. Alternatively, the pattern information recognized by the pattern recognizing means is characteristic pattern information at least for each type of photographing object.
Further, for example, the image generation means includes an imaging means provided in a portable terminal device capable of communicating with others via a communication medium, and a transmission means for transmitting a captured image of the imaging means via the communication medium, The regeneration instruction means instructs the portable terminal to re-photograph and re-send the image information when the image information is not satisfactory quality.

  Also, for example, an image processing system including a terminal device including image generation means for generating image information and a center device capable of communicating with the terminal device via a communication medium, the terminal device further including the communication medium A communication unit capable of communicating information with the center device via an image, an image transmission unit that transmits an image generated by the image generation unit to the center device by the communication unit, and the communication unit includes the center device. Display means capable of displaying a received message when receiving an image resending request from said image, and image resending means for sending image information generated after receiving said image resending request as resending image information to said center apparatus, The device is an image from the terminal device sent via the communication medium. Image receiving means for receiving information, quality judging means for judging the quality of image information received by the image receiving means, and the image sent when the quality judged by the quality judging means is not satisfactory Retransmission request means for transmitting an image retransmission request to the terminal device, character recognition means for performing character recognition on the image when the quality determined by the quality determination means is satisfactory, and character recognition by the character recognition means It is characterized by comprising registration means for registering the result and the character-recognized image information in association with each other.

  Further, for example, the image generation means of the terminal device includes a camera function capable of photographing image information, and the display means displays image information photographed by the camera function and can visually confirm a photographing result. And

  Further, for example, the determination unit of the center device includes a determination method of performing pattern recognition processing on a partial area of the image information and determining the quality of the image information according to the recognition result.

  In addition, for example, the center device further includes a search unit that searches for image information and character recognition results registered in the registration unit, and a registration information display unit that displays image information and character recognition results searched based on the search results. And a correction unit that receives a correction instruction for the character recognition result, corrects the character recognition result in response to the received correction instruction, and re-registers the character recognition result.

  In addition, for example, the image information generated by the terminal device is slip information, and the character recognition unit performs character recognition on the received slip image and enables slip processing based on a character recognition result.

  Further, for example, the quality determination unit performs the character recognition after executing at least one of an image cutout process, an inclination correction process, and a trapezoid correction process on the received image information.

  Alternatively, an image processing method in an image processing system including a terminal device including image generation means for generating image information and a center device capable of communicating with the terminal device via a communication medium, wherein the terminal device An image generated by the image generation means is transmitted to the center apparatus by a communication means capable of information communication with the center apparatus via a communication medium, and the communication means receives an image retransmission request from the center apparatus. Then, a reception message is displayed, and the image information generated after receiving the image retransmission request is transmitted to the center apparatus as retransmission image information. The center apparatus includes storage means and is sent via the communication medium. The image information received from the terminal device is received, and the received image information When the quality is judged and the judged quality is not satisfactory, an image retransmission request is transmitted to the terminal device that sent the image, and when the judged quality is satisfactory, the character for the image Recognition is performed, and the character recognition result and the character-recognized image information are associated and registered in the storage unit.

  And, for example, the image generation means of the terminal device includes a camera function capable of photographing image information, and the terminal device displays image information photographed by the camera function and can visually confirm a photographing result. The image processing method is a feature.

  Further, for example, the center apparatus performs a pattern recognition process on a partial area of the image information and adopts a determination method for determining the quality of the image information according to the recognition result.

  Further, for example, the center device can further search the image information and the character recognition result stored in the storage means, displays the image information and the character recognition result retrieved by the search result, and instructs to correct the character recognition result. The image processing method is characterized in that the character recognition result is corrected and re-registered in the storage means in response to the received correction instruction.

  Further, for example, the image information generated by the terminal device is slip information, and the center device performs character recognition on the received slip image and enables slip processing from the character recognition result. The method.

  Further, for example, the quality determination of the generated image information is performed by performing the character recognition after performing at least one of an image cutout process, an inclination correction process, and a trapezoid correction process on the received image information. The method.

  According to the present invention, satisfactory quality image information can be obtained from the terminal device, and even when processing such as character recognition is performed, image data that can be used for subsequent processing is easily obtained. And various processes can be easily performed.

  Hereinafter, a specific embodiment to which the present invention is applied will be described in detail with reference to the drawings. In the following description, an example of managing and processing various types of processing target image information using a general-purpose terminal that is generally commercially available, such as a mobile phone or a portable game machine, will be described.

[First Embodiment]
FIG. 1 shows an image processing system according to an embodiment of the present invention. FIG. 1 shows an example of processing a slip image.

  In FIG. 1, reference numeral 100 denotes a server device capable of information communication with another information processing device or a terminal device (communication terminal device) such as a mobile phone via a network 300. For the terminal device (communication terminal device), It becomes the center device. The server device 100 is configured to receive generated image information from the communication terminal device, for example, shooting information in the camera mechanism, and can be registered by assigning a unique number, and can be searched for registered image information. Receive instructions and send search results.

  200 is a slip that exchanges various types of information directly with the server apparatus 100 via the network system 300 or via a communication medium, and receives image information to be processed, for example, slip reading information, and performs slip processing. It is a processing system.

  Reference numeral 300 denotes a network such as a public communication network or an Internet network. Reference numerals 410 to 440 denote terminal devices having a camera function capable of generating a processing target image, such as a mobile phone or a portable information processing device. Each of the terminal devices 410, 420, 430, and 440 includes a display unit that can visually check a captured image (for example, a slip captured image to be processed) captured using the camera function.

  500 is a specific certification authority for E-documents that issue electronic signatures and time stamps as required by the Act on the Use of Information and Communication Technology (hereinafter referred to as “E-Document Law”) for document preservation, etc. conducted by private companies, etc. In response to a request from the server apparatus 100, a necessary electronic signature and time stamp are issued.

  Reference numeral 510 denotes a time distribution station that distributes time information used for the time stamp, and it is needless to say that the configuration may be different from that of the specific certificate authority. A time stamp station 550 issues a time stamp based on time information distributed from the time distribution station.

  In the server 100, reference numeral 110 denotes a display unit capable of displaying various processing information and communication information with others, and displays, for example, image information and character recognition results sent from a mobile phone.

  Reference numeral 120 denotes an operation unit capable of inputting various instructions, and 130 denotes a quality evaluation unit that evaluates the quality of received image information, which will be described in detail later, and determines whether the quality is good. A character recognition unit 140 recognizes a pattern of received image information or registered image information, and generates and outputs code information corresponding to the recognized pattern.

  The patterns to be recognized include specific patterns (including specific barcode patterns, specific symbol patterns, etc.), character patterns, and predetermined symbol patterns for each type of shooting target for specifying the type of shooting target. included. The character recognition unit 140 is configured to be able to perform correction processing on received image information, which will be described later.

  A registration control unit 150 registers the image information evaluated by the quality evaluation unit 130 and the character recognition result recognized by the character recognition unit 140 in association with the image information in the database 180 and is registered in the database 180. Search and read out image information from character recognition results.

  In addition, when the image to be registered is an image to be registered as an E document, the registration control unit 150, when registering the image information in the database 180, sends a digital signature and a time from the specific certificate authority 500 of the E document. A request for issuing a stamp is made so that an electronic signature issued at the time of registration of the image information, a necessary electronic signature and a time stamp are added to the image information and registered together.

  Reference numeral 180 denotes a database for registering the image information evaluated by the quality evaluation unit 130 and the character recognition result recognized by the character recognition unit 140 in association with the image information. It is a database which is a storage means configured to be able to re-register recognition results and processing history information.

  An example of registration data in the database 180 is shown in FIG. FIG. 2 is a diagram illustrating a configuration example of a transmission data registration unit from a terminal device in the database according to the present embodiment.

  In FIG. 2, reference numeral 181 denotes a unique number part in which a unique number (registration address) is registered for each image information assigned for each image information, and the image information from the terminal device specified by the unique number part 181 includes Attached information of image information is registered in association.

  Reference numeral 182 denotes a terminal device that has transmitted the information, for example, a transmission terminal information unit in which terminal specification information for specifying the terminal device is registered. Reference numeral 183 denotes a picture taken from a terminal device evaluated as good quality in an image quality evaluation described later. A registered image part based on the image information, 184 a character recognition result part for the registered image information, 185 an E document flag part indicating whether or not the registered image information is registered as an E document, and 186, for example, registered image information Is a processing history section indicating whether or not a slip processing has been performed on the slip data.

  Image information desired to be processed, for example, an image of a slip desired to be processed is captured by a method such as image capturing from the terminal device according to the present embodiment having the above-described configuration, and the image is transferred to the server device 100 via a communication medium. The data processing according to the present embodiment, in which the server apparatus 100 performs processing on the sent slip image, will be described below with reference to FIGS.

  FIG. 3 is a diagram for explaining a control example of the terminal device when mainly performing slip processing in the present embodiment, and FIG. 4 is a diagram of the server device 100 when performing slip processing mainly in the present embodiment. FIG. 5 is a diagram for explaining a control example, and FIG. 5 is a diagram for explaining the details of the image quality evaluation processing of FIG.

  First, with reference to FIG. 3, a case where a slip to be processed is photographed using a terminal device with a camera function, for example, a mobile phone with a camera mechanism, and is set in a state where it can be processed by the slip processing system 200 via the server 100 will be described as an example. To do. However, the present invention is not limited to slip processing, and can be applied to any document or object that can be photographed. For example, it may be an example of photographing and processing a purchased article in catalog sales, or photographing of a purchase desired portion in a newspaper advertisement column, and is not limited by type or the like as long as it can be photographed by a terminal device.

  The following description will be given by taking as an example the case where the terminal device, for example, the holder of the cellular phone 430 in FIG.

  In the terminal device, first, in step S10, it is checked whether or not a key on the keyboard is operated. If a key operation has been performed, the process proceeds to step S13 to check whether or not a cancel function key has been operated. If it is not a cancel instruction, the process advances to step S15 to check whether or not it is a transmission instruction to the server apparatus.

  If it is not a transmission instruction, the process advances to step S17 to check whether it is an instruction to search for an image already sent to the server apparatus 100. If it is not a search instruction, the process advances to step S19 to perform processing according to the input key. And it returns to an initial state.

  On the other hand, if the input is a cancel key, the process proceeds from step S13 to step S20, where the process being executed is stopped and the initial screen display state is set. Therefore, the process being executed halfway can be returned to the initial state.

  Further, if the input is a search instruction, the process proceeds from step S17 to step S24, and an image image registered in the server device can be specified from an already captured image. For example, a unique number assigned to the registered image is searched as a search keyword. Display as. Then, the selection of the search target displayed in step S26 is monitored.

  When the selection of the search target is performed, a search instruction is sent to the server apparatus 100 via the network 300 together with a unique number (unique number) for specifying an image registered in advance for the server apparatus 100.

  As a search keyword in this search process, information on the own terminal is transmitted as the transmission terminal information 182 in FIG. 2, and the own terminal transmission image image registered in the database 180 is sent as primary search information. This may be displayed to narrow down the search, or various search keywords may be sent to the server apparatus 100 to send a search result list to gradually narrow down the search target.

  This search refinement is not limited, and it may be refined using various search engines already provided elsewhere. Alternatively, it is possible to refer to information specifying a character recognition result instead of information specifying a registered image. Or you may make it select from the log | history regarding the information which the own machine transmitted already.

  In step S30, the search result is monitored. If a search result has been sent, the process proceeds to step S32, and a recognition result corresponding to a desired registered image in the database 180, a processing history, and the like are displayed.

  On the other hand, if the key operation is not performed in step S10, the process proceeds from step S10 to step S35 to check whether or not a shooting instruction has been issued.

  When the owner of the mobile phone 430 intends to process a slip, the owner prepares the slip to be processed and places it on a flat object such as a desk or a table. Then, an operation for photographing a slip is performed using the camera mechanism of the mobile phone. That is, a shooting instruction using the camera mechanism is detected. When a photographing instruction is given, the process proceeds from step S35 to step S37, and for example, an image photographed as described above is captured.

  Specifically, when the shooting mode with the camera is selected, the shooting status of the subject is displayed on the display unit. For this reason, while confirming the display state of the display unit, positioning is performed so that the necessary part of the slip is displayed, the photographing of the subject, for example, the slip is instructed, and the captured image is stored in a memory (not shown) in the mobile phone 430. Information is memorized.

  Here, the photographed image information is displayed on the display unit as shown in step S39. For this reason, it is desirable to visually confirm the photographing result displayed on the display unit to visually confirm whether the necessary part of the slip is photographed or whether the photographed image is in focus.

  If you check the display and determine that the subject has not been shot successfully, stop the current process (cancel it), wait for the next subject to be shot, and give an instruction to perform the slip reading process again. It is desirable to perform the above process again. This leads to prevention of image retransmission processing described later.

  If the image on the display unit is confirmed and it is determined that the image can be sent to the server apparatus 100, a transmission instruction is issued, the process proceeds from step S15 to step S22, and the displayed captured image is sent to the server apparatus 100. The transmission format to the server device 100 includes at least “photographed image information” and “terminal specifying information”.

  On the other hand, if it is not a shooting instruction in step S35, the process proceeds to step S40, and it is checked whether or not a re-transmission instruction for a shot image from the server apparatus 100 has been received. If a re-transmission instruction has not been received, the process proceeds to step S44 to check whether or not a transmission image acceptance notification has been received.

  If a transmission image acceptance notification has not been received, the process proceeds to step S48 to perform corresponding processing. For example, when a mail from a third party is received, the mail is received and stored in a memory, and processing such as notifying of the mail reception is performed.

  On the other hand, when the retransmission instruction is received in step S40, the image information sent to the server 100 side is determined to be not satisfactory quality as a result of transmitting the shooting information to the server 100. This indicates that the transmission information is not discarded and the processing of the transmitted image information is not performed.

  For this reason, since it is necessary to re-photograph and re-transmit the captured image transmitted most recently, or the captured image specified by the re-transmission instruction, the process proceeds from step S40 to step S42, and the display unit is instructed to re-transmit. Displays the reception of.

  For example, a captured image instructed to be retransmitted and that the quality of the captured image is poor and cannot be processed by the server device 100 are displayed indicating that the image should be captured again and sent. A mobile phone operator who confirms this display, for example, who wishes to perform slip processing, sets the slip again and performs the photographing operation again, and performs the processing from step S35 to step S39 again. At this time, it is desirable to pay particular attention to shooting, to focus on it, and to ensure that the entire slip can be shot.

  In this way, the operator of the terminal device simply captures and sends the information of the slip desired to be processed by the camera mechanism, and does not apply any other special load. The slip photographed image information can be sent to the server 100 with a simple operation without installing a complicated program.

  The captured image information may be transmitted, for example, by transmitting a captured image to the server 100 using a packet communication function or the like, or using a mail transmission function of a mobile phone, and establishing a communication path by some method. Any method may be adopted as long as it can send the captured image to the server 100.

  Further, if the transmission image acceptance notification is received in step S44, the process proceeds from step S44 to step S46, and the information attached to the acceptance notification is added to the search target, for example, in addition to the search keyword. The acceptance notification includes, for example, unique number information assigned to the transmission image, information indicating that the E document has been created, and the like.

  Processing on the server 100 side will be described with reference to FIGS. When the server device 100 detects access from the terminal device via the network 300, such as receiving packet communication from a terminal device, for example, a mobile phone terminal, or receiving mail, via the network 300, the steps in FIG. The process proceeds to the process related to the terminal device shown in S100.

  First, in step S101, it is checked whether or not what is sent from the terminal device is a search instruction for a registered image registered in the database 180. If it is not a search instruction for registered image information, the process advances to step S102 to check whether an image to be registered has been sent from the terminal device. If no image to be registered has been sent from the terminal device, the process proceeds to step S103, and it is checked whether or not the process is a search process for registered image information in the database 180 from the terminal. If it is not a search instruction from the terminal, the process proceeds to step S104, the corresponding process is executed, and the process returns.

  When an image to be registered is sent from the terminal device, the process proceeds from step S102 to step S105, and the captured image sent in the process shown in step S22 of the terminal device is received.

  In step S200, an image quality evaluation process is performed to evaluate whether the quality of the image received by the quality evaluation unit 130 is satisfactory. Details of the image quality evaluation process will be described later. When the evaluation by the image quality evaluation process is performed, the process proceeds to step S106, and it is checked whether or not the quality of the result image is evaluated as good quality.

  If the quality is not good, the process proceeds to step S107, and a re-transmission instruction of the captured image is transmitted to the terminal device that has transmitted the image via the communication control unit 160 and the network 300. For example, the re-transmission instruction may include sending back a captured image and including a message indicating that re-transmission is performed due to poor transmission image quality in the image.

  Here, as long as the transmitted image can be identified, the captured image may be identified by the image number. And the process between terminal devices is complete | finished. Thereafter, when the imaging information is sent again in response to this re-transmission instruction, the above processing is performed again.

  On the other hand, if the quality of the image is good quality in step S106, the process proceeds from step S106 to step S110, and the electronic signature and time are sent from the specific authentication center 500 that authenticates that the document is an E document in order to make an E document. The stamp is received and added to the registered image to clarify that it has been authenticated as an E document.

  Subsequently, in step S111, a unique number is assigned to the received image, and when the image evaluation process is not performed by the quality evaluation unit 130, the received image image in which the E document flag is set is used when the image correction process is performed. 2 registers an image image in which the E document flag is set in the corrected image image in the database 180 in the format shown in FIG. At the same time, the terminal specifying information sent from the terminal device is registered together. At this time, the character recognition result 184 is not included.

  If the received image information is an image that does not need to be converted into an E document, the process proceeds from step S106 to step S111 without performing the process in step S110. In this case, an image with no E document flag set is registered.

  Thereafter, in step S112, the character recognition unit 140 is operated to perform character recognition processing on the necessary area of the image. The character recognition result is registered in the database 180 in association with the corresponding image image information as shown in step S112. With this processing, the character recognition result 184 of FIG. 2 is stored, and the processing ends.

  The character recognition range is the necessary characters in the frame of the slip when the slip image is recognized, and the object to be recognized is a pre-registered external character pattern or a specific mark in addition to numbers and characters. Can be included.

  This character recognition result is displayed in the form of a normal color character when, for example, the recognition result is confident (if the recognition result is expected to have no error), the recognition result is displayed. If there is no certainty or there is a possibility of misrecognition, the display color of the recognition result is changed, blinking, or underlined to distinguish it from the recognition result display that seems to be normal recognition. It is desirable to make the display with a mark.

  In the processing of the terminal device, if it is a registered image search instruction from the operation unit 120 of the server apparatus 100 or the slip processing system 200 in step S101, the process proceeds to step S150, and the database 180 is registered according to the search instruction. An information search process is executed, and the registered image information searched and the character recognition result are associated with each other and displayed on the display unit 110, for example. For example, they are displayed side by side in parallel, and are displayed in such a manner that the recognized location and the corresponding location of the captured image can be visually confirmed.

  If the search instruction is from the slip processing system 200, the search result is transmitted to the slip processing system 200 and displayed on the display unit (not shown) of the slip processing system 200 in the same manner as in the server apparatus 100. The same processing as the following processing is performed, and the correction result and the registration instruction are transmitted. Since these processes are the same as those of the server apparatus 100, detailed description thereof is omitted.

  This search instruction can use any item of the registration information shown in FIG. 2 as a search keyword. For example, in addition to the case where the unique number is a keyword, the read terminal device information is used as a keyword, or the character recognition result It is also possible to use some of the items as keywords. Note that the search processing such as display of search results according to keywords, addition of new search keywords, display of narrowed search results, and narrowing down again as necessary is repeated as necessary.

  In subsequent step S151, it is checked whether or not there is a correction instruction for the search result. If there is no correction instruction, it is checked in step S154 whether there is a registration instruction (whether there is an instruction to overwrite the recognition result). If there is no registration instruction, the process returns to step S151 to monitor the correction instruction or registration instruction.

  If there is a correction instruction, the process proceeds from step S151 to step S152, and the recognition result is corrected in accordance with the correction instruction. When the correction is performed, the recognition result for the retrieved search result is corrected, and the process proceeds to step S154. In this process, for example, the operator compares the captured image with the recognition result, confirms the character recognition result, and corrects the recognition result for necessary portions.

  In this embodiment, when imaged image information is received from a terminal device (such as a mobile terminal or a mobile phone), the image recognition information is registered in association with the character recognition result instead of completing the character recognition process continuously. Then, by making the search possible thereafter, the character recognition result can be confirmed by batch processing.

  For this reason, for example, registration results can be sent to other countries where low wages can be processed with low labor costs, and this confirmation process can be performed. It is possible to check the contents of slips.

  As a result, the quality of the image taken with the camera function of the terminal device can be dealt with even if it is not very high quality, and the quality of the image taken using the camera mechanism of a widely used mobile phone camera. Even if it exists, it becomes possible to process reliably and without mistake. Therefore, it is possible to use a photographed image taken by a general person who has not undergone special training, and the range of use can be greatly expanded.

  In this way, the recognition result is confirmed, and if the registration content is to be corrected, the re-registration of the correction result is instructed. Then, the process proceeds from step S154 to step S155, and the corrected recognition result is registered as a new character recognition result in the database 180.

  When re-registration of the recognition result is performed, the process proceeds to step S157, the process according to the recognition result is executed, and the process returns. Here, for example, it is conceivable that information re-registered in the slip processing system 200 is notified that preparation for the slip processing is ready.

  In this case, it is confirmed that the recognition result has been checked at least once by adding a flag to the processing history 186 indicating that the slip processing is not performed but the recognition result is re-registered. Recognize. For this reason, the slip processing system 200 can handle the actual slip processing after confirming that the character recognition result is re-registered, thereby improving the accuracy of the slip processing.

  In the present embodiment, in the character recognition processing in the character recognition unit 140, recognition processing is performed until no errors are found for all characters, and the result is not fixed but is read by, for example, a camera mechanism of a mobile phone. Based on image information as the basis of processing, the recognition results for image information are managed in association with each other so that they can be searched and read out, and both can be visually checked and the recognition results can be corrected as necessary. Even if it exists, it can be maintained in a state where it can be corrected at any time, and even when a slip process described later is performed, the progress of the process can be confirmed at any time, and a final processing error can be prevented.

  Details of the image quality evaluation process in step S200 will be described below with reference to the flowchart of FIG.

  In the image quality evaluation process, a cut-out process for cutting out a necessary image portion from the image image information received from the terminal device in step S201 is executed. This is because, for example, in the case of a voucher image, necessary information can be obtained if there is only a range surrounded by a predetermined ruled line according to the voucher format, so that the burden of processing an extra portion is reduced. .

  In step S202, image tilt correction (skew correction) is performed. For example, some vertical ruled lines or horizontal ruled lines are extracted and corrected so that the inclination of the upper and lower horizontal ruled lines approaches 180 degrees, or the inclination of the vertical ruled lines at the left and right end parts approaches 90 degrees. to correct.

  In step S203, keystone correction is performed. This is to make the size of the photographed image uniform when a portion having a short focal length is large and a portion having a long focal length is photographed small at the time of photographing.

The captured image is corrected by these processes, and then a character recognition process is performed only in a predetermined area in step S204. As the character recognition area, for example, character recognition is performed for each of the four corners of the photographed image. Then, the process proceeds to step S205.
Character recognition is performed for the recognition of the “logo” displayed in a specific place indicating the slip type, or the recognition processing for the mark with a predetermined shape, the recognition processing for the identification mark, and the detection of the ruled line near the edge. And the threshold level is set in accordance with the required level of the required image.

  The character recognition area is not limited to the above example, and some characters may be recognized in the central portion, in the vicinity of the upper and lower ends, or in the vicinity of the left and right ends. It may be determined whether the important part can be recognized in association with the slip format.

  In the present embodiment, the processing from step S201 to step S203 is performed as correction processing of the captured image. However, the present invention is not limited to the above processing, and only arbitrary processing may be performed. Alternatively, other processing may be added.

  As the additional processing, arbitrary processing such as noise removal processing, blurring processing, sharpness processing, contour enhancement processing, photo correction processing, and background removal processing may be appropriately selected and executed.

  In step S205, the character recognition result is determined, and if the probability of correct answer as a result of character recognition is equal to or higher than a predetermined threshold (or if it is uniquely determined, a specific logo or mark can be recognized), the character can be recognized. Judge that the image quality is good.

  If it is determined that character recognition is possible, the process proceeds to step S206. The photographed image is evaluated as a good quality image, and the photographed image subjected to the correction process is returned as a received image.

  On the other hand, if character recognition is not possible at all in step S205, or if the number of characters that can be recognized is small, the process proceeds to step S207, where it is evaluated as a low-quality image, the correction process is canceled, and the process returns. To do.

  As described above, according to the present embodiment, it is possible to obtain a necessary captured image by using a general commercial terminal device such as a mobile phone as it is, and it is possible to construct a very versatile processing system. Moreover, since necessary information can be taken in from the terminal device, a dedicated store and a dedicated device are not required for processing, and anyone can perform various processes such as desired slip processing from anywhere.

  In the above description, the shooting information is transmitted to the server device 100 after shooting by the terminal device, but the server device 100 is accessed from the terminal device side in advance with the slip prepared, and the slip processing is performed. It may be performed after making a request and receiving guidance information and a necessary control procedure for performing slip processing from the server apparatus 100.

  In this case, guidance for slip processing is displayed from the display unit of the terminal device, the slip is photographed according to the guidance, the photographing result is confirmed, and then the photographed image is transmitted, and a retransmission instruction or photographing information reception is performed. It can be controlled to confirm and end the reading process.

  And you may control so that the condition of a slip process may be inquired after that. As a result, the processing slip type and the like can be specified in advance, and the processing reliability can be improved.

  In the above description, an example in which the quality of photographing information is evaluated based on a character recognition result of a predetermined area has been described. However, quality evaluation may be performed based on whether the slip format can be correctly recognized. In this case, it may be determined whether or not the outermost ruled line portion of the necessary slip has been photographed and recognized, and may be comprehensively determined in combination with the character recognition result indicating whether or not this portion has been recognized.

  Alternatively, when the slip format is known, the thickness of the ruled line compared to the size of the entire image is determined, and if the ruled line thickness is within a certain range, the image is focused and a predetermined resolution can be secured. It may be determined that

  In the present embodiment, since the subsequent processing is possible without completing all the processes such as the character recognition process at the stage of shooting the processing target, the shooting quality of the processing target being simply shot is somewhat Since it is sufficient if the quality is secured, the necessary processing can be performed even by using a terminal device operated by an ordinary consumer, for example, a mobile phone.

[Second Embodiment]
In the above description, the terminal device that can be connected to the network 200 only shoots the processing target using the camera function and transmits it to the server device 100, and only re-shoots the captured image. It was.

  However, the present invention is not limited to the above example, and when the terminal device has a high function, not only photographing on the terminal device side but also correction of the photographed image or character recognition is performed. You may make it have the judgment function of whether it is an image.

  The second embodiment according to the present invention has a function of determining whether or not the image is of good quality, such as correction or character recognition on the captured image as well as shooting on the terminal device side. Is described below.

  In the second embodiment, even if a program for performing image processing and character recognition processing is stored in advance on the terminal device side, or before the image processing is performed, for example, the server device 100 or other You may make a request to download the program to the center unit, download the program sent in response to the request, and perform the necessary processing. It may be downloaded and executed little by little each time.

  The following description will be made with respect to an example in which programs are downloaded and executed little by little each time a program in the minimum necessary range is required for processing. FIG. 6 is a flowchart for explaining the processing on the terminal device side according to the second embodiment of the present invention.

  Also in the second embodiment, the control except for the image processing of the terminal device is the same as that of the terminal device in the first embodiment described above. The difference from the processing shown in FIG. Instead of returning to step S10 after taking an image, the only difference is that the processing in step S50 and subsequent steps is performed. For this reason, the following description is performed about the process after step S50, and description about another process is abbreviate | omitted.

  In the second embodiment, a “correction button” or “correction button” and “no correction button” for instructing whether or not the image needs to be corrected are displayed on a part of the display screen of the captured image in step S39. Is displayed.

  For example, when a slip image or the like is photographed, the process proceeds to the next process by instructing either the “correction button” or the “no correction button” displayed on the captured image display screen in step S39. Therefore, in step S50, the photographer confirms the captured image and determines whether or not to perform correction processing on the image. As a result of the determination, if it is determined that no correction is necessary, no correction is instructed. Then, the process returns from step S50 to step S10.

  On the other hand, if it is determined that correction is necessary as a result of the determination, correction is instructed. Then, the process proceeds from step S50 to step S52, the correction to be performed is selected, and a required program download request is made to the server apparatus 100, for example.

  FIG. 7 shows an example of a display screen of the terminal device when a request for downloading the necessary program is made. In FIG. 7, a selection display guide indicating that a correction process to be executed by 610 has been selected, correction process type display guides 611 to 617 to be selected by 630 are correction process selection display frames, and a correction process in which 631 to 637 can be selected. A display unit, 650 is a correction execution button, and 660 is a cancel button.

  Here, the correction processing that can be selected by the terminal device depends on the server device side that supplies the program, and is not limited to the example of FIG. 6, but can be arbitrarily selected and executed. .

  In the example of FIG. 7, the cut-out process and the inclination process are already checked (611, 612), and the inclination correction is selected, and the selection frame is checked by pressing the “decision button” (not shown). Is entered, and when a correction process that has been checked and already selected is selected again, the selection is canceled.

  When the correction process to be performed is selected and the correction execution button 650 is selected, the process proceeds from step S52 of FIG. 6 to step S54, the selected correction program is downloaded, and the correction process is executed in the subsequent step S56. Is done. In the case where the correction process is ended in step S58 and the correction end is instructed, the process returns to step S10. On the other hand, when the next correction process is performed, the process returns to step S50.

  When a large number of selected correction processes are performed, a correction program is downloaded one by one, necessary correction is performed, and then the next correction program is downloaded to perform the next correction. . As a result, the load on the terminal device can be minimized.

  In the example shown in FIG. 7, a plurality of correction processes can be selected at a time. However, only one of these correction processes is controlled to be selectable. You may control to perform. The control in this case is the same as that shown in FIG.

  As described above, according to the second embodiment, since necessary correction processing can be executed on the terminal device side, image information with good quality can be sent and sent to the server device. The quality of the image is improved, and the number of times that image information is regenerated and retransmitted can be reduced.

  This increases the opportunity for the photographer to pay attention to the quality of the captured image, and can contribute to the improvement of the captured image quality.

[Third Embodiment]
In the above description, the example in which the information sent from the terminal device to the server device is specified mainly based on the terminal device information has been described. However, rather than only the terminal information and the time information when registering the image being the basis information for specifying the image information, it is possible to specify the image information based on more diverse information, for example, It is convenient to consider the case.

  In the third embodiment, paying attention to this point, a global positioning system (GPS) function (hereinafter referred to as “GPS function”), which is a terminal device, such as a mobile phone, has become standard equipment in recent years. The position information of the terminal device that generated the image information is transferred to the server device together with the image information, for example, and the server device transmits the terminal position information together with the terminal specifying information sent together with the image information. Register it in the database in association with the image information.

  In the third embodiment, in the captured image transmission processing in step S22 of FIG. 3 or FIG. 6 in the above-described embodiment, in addition to “image information” and “terminal specifying information”, “location information of terminal device” "Is also sent.

  Upon receiving this information, for example, the server device adds the terminal position information to the registration information in the database, and enables the search using the position information. In this way, it becomes possible to measure the location information of the terminal device on the earth and register it together with the image information, so that more detailed information can be obtained, not only slip processing, but also application for regional campaigns, etc. The scope of application can be further expanded, such as application to an application that can effectively utilize the location information of the terminal device.

  The position information in the third embodiment places a special burden on the user of the mobile phone by using the position information notification system that is generally adopted in the GPS function of the mobile phone that is commercially available. In addition, position information can be automatically collected, and it can be used in a wider range only by adding a simple function.

[Other Embodiments]
In the embodiment described above, an example is described in which whether or not image information captured by a terminal device, such as a mobile phone, has appropriate quality, is all performed on the center device side. However, the present invention is not limited to the above example, and an example in which a part of quality certification processing is performed on the mobile terminal side on the mobile terminal side is also included in another embodiment of the present invention.

  For example, the terminal device having the photographic image correction processing function of the second embodiment described above is provided with a pattern recognition function for quality discrimination, and the type of the photographic subject is placed in a predetermined region portion such as a photographic subject document. Predetermined image information for specifying the predetermined image information, for example, a specific barcode pattern, a specific symbol pattern, a specific character string, etc. Whether or not the information is recognized is automatically recognized, and pattern information in which the predetermined image information can be recognized in the predetermined area is transmitted to the center device.

At this time, since the type of image information captured can be determined to some extent from the recognized predetermined image information, the captured image type information may be added and transmitted simultaneously with transmission of the captured image information to a network or the like. As a result, it is possible to prevent a risk that image information of poor quality is transmitted and retransmitted.
Although the above terminal device has been described with respect to the example in which the terminal device described in the second embodiment is provided with a quality discrimination function, the image correction function as described in the first embodiment is provided. Whether the terminal device that does not have the pattern recognition function for quality discrimination, performs pattern recognition for quality discrimination directly on the photographed image, and gives an instruction to re-shoot depending on whether or not the pattern recognition is correctly performed You may control.

  Furthermore, in the above description, the example which applied this invention to the slip process was demonstrated. For this reason, pattern recognition is performed by recognizing the character pattern and number pattern entered in the entry field of the slip image information in which necessary information is entered in the slip pattern and slip entry section, and confirming the consistency of the description contents. It has been explained that necessary slip processing, such as confirmation of accuracy and confirmation of the contents of the description, is performed.

  However, the present invention is not limited to the above example. For example, in image information processing, image information representing contract contents of a contract may be generated, photographed, and registered, and applied to various contract application work. In this case, it is not necessary to shoot all the contents of the contract, and the image information of this part may be registered by shooting only the part actually written by the contractor.

  Further, this system may be used for purchase / sales operations. When applied to a purchase operation (ordering operation), for example, a pattern that can identify a slip type displayed in a predetermined area of a purchase slip (ordering slip) is recognized, and a pattern to be recognized according to the slip type Character recognition processing or the like is performed on the character / symbol pattern, and the recognition result is registered in association with the slip image information registered with a unique identification number. As a result, even if doubt arises in the recognition result, the captured image information that is the original data can be confirmed at any time, and both can be displayed in parallel and visually confirmed. Can be done.

  Furthermore, also in this case, it is possible to visually check whether the recognition result is correct in the batch processing, and it is possible to easily keep the processing cost low by performing this batch processing in a land with low labor costs.

  Furthermore, as described above, image information generated by shooting using the shooting function is registered with a unique identification number, and the recognition result is registered in association with the registered image information. Thus, the original image information can be confirmed at any time using the recognition result.

  For this reason, each business process demonstrated above can be performed easily and correctly. Furthermore, even when a change / addition task associated with the task process is performed, the original image information can be confirmed at any time, and various processes can be performed at any time using the original image information as original data. Furthermore, by adding certification / authentication information as necessary to the original image information, for example, by creating an E document, the reliability of the original data serving as a reference can be improved, and it can be used in a wide range of fields. It becomes possible.

  In addition, image information that has been confirmed to have an appropriate quality is sent to the center device, and then image information recognition on the form, various checks / edits, and necessary data addition are performed. Since it is stored after being registered, the reliability of the registered image information can be ensured and the use can be confirmed at any time.

Since the registered image information is given a unique identification number and can be searched from an external device or an internal device, it can be transferred to an external system as necessary. In the case of documented information, the external system is maintained in a state in which reliability is ensured, and can be easily used for various applications as highly reliable information.
In addition, the claims can be variously modified without departing from the scope of the invention described in detail, and the scope of the present invention is not limited to the technology described above, but covers the functions and technical ideas expressed in the claims. Of course, it can be applied as it is within the range.

It is a block diagram for demonstrating the image processing system of one embodiment which concerns on this invention It is a figure which shows the structural example of the transmission data registration part from the terminal device in the database of this Embodiment. It is a figure for demonstrating the example of control of the terminal device in the case of mainly performing a slip process in this embodiment. It is a figure for demonstrating the example of a control of the server apparatus mainly in this Embodiment. It is a figure for demonstrating the detail of the image quality evaluation process of FIG. It is a figure for demonstrating the example of a control of the terminal device mainly of the 2nd example of an embodiment concerning the present invention. It is a figure which shows the example of a picked-up image correction instruction | indication display screen of the terminal device in a 2nd embodiment.

Explanation of symbols

DESCRIPTION OF SYMBOLS 100 Server apparatus 110 Display part 120 Operation part 130 Quality evaluation part 140 Character recognition part 150 Registration control part 180 Database 181 Unique number (registration address) part 182 Transmission terminal information part 183 Registration image part 184 Character recognition result part 185 Processing history part 200 Slip processing system 300 Network network 410 to 440 Terminal device provided with camera function 500 E Document identification authentication center 510 Time distribution station 550 Time stamp station

Claims (20)

Image generation means for generating an image;
Recognizing a pattern in a predetermined area in the image generated by the image generating means, and confirming whether or not the image information generated by the image generating means is satisfactory quality based on a pattern recognition result;
Regeneration instruction means for instructing the image generation means to regenerate image information when the quality evaluation means is not satisfactory quality image information;
A registration means for registering the image confirmed by the quality confirmation means with a satisfactory identification number with a unique identification number;
Pattern recognition means for performing pattern recognition processing on the image confirmed by the quality confirmation means and satisfactory quality,
The image processing system according to claim 1, wherein the registration unit registers the registered image information in association with a pattern recognition result for the registered image information recognized by the pattern recognition unit. 2. The image according to claim 1, wherein the quality confirmation unit performs pattern identification on a pattern of a predetermined area of the image information generated by the image generation unit, and confirms the quality of the image information based on a pattern identification result. Processing system. 3. The image processing system according to claim 1, wherein the pattern information recognized by the pattern recognition means is at least character pattern information, and the character pattern information can be converted into character data. 3. The image processing system according to claim 1, wherein the pattern information recognized by the pattern recognition means is pattern information specific to at least each type of photographing object. The image generation means includes an imaging means provided in a portable terminal device capable of communicating with others via a communication medium, and a transmission means for transmitting a captured image of the imaging means via the communication medium,
5. The image according to claim 1, wherein the regeneration instructing unit instructs the portable terminal to re-photograph and retransmit the image information when the image information is not satisfactory quality. Processing system. An image processing system including a terminal device including image generation means for generating image information, and a center device capable of communicating with the terminal device via a communication medium,
The terminal device
Communication means capable of information communication with the center device via the communication medium;
Image transmitting means for transmitting the image generated by the image generating means by the communication means to the center device;
Informing means for informing the reception of the image retransmission request from the center device via the communication means,
The center device receives image information from the terminal device sent via the communication medium;
Quality judging means for judging the quality of image information received by the image receiving means;
A registration means for assigning and registering a unique number to the image when the quality judged by the quality judgment means is a satisfactory quality;
Character recognition means for performing character recognition processing on the registered image information,
The image processing system according to claim 1, wherein the registration unit registers the character recognition result obtained by the character recognition unit in association with the image information recognized by the character. The terminal device includes display means for displaying image information and the like, and the image generation means includes a camera function capable of photographing image information,
The image processing system according to claim 6, wherein the display unit displays image information captured by the camera function and allows visual confirmation of a captured result. The quality determination means of the center device performs pattern recognition processing on at least a partial area of the image information whose quality is determined, and determines the quality of the image information according to the recognition result. The image processing system described in 1. The center device further includes a retransmission request unit that transmits an image retransmission request to a terminal device that has transmitted the image when the quality determined by the quality determination unit is not satisfactory.
9. The image processing system according to claim 6, wherein the image information sent previously is not registered. The center device further includes search means for searching image information and character recognition results registered in the registration means,
Registration information display means for displaying the searched image information and character recognition results;
Correction accepting means for accepting a correction instruction for the character recognition result,
10. The image processing system according to claim 6, wherein the registration unit corrects the character recognition result in association with the received correction instruction, and re-registers the character recognition result in association with image information. . The image information generated by the terminal device is slip information, the registration unit registers the received slip image, the character recognition unit performs character recognition on the slip image, and in addition to the slip processing based on the slip image, The image information processing system according to claim 6, wherein the character recognition result is usable. 12. The character recognition unit according to claim 6, wherein the quality determination unit performs the character recognition after executing at least one of an image cutout process, an inclination correction process, and a trapezoid correction process on the received image information. The image information processing system according to any one of the above. The image generation means of the terminal device transmits image information by the image transmission means after executing at least one of image cutout processing, tilt correction processing, and trapezoid correction processing on the generated image information. The image information processing system according to claim 6. An image processing method in an image processing system including a terminal device including image generation means for generating image information and a center device capable of communicating with the terminal device via a communication medium,
The terminal device transmits an image generated by the image generation unit to the center device by a communication unit capable of information communication with the center device via the communication medium,
When the communication means receives an image resend request from the center device, it displays a received message,
The center device includes storage means, receives image information from the terminal device sent via the communication medium,
Determine the quality of the received image information, and if the determined quality is not satisfactory quality, send an image retransmission request to the terminal device that sent the image,
When the judged quality is satisfactory, a unique number is assigned to the image information and stored in the storage means, and character recognition is performed on the registered image information, and the character recognition result is recognized. An image processing method comprising: storing image information in association with image information. The image processing method according to claim 14, wherein the center device employs a determination method of performing pattern recognition processing on at least a partial region of the image information and determining the quality of the image information according to a recognition result. The center device can further search the image information stored in the storage means,
The searched image information and character recognition result are displayed and a correction instruction for the character recognition result is received, and the character recognition result is corrected in association with the received correction instruction and associated with the image information stored in the storage means The image processing method according to claim 14, wherein re-registration is performed. The image information generated by the terminal device is slip information, and the center device registers the received slip image and performs character recognition on the slip image. In addition to slip processing based on the slip image, the character recognition result The image processing method according to claim 14, wherein the image processing method can be used. 15. The quality determination of the generated image information is performed by performing the character recognition after executing at least one of an image cutout process, an inclination correction process, and a trapezoid correction process on the received image information. Item 18. The image processing method according to any one of Items 17. The image generation means of the terminal device includes a camera function capable of photographing image information,
The image processing method according to claim 17, wherein the terminal device can display image information captured by the camera function and visually check a captured result. The image processing according to claim 19, wherein the terminal device transmits at least one of an image cutout process, an inclination correction process, and a trapezoid correction process to captured image information and then transmits the image information to the center apparatus. Method.

Images