JP7240164B2 - Image processing device, image management method, program - Google Patents

Description

The present invention relates to an image processing device, an image management method, and a program.

Various application procedures by using image data obtained by photographing various certificates such as driver's licenses with terminal devices such as mobile phones and smartphones, and character information obtained by character recognition of the image data are increasingly being done electronically. For example, there is a mobile terminal device described in Japanese Patent Application Laid-Open No. 2002-200313 as a device that performs character recognition by photographing with a camera of a terminal device.

In the technique disclosed in Patent Document 1, for example, character recognition is performed on a plurality of document images obtained while a document to be read is continuously photographed, and character information such as characters and values contained in the document images is detected. do. Furthermore, in the technique disclosed in Patent Document 1, character information indicating a predetermined item name is detected from among the detected character information, and recorded corresponding to the item name, for example, Character information such as names and addresses that differ for each individual is read and acquired.

Japanese Patent No. 6364051

By the way, in the case of a form larger than a driver's license, it may not be possible to capture the entire form at once with a camera such as a smartphone in order to obtain an image with the resolution required for character recognition. . For such a form, if the technology disclosed in Japanese Patent Laid-Open No. 2002-200010 is used, all the character information recorded in the form can be acquired by dividing the form into a plurality of areas and continuously photographing the form. It becomes possible to However, when frame images are captured by dividing each area in this way, one sheet of form is restored using a plurality of obtained frame images.
However, it is difficult to restore the entire form because it is not known to which position on the form each of the multiple captured frame images corresponds.

SUMMARY OF THE INVENTION The present invention has been made in view of such circumstances, and its object is to detect the position of each of a plurality of photographed frame images on a form and restore the image of the entire form. An object of the present invention is to provide an image processing device, an image management method, and a program that facilitate the processing.

In order to solve the above-described problems, the image processing apparatus of the present invention includes a frame image acquisition unit that acquires frame images obtained by partially continuously photographing one form, and character recognition of character images in the frame images. and an image processing unit for superimposing the frame image on the template image based on the feature information of the template image of the form and the feature information of the frame image from which the character information is obtained by the character recognition. When the frame image is superimposed on the template image, if the character information overlaps with the frame image superimposed one before, the image processing unit recognizes the character information in the overlapping portion. It is characterized by superimposing the frame image on the top .
Further, the image processing apparatus of the present invention comprises: a frame image obtaining unit for obtaining frame images obtained by partially continuously photographing one form; a character recognition unit for performing character recognition of character images in the frame images; an image processing unit that superimposes the frame image on the template image based on the feature information of the template image of the form and the feature information of the frame image from which the character information is obtained by the character recognition; and detecting an area where the character information is not recognized as an unnecessary area when the frame image is superimposed on the template image.

The image management method of the present invention includes a frame image acquisition process in which a frame image acquisition unit acquires frame images obtained by partially continuously imaging one form, and a character recognition unit in which characters of character images in the frame images are acquired. A character recognition process for performing recognition, and an image processing unit converts the frame image into the template image based on the feature information of the template image of the form and the feature information of the frame image from which the character information is obtained by the character recognition. and an image processing step for superimposing, wherein when the frame image is superimposed on the template image, if the character information overlaps with the frame image superimposed one before, the overlapping portion is removed. and superimposing the frame image in which the character information is recognized in the above .
Further, the image management method of the present invention includes a frame image acquisition process in which a frame image acquisition unit acquires frame images obtained by partially continuously capturing one form, and a character recognition unit acquires a character image in the frame image. and an image processing unit converts the frame image into the template based on the characteristic information of the template image of the form and the characteristic information of the frame image from which the character information is obtained by the character recognition. and an image processing step for superimposing the frame image on the image, wherein the image processing step includes detecting an area where the character information is not recognized as an unnecessary area when the frame image is superimposed on the template image. do.

The program of the present invention comprises: frame image obtaining means for obtaining frame images obtained by partially continuously photographing one form; character recognition means for performing character recognition on character images in the frame images; and template of the form. a program for functioning as image processing means for superimposing the frame image on the template image based on the feature information of the image and the feature information of the frame image from which the character information is obtained by the character recognition; In the means, when the frame image is superimposed on the template image, if the character information overlaps with the frame image superimposed one before, the frame image in which the character information is recognized in the overlapped portion is displayed. Including layering on top.
Further, the program of the present invention comprises: frame image obtaining means for obtaining frame images obtained by partially continuously photographing one form; character recognition means for performing character recognition of character images in the frame images; a program for functioning as image processing means for superimposing the frame image on the template image based on the feature information of the template image of and the feature information of the frame image from which the character information is obtained by the character recognition; The image processing means detects an area where the character information is not recognized as an unnecessary area when the frame image is superimposed on the template image.

According to the present invention, there is provided an image processing apparatus, an image management method, and a program that detect which position of a form was photographed by each of a plurality of photographed frame images, and facilitate restoration of the image of the entire form. be able to.

1 is a schematic block diagram showing the configuration of a form data management system according to a first embodiment; FIG. It is a figure which shows an example of the template of the form in the same embodiment. It is a figure which shows an example of the form in the same embodiment. It is an example (part 1) of a frame image captured in the same embodiment. FIG. 10 is an example (part 2) of a frame image captured in the same embodiment; FIG. It is a figure which shows the form definition table in the same embodiment. FIG. 4 is a diagram (part 1) showing a form information table in the same embodiment; It is a figure which shows the superimposition state table in the same embodiment. It is a figure which shows the form information total table in the same embodiment. It is a flowchart which shows the flow of the imaging|photography process of the form by the terminal device in the same embodiment. FIG. 11 is a diagram (part 2) showing a form information table in the same embodiment; 3 is an example (part 3) of a frame image captured in the same embodiment; FIG. 13 is a diagram (part 3) showing a form information table in the same embodiment; 10 is a flow chart showing a processing flow for selecting a form information table to be saved by the terminal device according to the same embodiment. 7 is a flow chart showing the flow of processing for superimposing a frame image on a template image to generate a superimposed image by the terminal device according to the same embodiment. It is a figure which shows an example of the process by the matching process part in the same embodiment. It is a figure which shows the structure of the superimposition image in the same embodiment. It is an example of the frame image image|photographed in the other structural example of the same embodiment. FIG. 11 is a diagram showing a form information table in another configuration example of the same embodiment; FIG. 10 is a diagram showing a state in which two frame images are superimposed in another configuration example of the same embodiment; It is a figure which shows the superimposition state table in the other structural example of the same embodiment. FIG. 11 is a diagram showing a form information table in the second embodiment; FIG. FIG. 10 is a diagram (part 1) showing an unnecessary area of a frame image in the same embodiment; FIG. 10 is a diagram (part 2) showing an unnecessary area of a frame image in the same embodiment; FIG. 4 is a diagram showing the positional relationship of frame images in the same embodiment;

(First embodiment)
A form data management system S including the terminal device 1 according to the first embodiment of the present invention will be described below with reference to the drawings. FIG. 1 is a schematic block diagram showing the configuration of a form data management system S. As shown in FIG. The form data management system S includes a terminal device 1 , a form data management device 3 , and a communication network 4 connecting the terminal device 1 and the form data management device 3 .

The terminal device 1 is, for example, a device such as a smartphone with a built-in camera, a mobile phone, a personal digital assistant, or a tablet PC (Personal Computer). The terminal device 1 executes a predetermined application program (hereinafter referred to as AP) downloaded from the form data management device 3 or another server device through the communication network 4, and functions, for example, as a communication device, or as described below. It also functions as a device for photographing documents.
The terminal device 1 includes an operation unit 10, an imaging unit 11, a display unit 12, a frame image acquisition unit 13, a storage unit 14, a character recognition unit 15, a selection unit 16, a recognition data management unit 17, a communication unit 18, and an image processing unit. 20.

In the first embodiment, the form to be photographed by the terminal device 1 includes, for example, a title (personal medical examination form) as shown in FIG. and ruled lines are printed in advance. When the information of characters and numbers is recorded in the template shown in FIG. 2, for example, a form shown in FIG. 3 is obtained.

The imaging unit 11 is, for example, a CCD (Charge Coupled Device) camera, and when instruction information to start imaging is supplied from the operation unit 10, it continuously performs imaging at a frame rate of 10 to 30 frames per second. , frame image data is obtained for each frame rate cycle. Further, the photographing unit 11 finishes photographing when instruction information for finishing photographing is supplied from the operation unit 10 . For example, FIGS. 4 and 5 are examples of frame images 61 and 62 obtained by continuously photographing the form shown in FIG. 3 from top to bottom.

The display unit 12 is, for example, a touch panel in which an input device such as a touch pad is arranged on the screen, and displays frame images captured by the imaging unit 11 or image data stored in the storage unit 14 on the screen. do.
The communication unit 18 communicates with a device external to the terminal device 1 , such as the form data management device 3 , through the communication network 4 .

The storage unit 14 includes a frame image storage unit 141, an application storage unit 142, a template image storage unit 143, a template characteristic information storage unit 146, a form definition information storage unit 144, a form information storage unit 145, and a superimposed image storage unit 147. .

The frame image storage unit 141 stores frame image data acquired by the frame image acquisition unit 13 from the imaging unit 11 together with frame identification information associated with each frame image. The application storage unit 142 stores a predetermined AP downloaded from the form data management device 3 or another server device through the communication network 4 .

The template image storage unit 143 stores template image data obtained by reading, for example, the template image of the form shown in FIG.

The form definition information storage unit 144 stores, for example, a form definition table 1441 shown in FIG. 6 in advance. The form definition table 1441 has items of "form identification information", "item name", "item number", and "definition information". The item "form identification information" contains information that can uniquely identify the types of form templates, for example, templates with different column formats and item names. is written. Information on the item name included in the form corresponding to the form identification information is written in the item of "item name". An item number assigned in advance to each item name is written in the "item number" item. In the "definition information" item, the item name, information indicating whether the information recorded corresponding to the item name (hereinafter referred to as item record information) is written horizontally or vertically, the item name, and Information indicating the positional relationship of item record information is written. Here, the information indicating the positional relationship between the item name and the item record information is, for example, information in which the item name and the item record information are positioned in one horizontal line or one vertical line as shown in FIG. For example, the item record information is located in any row or column on the top, bottom, left, or right.

The form definition information storage unit 144 stores a form definition table 1441 corresponding to a form type template designated in advance by the user. When photographing a form using another type of template, ie, a template with different form identification information, it is necessary to write the form definition table 1441 corresponding to the other template in the form definition information storage unit 144 .

The form information storage unit 145 stores a form information table having the same data format generated for each frame image. For example, the form information table 1451-1 shown in FIG. 7 is a table generated corresponding to the frame image 61 shown in FIG. In the form information table 1451, the item number in the corresponding frame image and the item record information as the acquired information are written in the respective columns of the items of "frame identification information", "item number", and "acquired information". The item of "acquired information" corresponding to the item number of the item name for which the character recognition unit 15 acquires the item name but has not acquired the item record information corresponding to the item name is blank.

The template feature information storage unit 146 stores feature information including feature points and feature amounts calculated by the image processing unit 20 based on the template image stored in the template image storage unit 143 .

The superimposed image storage unit 147 stores superimposed image data and a superimposed state table 1471 shown in FIG. Here, the superimposed image is, for example, an image in which a frame image that has undergone image conversion by the image processing unit 20 so as to match the size of the template image and the positions of items and ruled lines is superimposed on the template image. . As shown in FIG. 8, the superimposition state table 1471 has items of "frame identification information" and "arrangement state". In the "frame identification information" item, the frame identification information of the frame images superimposed on the superimposed image is written in the order in which they are superimposed. In the example shown in FIG. 8, the frame image with the frame identification information "1" is first superimposed on the template image, and then the frame image with the frame identification information "2" and the frame image with the frame identification information "3" are superimposed. It is shown that.

In the "arrangement state" item, information indicating which frame image is arranged above or below in relation to the frame images before and after when frame images are overlapped with an overlapping area. written. For example, in the example of FIG. 8, the frame image whose frame identification information is "1" is first superimposed on the template image, and there is no preceding frame image. A "-" is written to indicate that it does not exist. In the first embodiment, since the frame image to be overlaid later is superimposed, the frame image with the frame identification information "1" is positioned at the bottom in relation to the frame image with the frame identification information "2" to be superimposed next. Become. Therefore, "bottom" is written in the lower column of the "arrangement state" item of the frame identification information "1", and "upper" is written into the upper column of the "arrangement state" item of the frame identification information "2". is written. Note that FIG. 8 shows an example in which three frame images are superimposed, and a record (row) is generated according to the number of superimposed frame images.

Note that the frame image storage unit 141, the form information storage unit 145, and the superimposed image storage unit 147 of the storage unit 14 are not mixed with other forms when, for example, a plurality of forms having the same template type are photographed. Therefore, it is necessary to initialize before starting to shoot each form. Also, it is desirable that the storage area be released when the AP for photographing the form ends.
Further, the information stored in the template image storage unit 143, the form definition information storage unit 144, and the template feature information storage unit 146 in the storage unit 14 is stored when the AP for taking the form is terminated or the power of the terminal device 1 is turned off. may also be stored in a storage area, for example, preconfigured in a non-volatile storage area (flash memory, etc.). It should be noted that the template image and the form definition table 1441 may be re-registered each time the AP for photographing the form is activated.

The operation unit 10, the frame image acquisition unit 13, the character recognition unit 15, the selection unit 16, the recognition data management unit 17, and the image processing unit 20, which will be described below, are configured such that, for example, an AP for photographing a form is installed inside the terminal device 1. is a functional unit configured by being executed by the CPU of
The operation unit 10 is connected to, for example, a touch pad provided on the screen of the display unit 12, and when the user touches the screen with a finger, the touch pad outputs the touched position on the screen. Take action according to the information. For example, when the document photographing AP is activated, it displays a photographing start button for instructing the start of photographing processing and a photographing end button for instructing the end of photographing processing on the screen of the display unit 12 .

The operation unit 10 initializes the frame image storage unit 141, the form information storage unit 145, and the superimposed image storage unit 147 when the position information output by the touch pad is the position of the shooting start button. After performing the initialization, the operation unit 10 outputs to the image capturing unit 11 instruction information for starting image capturing, which instructs the image capturing unit 11 to start image capturing processing continuously. Further, when the position information output by the touch pad is the position of the shooting end button, the operation unit 10 outputs shooting end instruction information indicating the end of shooting processing to the shooting unit 11, the character recognition unit 15, and the selection unit 16. to output

The frame image acquisition unit 13 acquires a frame image generated and output by the imaging unit 11 . When acquiring a frame image, the frame image acquisition unit 13 generates new frame identification information, associates the acquired frame image with the generated frame identification information, and writes and stores them in the frame image storage unit 141 . For example, the frame identification information is generated by the frame image acquisition unit 13 by incrementing the value from the smallest number, for example, "1", so that the chronological order of the frame images can be known.

The character recognition unit 15 extracts and cuts out images of characters and numbers (character images) from the frame image by OCR (Optical Character Recognition) processing, performs character recognition using a pattern dictionary, a font dictionary, etc., and performs character recognition. Get character information that indicates a number. Here, the information obtained by character recognition is, for example, character information indicating characters and numbers, and may be, for example, text data. Further, the character recognition unit 15 acquires the coordinates of the circumscribing rectangle of each character or number during character recognition.

Further, when outputting the character information and the coordinate information, the character recognition unit 15 associates the frame identification information of the frame image subjected to the character recognition processing with the character information and the coordinate information to be output. When performing character recognition processing, the character recognition unit 15 detects all combinations of character information and coordinate information contained in the frame image at once. Character recognition processing is determined to be unsuccessful, and frame identification information corresponding to the frame image is not acquired. Then, using the frame identification information, the character information, and the coordinate information, the character recognition unit 15 refers to the form definition table 1441 in the form definition information storage unit 144, and recognizes the range of the specific coordinates from the acquired character information. Detects textual information contained.

In the case of horizontal writing, the specific coordinate range is a range specified by the coordinate information of each character included in a character string in which the characters are arranged in a line and the intervals between the characters are constant. The character recognition unit 15 identifies the character or character string of the item name from the character information included in the detected specific coordinate range. The character recognition unit 15 separates the character information included in the specified coordinate range including the character or character string of the specified item name into the item name information and the corresponding item record information and acquires them.

For example, in the case of the form shown in FIG. 3, the character recognition unit 15 acquires the information of the item name of "name" and the item record information of "Taro Tokkyo" for the item name of "name". Note that the character recognition unit 15 does not always acquire both the item name information and the corresponding item record information, and may acquire only the item name information. For example, in the case of the frame image 61 shown in FIG. 4, the frame image 61 does not include the item recording information corresponding to the item "age". Therefore, the character recognition unit 15 acquires only item name information. For example, when the item recording information is not included in the object to be photographed, when it is not written on the form from the beginning, or when it is unclear and the characters cannot be recognized. Therefore, even if the form information table 1451 is generated, the item record information may not be recorded. On the other hand, the character recognition unit 15 cannot separate the item record information without the item name information, and cannot acquire only the item record information.

The character recognition unit 15 reads the item numbers corresponding to all the acquired item names from the form definition table 1441, and associates the read item numbers with the acquired item record information. When the character recognition unit 15 completes acquiring the item name information and the item recording information in one frame image, it creates a new form information table 1451 in the form information storage unit 145 and stores the item in the generated form information table 1451 . A number, item recording information corresponding to the item number, and frame identification information are written and stored. When character information and coordinate information cannot be obtained from a frame image, the character recognition unit 15 does not obtain frame identification information corresponding to the frame image, and the form information table 1451 is not generated in the form information storage unit 145.

Upon receiving instruction information to end imaging from the operation unit 10 , the selection unit 16 selects a form information table 1451 to be saved from among the form information tables 1451 stored in the form information storage unit 145 . How to select the form information table 1451 to be saved can be determined arbitrarily. Suppose that it is decided to select the storage target so as to keep it. In this case, for example, the selection unit 16 performs the following processing.

The selection unit 16 deletes from the form information storage unit 145 the form information table 1451 in which the item record information is not recorded. Further, the selection unit 16 selects, for example, any one of the form information tables 1451, and records all item record information included in the selected form information table 1451 in any of the other form information tables 1451. If so, all the form information tables 1451 are deleted as redundant information. By performing these deletion processes, duplication of item record information among the form information tables 1451 can be reduced, and item record information can be stored without omission.

The selection unit 16 finally selects the form information table 1451 remaining in the form information storage unit 145 as a storage target. After completing the selection of the storage target, the selection unit 16 outputs instruction information for starting image processing to the image processing unit 20 .

The image processing unit 20 calculates feature information indicating the features of the template image stored in the template image storage unit 143, and writes the calculated feature information of the template image to the template feature information storage unit 146 for storage. Here, the feature information is feature points and feature amounts obtained from the target image. A feature point is information indicating a characteristic point such as a corner in an image of interest, and a feature amount indicates what kind of feature the feature point of the image of interest has. Information. As an algorithm for calculating feature information, for example, KAZE (an example of an image feature extraction algorithm) is applied.

In addition, the image processing unit 20 selects and reads out one of the form information tables 1451 from the form information storage unit 145, and sets the frame identification information indicated in the “frame identification information” item of the read form information table 1451. A corresponding frame image is read from the frame image storage unit 141 . The image processing unit 20 calculates feature information of the read frame image, and performs matching processing on the calculated feature information of the frame image, the frame identification information, and the frame image. When the form information storage unit 145 runs out of the form information table 1451 to be processed, the image processing unit 20 outputs instruction information to start transmission processing to the recognition data management unit 17 .

In addition, the image processing unit 20 performs matching (for example, feature value matching).
Based on the matching result, the image processing unit 20 converts the frame image so that it matches the size of the book template image and the positions of the items and ruled lines, and outputs the converted frame image and frame identification information. Here, the transforming process, for example, calculates a homography matrix from the matching result as follows, and performs perspective transformation for transforming the frame image based on the calculated homography matrix.

In this embodiment, the projective transformation matrix of the frame image is a matrix calculated based on the coordinate information of the points paired by the matching process, and is a matrix corresponding to the parameters for perspective transformation. Therefore, in the perspective transformation, coordinate transformation is performed on a frame image that has been distorted, enlarged, or reduced at the time of photographing, so that it corresponds to the coordinates of the template image that can be seen from the front. Therefore, the coordinate system of the frame image after perspective transformation and the coordinate system of the template image match.

The image processing unit 20 also reads the template image from the template image storage unit 143 , writes this template image to the superimposed image storage unit 147 as initial image data of the data of the superimposed image, and generates a superimposed state table 1471 . In addition, after perspectively transforming the frame image, the image processing unit 20 matches the coordinates of the perspectively transformed frame image with the superimposed image read out from the superimposed image storage unit 147, and superimposes a new coordinate on the front of the superimposed image. A superimposed image is generated, and the superimposed image storage unit 147 is overwritten with the generated superimposed image.

In addition, the image processing unit 20 writes the frame identification information corresponding to the superimposed frame image to the item “frame identification information” in the superimposed state table 1471 stored in the superimposed image storage unit 147, and writes the frame identification information to the item “arrangement state”. , writes any information of "-", "above" and "below" corresponding to the positional relationship between the newly added frame image and the previous frame image.

When the recognition data management unit 17 receives instruction information for starting transmission processing from the image processing unit 20, the recognition data management unit 17 recognizes the information indicated in the item “form identification information” of the form definition table 1441 stored in the form definition information storage unit 144. The form identification information and the superimposed image and superimposed state table 1471 stored in the superimposed image storage unit 147 are read. Further, the recognition data management unit 17 reads out all the form information tables 1451 stored in the form information storage unit 145 . The recognition data management unit 17 reads from the frame image storage unit 141 the frame image corresponding to the frame identification information indicated in the item “frame identification information” of the read form information table 1451 .

In addition, the recognition data management unit 17 associates all the read information with the user identification information assigned to the terminal device 1 in advance, and transmits the information to the form data management device 3 through the communication unit 18 . Here, the user identification information is information that individually identifies a user. 3, the user ID (IDentification) at the time of logging in may be used.

The form data management device 3 is connected to the communication network 4 as, for example, a web server device, and includes a recording processing section 31 and a storage section 32 . The recording processing unit 31 receives information transmitted by the recognition data management unit 17 of the terminal device 1 through the communication network 4 and writes the received information into the storage unit 32 .

The storage unit 32 stores a form information total table 321 shown in FIG. The form information total table 321 includes "user identification information", "form identification information", "superimposed image information", "superimposed image configuration information (frame identification information)", "pre-conversion frame image information", "item number", It has items of "acquired information" and "display state".

The user identification information, the form identification information, and the superimposed image data received from the recognition data management unit 17 are written in the items of "user identification information", "form identification information", and "superimposed image information" by the recording processing unit 31. . In the item of "superimposed image configuration information (frame identification information)", the frame identification information of the item of "frame identification information" of the superimposed state table 1471 received from the recognition data management unit 17 is displayed in the order recorded in the item. Therefore, it is written by the recording processing unit 31 . In the item of "frame image information before conversion", the recording processing unit 31 writes the data of the frame image before perspective change corresponding to the frame identification information recorded in the "superimposed image configuration information (frame identification information)". .

The “item number” and “acquisition information” items include the “item number” item and the “acquisition information” item of the form information table 1451 corresponding to the frame identification information recorded in the “superimposed image configuration information (frame identification information)”. Information” item is written by the recording processing unit 31 . In the "display status" item, the item record information displayed in the superimposed image among the item record information recorded in the "acquired information" item is indicated by "○", and the item record information not displayed is indicated by "○". is written. As a result, for example, the user can later confirm which item record information is to be saved while referring to the duplicate image.

(Image management processing by terminal device)
Next, referring to FIGS. 10 to 17, a frame image obtained by photographing a form and an image for storing text information obtained from the frame image in the storage unit 32 of the form data management device 3 in association with the template image. Management processing will be described.

The application storage unit 142 of the terminal device 1 stores a pre-downloaded AP for photographing a form. The template image storage unit 143 stores the data of the template image shown in FIG. I remember. In response to the user's instruction to activate the AP for photographing a form, the terminal device 1 activates the AP for photographing a form stored in the application storage unit 142 .

The activated AP for photographing a form activates the photographing unit 11 and the display unit 12 and displays a photographing start button and a photographing end button on the screen of the display unit 12 . The display unit 12 starts preview display for displaying an image formed on the imaging device of the imaging unit 11 on the screen.

FIG. 10 is a flow chart showing the flow of the document photographing process by the terminal device 1 in the image management process. While referring to the preview display on the display unit 12, the user points the lens of the photographing unit 11 toward the top area of the form shown in FIG. 3, and selects the photographing start button (step Sa1).

When the shooting start button is selected, the operation unit 10 initializes the frame image storage unit 141, the form information storage unit 145, and the superimposed image storage unit 147, and outputs shooting start instruction information to the shooting unit 11. do. Upon receiving instruction information to start shooting from the operation unit 10, the shooting unit 11 starts a process of continuously shooting at a frame rate of 10 to 30 frames per second, and generates frame images for each shooting. Output to the image acquisition unit 13 .

While referring to the preview display on the display unit 12 of the terminal device 1, the user moves the terminal device 1 from the beginning to the end of the form, with a part of the form as the shooting target. Accordingly, the user does not perform an operation such as pressing the shutter button. Then, the photographing unit 11 photographs a frame image of a part of the form from the beginning to the end of the form at, for example, 10 to 30 frames per second. In this case, the user does not need to perform a special operation such as pressing the shutter button, and shoots as if shooting a moving image.

When acquiring the frame image output by the photographing unit 11, the frame image acquisition unit 13 generates new frame identification information, associates the generated frame identification information with the acquired frame image, and writes the generated frame identification information to the frame image storage unit 141. store (step Sa2). Here, "1" is generated as frame identification information for the frame image 61 and associated with it.

When a new frame image is written in the frame image storage unit 141, the character recognition unit 15 performs character recognition processing on the written frame image, and extracts character information and coordinate information corresponding to the character information from the frame image. Detect (step Sa3).

The character recognition unit 15 determines whether or not a combination of one or more character information and coordinate information has been detected from one frame image, and determines whether or not the OCR processing has succeeded (step Sa4). If the character recognition unit 15 does not detect one or more combinations of character information and coordinate information from one frame image, the character recognition unit 15 determines that the OCR process is unsuccessful (step Sa4, No), and advances the process to step Sa2. , the frame image acquiring unit 13 acquires the next frame image.

On the other hand, when the character recognition unit 15 detects a combination of one or more character information and coordinate information from one frame image, it determines that the OCR processing has succeeded (step Sa4, Yes), and all the detected character information and Frame identification information of a frame image is associated with a combination of coordinate information.

The character recognition unit 15 corresponds to all the character information and coordinate information corresponding to the frame identification information and the form definition table 1441 of the form definition information storage unit 144, and recognizes item names and item record information from the character information. is detected and obtained. The character recognition unit 15 refers to the form definition table 1441 and associates the acquired item record information with the item number (step Sa5).

The character recognition unit 15 generates a new form information table 1451 in the form information storage unit 145, and writes the frame identification information to the “frame identification information” item of the generated form information table 1451. FIG. The character recognition unit 15 writes the acquired item number in the item of "item number" and writes the item recording information corresponding to the item number in the item of "acquired information" for storage (step Sa6). For example, in the case of the frame image 61 shown in FIG. 4, the character recognition section 15 generates a form information table 1451-1 shown in FIG. Here, the item number "007" is the item "age", and since the item record information corresponding to the item name "age" is not included in the frame image 61 as shown in FIG. At -1 only the item number is written.

The character recognition unit 15 determines whether or not the shooting end button has been selected from the operation unit 10 (step Sa7). When the character recognizing unit 15 determines that it has received instruction information to end shooting (step Sa7, Yes), it ends the process. On the other hand, when the character recognition unit 15 determines that the instruction information to end shooting has not been received (step Sa7, No), the process proceeds to step Sa2, and the frame image acquisition unit 13 acquires the next frame image.

For example, next, when the frame image 62 shown in FIG. 5 is acquired by the frame image acquisition unit 13, the form information table 1451-2 shown in FIG. 12 is obtained by the frame image obtaining unit 13, a form information table 1451-3 shown in FIG. 13 is generated in the form information storage unit 145. FIG.

FIG. 14 is a flowchart showing a flow of processing for selecting a form information table to be stored by the terminal device 1 in the image management processing. When receiving instruction information to end shooting from the operation unit 10, the selection unit 16 refers to the form information storage unit 145 and deletes all the form information tables 1451 that do not contain item record information (step Sb1).

Next, the selection unit 16 reads out the last form information table 1451 among the unprocessed form information tables 1451, that is, not yet to be processed, from the form information storage unit 145 (step Sb2).

The selection unit 16 stores each of the item record information recorded in the item of “acquired information” of the read-out form information table 1451 as stored in any of the other form information tables 1451 of the form information storage unit 145 . It is determined whether or not it is an object (step Sb3).

The selection unit 16 records each of the item record information recorded in the “acquired information” item of the form information table 1451 in one of the other form information tables 1451 stored in the form information storage unit 145 . If so, it is determined that it is not to be saved (step Sb3, No), and the read form information table 1451 is deleted from the form information storage unit 145. FIG.

On the other hand, the selection unit 16 selects which of the other form information tables 1451 stored in the form information storage unit 145 each of the item record information recorded in the item “acquired information” of the form information table 1451 is. is not recorded, it is determined to be a storage target (step Sb3, Yes). This is because there is item record information that is lost when the form information table 1451 is deleted. The selection unit 16 determines whether or not the unprocessed form information table 1451 exists in the form information storage unit 145 (step Sb5).

If the selection unit 16 determines that the unprocessed form information table 1451 exists in the form information storage unit 145 (step Sb5, Yes), the process proceeds to step Sb2. On the other hand, when the selection unit 16 determines that the unprocessed form information table 1451 does not exist in the form information storage unit 145 (step Sb5, No), the selection unit 16 outputs instruction information for starting image processing to the image processing unit 20 ( Step Sb6), the process ends.

In addition, in the process of step Sb2 shown in FIG. 14, the selection unit 16 reads out the form information table 1451 last written in the form information storage unit 145 among the unprocessed form information tables 1451. For example, The form information table 1451 written in the form information storage unit 145 first may be read.

FIG. 15 is a flow chart showing the flow of processing for superimposing a frame image on a template image by the terminal device 1 to generate a superimposed image in the image management processing.

Upon receiving instruction information for starting image processing from the selection unit 16 , the image processing unit 20 calculates feature information of the template image stored in the template image storage unit 143 and writes the calculated feature information to the template feature information storage unit 146 . memorize with When the image processing start instruction information is supplied, the image processing unit 20 reads the template image from the template image storage unit 143 and writes the read template image to the superimposed image storage unit 147 as the initial image data of the data of the superimposed image. . Also, the image processing unit 20 generates a superimposed state table 1471 in the superimposed image storage unit 147 (step Sc1).

In addition, the image processing unit 20 detects an unprocessed form information table 1451 from the form information storage unit 145, and selects the first written form information table 1451 in the detected unprocessed form information table 1451 to the form information storage unit 145. The form information table 1451 is read (step Sc2). That is, when the image processing unit 20 arranges the detected unprocessed form information tables 1451 in the order written in the form information storage unit 145, the image processing unit 20 reads out the form information tables 1451 in this order.

The image processing unit 20 reads from the frame image storage unit 141 the frame image corresponding to the frame identification information in the item "frame identification information" of the read form information table 1451 (step Sc3). The image processing unit 20 calculates feature information of the read frame image (step Sc4).

The image processing unit 20 compares the calculated feature information of the frame image with the feature information of the template image in the template feature information storage unit 146. For example, as shown in FIG. A matching process for detection is performed (step Sc5). That is, the image processing unit 20 detects, for example, the coordinates of the character "ken" indicated by reference numeral 700 in the frame image 61 and the coordinates of the character "ken" indicated by reference numeral 600 in the template image 50 as a pair. The image processing unit 20 also detects the coordinates of the corner 701 of the frame image 61 and the coordinates of the corner 601 of the template image 50 as a pair.

The image processing unit 20 calculates a projective transformation matrix corresponding to the matching result, and perspectively transforms the frame image using the calculated projective transformation matrix (step Sc6).

After perspectively transforming the frame image, the image processing unit 20 reads out the superimposed image from the superimposed image storage unit 147 . The image processing unit 20 generates a new superimposed image by superimposing the frame image in front of the superimposed image so that the coordinates of the read superimposed image match, and overwrites the superimposed image storage unit 147 with the generated superimposed image. The image processing unit 20 writes the frame identification information corresponding to the frame image superimposed on the superimposed state table 1471 stored in the superimposed image storage unit 147 in the “frame identification information” item, and updates the “arrangement state” item. (Step Sc7).

In the case of the frame image 61, only one frame image 61 does not overlap with the template image 50, and there is no preceding frame image. Therefore, the image processing unit 20 writes “-” in both the upper and lower columns of the “arrangement state” item of the frame identification information “1” corresponding to the frame image 61 . In the case of the frame image 62 next to the frame image 61, the image processing unit 20 writes "bottom" in the column under the item "arrangement state" of the frame identification information "1" corresponding to the frame image 61, "Up" is written in the upper column of the "arrangement state" item of the frame identification information "2" corresponding to the frame image 62, and "-" is written in the lower column.

After completing the process of step Sc7, the image processing unit 20 determines whether or not an unprocessed form information table 1451 exists in the form information storage unit 145 (step Sc8). When the image processing unit 20 determines that the unprocessed form information table 1451 exists in the form information storage unit 145 (step Sc8, Yes), the process from step Sc2 is performed.
On the other hand, when the image processing unit 20 determines that the unprocessed form information table 1451 does not exist in the form information storage unit 145 (Step Sc8, No), the image processing unit 20 sends the recognition data management unit 17 instruction information to start transmission processing. to output

For example, when the frame images 61, 62, and 63 are captured, the superimposed image storage unit 147 stores the superimposed image 40 shown in FIG. generated. In the superimposed image 40, the template image 50 exists in the rearmost plane, and the frame images 61-r, 62-r, and 63-r are the perspective-transformed frame images 61, 62, and 63, respectively.

First, the frame image 61-r is overlaid on the template image 50, then the frame image 62-r and the frame image 63-r are overlaid in this order. As described above, in the first embodiment, the frame image to be superimposed later is superimposed so that the frame image 61-r and part of the frame image 62-r are not visible. ing. Also, a superimposed state table 1471 shown in FIG. 8 is finally generated in the superimposed image storage unit 147 .

For example, when the frame images 61, 62, and 63 are captured, the recognition data management unit 17, upon receiving instruction information for starting transmission processing from the image processing unit 20, stores the form definition table stored in the form definition information storage unit 144. The form identification information "0001" shown in the item 1441 "form identification information" is read. The recognition data management unit 17 also reads out the superimposed image 40 and the superimposed state table 1471 stored in the superimposed image storage unit 147 . The recognition data management unit 17 also reads out the form information tables 1451-1, 1451-2 and 1451-3 stored in the form information storage unit 145. FIG. The recognition data management unit 17 retrieves the frame identification information "1", "2", "3" indicated in the "frame identification information" item of the read form information tables 1451-1, 1451-2 and 1451-3. frame images 61 , 62 , 63 corresponding to . Further, the recognition data management unit 17 associates all the read information with the user identification information assigned in advance to the terminal device 1, and transmits the information to the form data management device 3 through the communication unit 18 (step Sc9). , terminate the process.

When the recording processing unit 31 of the form data management device 3 receives the information transmitted by the recognition data management unit 17 of the terminal device 1 through the communication network 4, it writes the received information into each item of the form information total table 321 and stores it. .

The recording processing unit 31 refers to the “arrangement state” information of the superimposition state table 1471 received from the recognition data management unit 17 when writing information in the “display state” item of the form information total table 321 . The recording processing unit 31 sets the display state of the frame image whose item number is the same as that of the frame image before and after the frame image whose item number is the same as the display state of the frame image whose “arrangement state” information is “top”. The display state corresponding to the frame image that is "lower" is recorded as "x".

For example, from the frame image 61 shown in FIG. 4, as shown in the form information table 1451-1 in FIG. has been obtained. Also, from the frame image 62 shown in FIG. 5, as shown in the form information table 1451-2 in FIG. Record information is obtained. Therefore, in the frame image 61 and the frame image 62, the item number "005" "sex", the item number "006" "medical checkup date", and the item number "007" "age" are duplicated.

As shown in the superimposed image 40 in FIG. 17, the frame image 62-r is superimposed on the front surface of the frame image 61-r. Therefore, as shown in FIG. , the column below the frame identification information "1" corresponding to the frame image 61-r is "lower", and the column above the frame identification information "2" corresponding to the frame image 62-r is "upper". "It has become. Therefore, as shown in FIG. 9, the recording processing unit 31 sets the item numbers "005", "006", and "007" corresponding to the frame identification information "1" in the "display state" item of the form information total table 321. , and writes "o" for the item numbers "005", "006", and "007" corresponding to the frame identification information "2".

In the above-described first embodiment, the image processing unit 20, in the process of step Sc2 shown in FIG. 1451 is read, for example, the form information table 1451 last written in the form information storage unit 145 may be read. As shown in FIG. 15, when the form information table 1451 first written in the form information storage unit 145 is processed in order, as shown in FIG. A new frame image 62-r is superimposed in front of r. Conversely, if the form information table 1451 lastly written in the form information storage unit 145 is processed in order, for example, when arranged in chronological order, the old frame image 62-r is placed in front of the new frame image 63-r. be superimposed. Further, the image processing unit 20 may randomly read out the form information table 1451 from the unprocessed form information table 1451 .

Further, in the above-described first embodiment, the terminal device 1 is moved from the top to the end of the form to take an image. Random areas may be photographed.

Further, in the first embodiment described above, the processing shown in FIG. 14 is performed after the processing shown in FIG. 10 is completed, and the processing shown in FIG. 15 is performed after the processing shown in FIG. 14 is completed. . When shooting arbitrary areas of a form at random, it is necessary to do so in this order. When photographed, the obtained frame images are in chronological order, and duplication of item record information between frame images can be confirmed within a certain range of frame images. Therefore, when frame images are obtained in time series, the process shown in FIG. 14 is started after the process shown in FIG. The processing shown in FIG. 15 may be started and the processing shown in FIGS. 10, 14, and 15 may be performed in parallel.

In the above-described first embodiment, the recognition data management unit 17 stores the frame images corresponding to the frame identification information indicated in the "frame identification information" item of all the read form information tables 1451. Although the frame image is read from the unit 141 , the frame image corresponding to the frame identification information indicated in the “frame identification information” of the superimposed state table 1471 may be read from the frame image storage unit 141 .

In the configuration of the first embodiment described above, in the terminal device 1, the frame image acquisition unit 13 sets the arbitrarily continuously determined partial area of the form as the shooting target, and each time the shooting unit 11 successively shoots the Get the resulting frame image. The character recognition unit 15 performs character recognition processing for each frame image. The image processing unit 20 converts the frame image corresponding to the feature information indicating the feature of the image of the form template given in advance and the feature information for each frame image in which the character information is detected by the character recognition process, and converts the frame image. Overlay the frame image on the template image. As a result, the template image can be associated with the frame image from which the character information has been obtained among the frame images obtained by photographing a part of the form. You can easily find out if there is In addition, since the frame images are superimposed based on the template image, even if the frame image is shot tilted, the tilt can be corrected. A good superimposed image is obtained. In addition, even if there is an omission of photographing, it can be easily found.

(Another configuration example of the first embodiment)
In the first embodiment described above, the image processing unit 20 superimposes the frame image on top of the frame image already superimposed on the template image 50 when superimposing the frame image in the process of step Sc7 shown in FIG. rice field. When performing the process of step Sc7, the image processing unit 20 may change the order in which the frame images are superimposed based on the recording status of the items "item number" and "acquired information" in the form information table 1451. . For example, in the frame image 62a shown in FIG. 18, the item record information of "gender" and the item record information of "medical examination date" are unclear. Therefore, the character recognition unit 15 cannot acquire the item record information for these items, and acquires only the character information of the item names "sex" and "medical examination date".
When the processing of the image processing unit 20 described in the first embodiment is applied to the frame image 62a, an image obtained by perspectively transforming the frame image 62a (hereinafter referred to as a frame image 62a-r) is superimposed on the frame image 61-r. , the information of “sex”/“male” and “date of medical checkup”/“June 10, 2017” displayed in the frame image 61 disappears in the superimposed image 40 .

FIG. 19 shows a form information table 1451a-2 generated in the form information storage unit 145 corresponding to the frame image 62a. The image processing unit 20 refers to the items of "acquired information" corresponding to the common item numbers "005" to "007" of the form information table 1451-1 and the form information table 1451a-2 shown in FIG. . For the item numbers "005" and "006", "Male" and "June 10, 2017" are recorded as item record information in the form information table 1451-1. Item record information is not recorded in 1451a-2.

In such a case, the image processing unit 20 determines that the item record information corresponding to the item numbers “005” and “006” is hidden when the frame images 62a-r are superimposed on the frame images 61-r. 20, the frame images 62a-r are superimposed on the back of the frame image 61-r. In addition, as shown in FIG. 21, the image processing unit 20 writes "above" in the column below the item "arrangement state" of the frame identification information "1" in the superimposition state table 1471, and sets the frame identification information "2". Write "bottom" in the upper column of the "placement state" item.

In another configuration example of the first embodiment described above, when a frame image is superimposed on a template image, if the character information overlaps with the frame image superimposed one before, the image processing unit 20 replaces both frame images. Which frame image should be superimposed is selected based on the included character information. As a result, in a superimposed image generated by superimposing a frame image on a template image, character information is not hidden by other frame images, and a large amount of character information can be displayed.

(Second embodiment)
The second embodiment of the present invention is similar in configuration to the first embodiment, but differs in operation. Only the operations of the character recognition unit 15 and the image processing unit 20 that differ from those of the first embodiment will be described below.
The character recognition unit 15 performs the following processes in addition to the processes of the first embodiment. That is, the character recognition unit 15 stores data in the form information storage unit 145 in which the item "coordinate information" is added as shown in FIG. 22 instead of the data format of the form information table 1451 of the first embodiment. Generate a format table. This form information table 1451b-1 shown in FIG. In the second embodiment, any one table stored in the form information storage unit 145 is indicated as a form information table 1451b without a branch number. The four pieces of coordinate information written in the "coordinate information" in FIG. 22 are coordinate information indicating the area surrounding the item name corresponding to the item number of the record and the character information of the item record information. The character recognition unit 15 calculates four pieces of coordinate information based on the coordinate information for each piece of character information, and writes them to the form information table 1451b.

In the example shown in the other configuration example of the first embodiment, the item number "007" common to the form information table 1451-1 shown in FIG. 7 and the form information table 1451a-2 shown in FIG. In this case, there is no information in the "acquired information" item of the form information table 1451-1, and the item record information "40 years old" exists in the "acquired information" item of the form information table 1451a-2. rice field. In the frame image 61 shown in FIG. 4, the column for recording the item record information corresponding to the item name "Age" is obliquely cut. "40 years old" displayed on r was visible. On the other hand, if the item record information corresponding to the item name "Age" of the frame image 61 is recorded only indistinctly and the frame image 61 has an area for recording the item record information, the frame image When the frames 61-r are superimposed, the "40 years old" displayed in the frame images 62a-r are hidden.

Therefore, in the second embodiment, the image processing unit 20 performs the following processing together with the processing of the first embodiment. After superimposing the frame image 61-r on the template image 50, the image processing unit 20 refers to the “acquired information” item of the form information table 1451b-1, and records the item recording information in the “acquired information” item. Detect lines that are not In the case of the form information table 1451b-1, the item number "007" is detected.

The image processing unit 20 extracts four pieces of coordinate information (xg1, yg1), (xg2, yg2), (xg3, yg3), (xg4, yg4) of the item of "coordinate information" corresponding to the detected item number "007". ). The image processing unit 20 calculates the unnecessary area 150 of the frame image 61-r as shown in FIG. The calculated unnecessary area 150 is removed from the frame image 61-r superimposed on the template image 50. FIG. In the case of the frame image 62a, the image processing unit 20 calculates unnecessary regions 1501 and 1502 in the frame images 62a-r, and removes the unnecessary regions 1501 and 1502 from the frame images 62-r, as shown in FIG. .

When calculating the unnecessary areas 150, 1501, and 1502, the image processing unit 20 may calculate the unnecessary areas 150, 1501, and 1502 so that the unnecessary areas 150, 1501, and 1502 can be removed at the ruled line positions. For example, in the case of the frame image 61, the image processing unit 20 detects the coordinate information of the horizontal ruled line of "age", which is an item for which item recording information does not exist, and uses the detected coordinate information to detect the horizontal ruled line. An unnecessary area 150 along the line is calculated. In the case of the frame image 62a, the image processing unit 20 detects the coordinate information of the horizontal ruled lines of the items "sex", "medical examination date", and "viewing", which are items for which item recording information does not exist, and Using the information, unnecessary areas 1501 and 1502 along the horizontal ruled lines are calculated. The positional relationship between the frame image 61b-r after the unnecessary regions 150, 1501, 1502 have been removed and the frame image 62b-r is the positional relationship shown in FIG. It can solve the problem of hidden information.

Instead of removing the unnecessary areas 150, 1501, and 1502 as described above, the areas other than the unnecessary areas 150, 1501, and 1502, i.e., the areas where the item recording information exists, are cut by ruled lines, and the cut areas are used as templates. It may be superimposed on the relevant portion of the image 50 . Since the item name is shown in the template image 50, the item record information recorded in the form information table 1451 is used as character information, and the character size is adjusted to match the resolution of the template image 50. 50 may be arranged correspondingly.

Further, in the first and second embodiments, the template image storage unit 143 is stored in advance with an image of the template read by an external image scanner or the like. On the other hand, a template image obtained by photographing a still image of the template by the photographing unit 11 may be stored in the template image storage unit 143 .
Further, in the first and second embodiments, if the image processing unit 20 performs perspective transformation based on an erroneous matching result, the transformed image may become unreadable. For example, the smaller the value of the distance value of the matching result, the higher the accuracy. may be used to improve matching accuracy. Also, if the size of the frame image obtained as a result of the perspective transformation by the image processing unit 20 is too large, it may be excluded as the transformation has failed.

In addition, in the configurations of the first and second embodiments described above, if another form or an arbitrary object is caught in the image when the form is photographed, the other form or the arbitrary object is not captured in the processing by the image processing unit 20. may be calculated as a feature. Therefore, when the image processing unit 20 reads out the frame image from the frame image storage unit 141, the line indicating the outer circumference of the frame image may be detected, and the characteristic information of the frame image may be calculated after trimming the outer circumference. .

In addition, in the first and second embodiments, the terminal device 1 includes the image processing unit 20 and the superimposed image storage unit 147, and functions as one "image processing device" that performs image processing for generating a superimposed image. However, the image processing unit 20 and the superimposed image storage unit 147 may be separated, and an external device (for example, an image processing server device, etc.) may include the image processing unit 20 and the superimposed image storage unit 147. . In this case, the terminal device 1 outputs the information stored by the frame image storage unit 141, the information stored by the form template image storage unit 413, and the information stored by the form information storage unit 145 to an external device. The image processing unit 20 of the external device generates a superimposed image based on the information output by the terminal device 1 and outputs the finally obtained superimposed image to the terminal device 1 . In such a case, the terminal device 1 and the external device are integrated to form an "image processing device" that generates a superimposed image.

In addition, in the first and second embodiments, the superimposition state table 1471 stores the information of the “arrangement state” indicating which frame image is on top. The superimposition state table 1471 may not be provided if predetermined processing such as superimposing an image or superimposing an image on the bottom is performed. In this case, since the recording processing unit 31 cannot refer to the superimposition state table 1471, it is not necessary to provide the "display state" item of the form information total table 32 shown in FIG.

The terminal device 1 in the above-described embodiment may be realized by a computer. In that case, a program for realizing this function may be recorded in a computer-readable recording medium, and the program recorded in this recording medium may be read into a computer system and executed. It should be noted that the "computer system" referred to here includes hardware such as an OS and peripheral devices. The term "computer-readable recording medium" refers to portable media such as flexible discs, magneto-optical discs, ROMs and CD-ROMs, and storage devices such as hard discs incorporated in computer systems. In addition, "computer-readable recording medium" refers to a volatile memory inside a computer system that acts as a server or client when sending a program via a network such as the Internet or a communication line such as a telephone line. It may also include those holding programs. Further, the above program may implement a part of the functions described above, or may implement the functions described above in combination with a program already recorded in a computer system, such as FPGA (Field Programmable Gate Array). It may be implemented using a programmable logic device.

Although the embodiments of the present invention have been described in detail above with reference to the drawings, the specific configuration is not limited to these embodiments, and designs and the like are included within the scope of the present invention.

DESCRIPTION OF SYMBOLS 1... Terminal device 3... Form data management apparatus 4... Communication network 10... Operation part 11... Photographing part 12... Display part 13... Frame image acquisition part 14... Storage part 15... Character recognition part 17... Recognition data management part 18... Communication Section 20... Image processing section 31... Recording processing section 32... Storage section

Claims (6)

a frame image obtaining unit that obtains frame images obtained by partially continuously photographing one form;
a character recognition unit that performs character recognition of the character image in the frame image;
an image processing unit that superimposes the frame image on the template image based on the feature information of the template image of the form and the feature information of the frame image whose character information is obtained by the character recognition ,
The image processing unit
When the frame image is superimposed on the template image, if the character information overlaps with the previously superimposed frame image, the frame image in which the character information is recognized in the overlapped portion is placed upward. overlap
An image processing apparatus characterized by: a frame image obtaining unit that obtains frame images obtained by partially continuously photographing one form;
a character recognition unit that performs character recognition of the character image in the frame image;
an image processing unit that superimposes the frame image on the template image based on the feature information of the template image of the form and the feature information of the frame image whose character information is obtained by the character recognition ,
The image processing unit
When the frame image is superimposed on the template image, an area where the character information is not recognized is detected as an unnecessary area.
An image processing apparatus characterized by: a frame image acquisition process in which the frame image acquisition unit acquires frame images obtained by partially continuously imaging one form;
a character recognition process in which the character recognition unit performs character recognition of the character image in the frame image;
an image processing process in which an image processing unit superimposes the frame image on the template image based on the feature information of the template image of the form and the feature information of the frame image from which the character information is obtained by the character recognition. fruit,
The image processing process includes:
When the frame image is superimposed on the template image, if the character information overlaps with the previously superimposed frame image, the frame image in which the character information is recognized in the overlapped portion is placed upward. including overlapping
An image management method characterized by: a frame image acquisition process in which the frame image acquisition unit acquires frame images obtained by partially continuously imaging one form;
a character recognition process in which the character recognition unit performs character recognition of the character image in the frame image;
an image processing process in which an image processing unit superimposes the frame image on the template image based on the feature information of the template image of the form and the feature information of the frame image from which the character information is obtained by the character recognition. fruit,
The image processing process includes:
detecting an area where the character information is not recognized as an unnecessary area when the frame image is superimposed on the template image.
An image management method characterized by: the computer,
frame image acquisition means for acquiring frame images obtained by partially continuously photographing one form;
character recognition means for performing character recognition of the character image in the frame image;
a program for functioning as image processing means for superimposing the frame image on the template image based on the feature information of the template image of the form and the feature information of the frame image from which the character information is obtained by the character recognition; ,
In the image processing means,
When the frame image is superimposed on the template image, if the character information overlaps with the previously superimposed frame image, the frame image in which the character information is recognized in the overlapped portion is placed upward. including overlapping
program. the computer,
frame image acquisition means for acquiring frame images obtained by partially continuously photographing one form;
character recognition means for performing character recognition of the character image in the frame image;
a program for functioning as image processing means for superimposing the frame image on the template image based on the feature information of the template image of the form and the feature information of the frame image from which the character information is obtained by the character recognition; ,
In the image processing means,
detecting an area where the character information is not recognized as an unnecessary area when the frame image is superimposed on the template image.
program.

Images