JP2015102915A - Information processing apparatus, control method, and computer program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an information processing apparatus that can prevent characters included in a photographed document from being printed before being resolved.SOLUTION: A portable terminal 101 detects a document frame from a photographed image and specifies character blocks from an area surrounded by the document frame. The portable terminal 101 determines if characters included in the character blocks can be resolved, and if determined that the characters cannot be resolved, creates a recommendation document frame for resolving the characters. The portable terminal 101 then displays the recommendation document frame.

Description

  The present invention relates to an information processing apparatus having an imaging function.

  Using an information processing device such as a portable terminal equipped with a camera, it is now possible to capture a document and obtain a document image. In recent years, a portable terminal can transmit a photographed document image to a printing apparatus via a network and print it with a simple operation. That is, an information processing apparatus and a printing apparatus having a photographing function can be used as a simple digital multi-function peripheral.

  When a document image photographed with a mobile terminal is printed as it is, when the imaging surface is not parallel to the paper surface, the photographed image is distorted by the perspective effect. Patent Document 1 discloses an image processing apparatus that corrects distortion of a captured image.

JP 2000-307947 A

  However, when a document is photographed by an information processing apparatus having a photographing function, characters far from the photographing lens are difficult to resolve due to perspective effects and blurring. On the other hand, even if the character is close to the photographic lens, if the character size is small, it cannot be resolved unless the camera is further brought closer. In such a case, it is conceivable that even if the image processing apparatus disclosed in Patent Document 1 is applied to correct image distortion, an image including characters that cannot be resolved is output. Characters that could not be resolved do not look good when digitized. Furthermore, when printing is performed without confirming a captured image that includes characters that cannot be resolved, the characters that cannot be resolved are printed, which requires the user to re-shoot, etc. Will be damaged.

  An object of the present invention is to provide an information processing apparatus that prevents characters included in a photographed document from being output without being resolved.

  An information processing apparatus according to an embodiment of the present invention includes a detecting unit that detects a document frame from a photographed image, a specifying unit that specifies a character block from an area surrounded by the document frame, and a character included in the character block. A determination unit that determines whether the character can be resolved, a generation unit that generates a recommended document frame for enabling the character to be resolved when it is determined that the character cannot be resolved, and the recommended document frame. Display means for displaying.

  According to the information processing apparatus of the present invention, it is possible to prevent characters included in a photographed document from being output without being resolved.

It is a figure which shows an example of the system configuration | structure of this embodiment. It is a figure which shows an example of the external appearance of a portable terminal. It is a figure which shows an example of an internal structure of a portable terminal. It is a flowchart explaining the example of the whole operation | movement process of a portable terminal. It is a flowchart explaining the example of a detection process of a character area. It is a diagram for explaining detection of a block in a document frame. It is a figure explaining the detection of the characteristic of a character block. It is a flowchart explaining the determination process of the resolution of a character block. It is a figure explaining the division | segmentation of a character block, and determination of the resolution. 10 is a flowchart illustrating an example of a recommended document frame generation process. FIG. 6 is a diagram for explaining generation of a recommended document frame from a document frame. It is an example of a display of a recommendation frame. FIG. 10 is a diagram for explaining a comparison between a re-detected document frame and a recommended document frame. It is a flowchart explaining the example of the whole operation | movement process of a portable terminal. It is a display example of a warning screen.

Example 1
FIG. 1 is a diagram illustrating an example of a system configuration of the present embodiment.
The system shown in FIG. 1 includes a mobile terminal 101, a wireless router 102, a printer 104, and a server 105. A printer 104, a server 105, and a wireless router 102 are connected to the LAN 110. The portable terminal 101 can communicate with the printer 104 and the server 105 via the wireless router 102.

  The portable terminal 101 photographs a document placed on a desk (not shown) that is close to the horizontal. Of course, the mobile terminal 101 is not limited to a document placed on a desk, and may shoot, for example, a posting attached to a wall, such as a poster. The server 105 generates a print job corresponding to the image captured by the mobile terminal 101 and inputs it to the printer 104. The printer 104 executes the print job generated by the server 105 and prints it out.

  FIG. 2 is a diagram illustrating an example of the appearance of the mobile terminal according to the present embodiment. FIG. 2A shows the surface of the mobile terminal 101. A touch panel 201 and operation buttons 202 are provided on the surface. The touch panel 201 is a device in which a display device for displaying information, buttons, images, and the like necessary for operation in detail and a device for inputting a position by touching with a human finger or the like are combined.

  FIG. 2B shows the back surface of the mobile terminal 101. An imaging unit 203 is provided on the back surface. Note that the present invention is not limited to a portable terminal and can be applied to any information processing apparatus having a camera function. In other words, the present invention can be applied to a smartphone or a mobile phone having a camera function and a digital camera having a communication function. In addition, as a camera function included in the portable terminal 101 illustrated in FIG. 2B, an autofocus device (not illustrated) is mounted. With this autofocus device, it is possible to automatically detect the in-focus state based on the contrast ratio, infrared transmission / reception timing, etc., and measure the focal length and subject distance. The autofocus device used in this embodiment is merely an example, and any device that can measure the focal length and subject distance can be used.

FIG. 3 is a diagram illustrating an example of the internal configuration of the mobile terminal. FIG. 3 shows only the configuration necessary for this embodiment. The mobile terminal 101 includes a CPU 301 to an image processing unit 310.
The CPU 301 transmits and receives programs and data to and from the RAM 302 and ROM 303 via the data bus 311. The CPU 301 controls each processing unit included in the mobile terminal 101 based on a program stored in the ROM 303.

  In addition, a touch panel 201, an imaging device 203, an image processing unit 310, and a motion sensor 309 are connected to the data bus 311. The image processing unit 310 includes a document frame determination unit 3101, a character block detection unit 3102, a resolution determination unit 3103, a recommended document frame generation unit 3104, a document frame comparison unit 3105, and a distortion correction processing unit 3106.

  A document frame determination unit 3101 specifies a document region from frames of images (moving images) of documents that are continuously shot. The target of the document area specifying process is a captured image for confirming the image content. A character block detection unit 3102 detects a character block from an area surrounded by a document frame. The resolution determination unit 3103 determines the resolution of the character block. The recommended document frame generation unit 3104 generates a recommended document frame based on the determined resolution of the character block. The recommended document frame is a frame (guide frame) that is displayed when a document is photographed so that characters can be resolved. The document frame comparison unit 3105 compares the document frame obtained after the recommended document frame is displayed with the recommended document frame. The distortion correction processing unit 3106 corrects image distortion according to the comparison result between the document frame and the recommended document frame.

  Each processing unit included in the image processing unit 310 can be realized by a program calculation by a DSP or CPU. In addition, a wireless LAN device 306 is connected to the data bus 311, and data transmission / reception with the wireless router 102 is possible.

  The motion sensor 309 includes a three-axis acceleration sensor, an electronic compass, and a three-axis angular velocity sensor, and can detect the posture and movement of the mobile terminal 101 by using a known technique.

  FIG. 4 is a flowchart for explaining an example of the overall operation process of the mobile terminal. Unless otherwise specified, the processing of the flowchart of FIG. 4 is performed by the CPU 301 in accordance with a program stored in the ROM 303.

  First, the CPU 301 displays an image including a document imaged as a moving image by the imaging device 203 on the touch panel 201 (step S1001). At this time, each frame of the captured moving image is stored in the RAM 302.

  Next, the document frame determination unit 3101 detects and identifies a document frame from each frame image (step S1002). Specifically, the document frame determination unit 3101 performs an edge extraction process in each frame image, and specifies a closed rectangular object among the extracted edges as a document frame. Alternatively, specific marks are printed in advance at the four corners of the document, and the document frame determination unit 3101 detects these marks from the respective frame images, thereby specifying the four corners of the document and connecting them with straight lines to specify the document frame. You may make it do. The document frame determination unit 3101 stores the coordinates of the four corners forming the specified document frame in the RAM 302.

  Next, the character block detection unit 3102 detects a character region (character block) in the specified document frame (step S1003).

FIG. 5 is a flowchart for explaining an example of the character area detection processing in step S1003 of FIG.
First, the character block detection unit 3102 performs block division within the document frame (step S2001). Various methods have been studied and proposed for block division as area division. For example, the character block detection unit 3102 obtains a feature amount such as an edge of an image and obtains a region division from the distribution density of the edge. In addition, the character block detection unit 3102 may divide the feature amount distribution into blocks by pattern recognition or a statistical method. Thereby, for example, as shown in FIGS. 6A and 6B, a block can be detected in the document frame.

The character block detection unit 3102 stores the coordinates of the divided blocks and information indicating whether the block is a character block in a table (Table 1) in the RAM 302.

  For example, when the block is divided as shown in FIG. 6A, the coordinates of the block 1 and the block 2 and information indicating whether the block is a character block (“◯”, “ × ”) is stored in the table. When the blocks are divided as shown in FIG. 6B, the coordinates of the blocks 1 to 3 and information indicating whether or not the block is a character block are stored in the table. If no block is detected in step S2001 in FIG. 5, the process ends.

Returning to FIG. 5, the character block detection unit 3102 analyzes the divided blocks and identifies the character block in the processing after step S2002.
The character block detection unit 3102 determines whether all the blocks have been analyzed (step S2002). If all the blocks have been analyzed, the process ends. If there is a block that has not been analyzed, the process advances to step S2003.

  Next, the character block detection unit 3102 detects the feature of the character block (step S2003). Character blocks have various characteristics. However, since the document is a document photographed by the imaging apparatus, it may be difficult to analyze the feature due to blur. Therefore, it is desirable to combine detection methods. For example, the character block detection unit 3102 determines a character block if a character can be detected by OCR determination including binarization processing.

  If the block is large at the top and bottom of the document frame, as in block 2 in FIG. 6A, there are many places where the focus position of the imaging device 203 is in alignment. Therefore, since an area with less blur is included, determination by OCR processing is possible.

  Returning to FIG. 5, the character block detection unit 3102 determines whether there is a character block characteristic (step S2004). If a character block characteristic is detected even in a part of the block being calculated, the character block detection unit 3102 determines that the block is a character block (step S2005). If no character block feature is detected, the process returns to step S2002.

  As shown in FIGS. 7A and 7B, the character block detection unit 3102 may detect the feature of the character block based on the feature as a sentence. As shown in block 2 of FIG. 6B, even if the character region is small and the OCR determination is performed, the blur may be large and determination may not be possible. In this case, the feature that the space is vacant in the direction (line direction) perpendicular to the direction of writing characters is used. Specifically, the character block detection unit 3102 detects whether there is a pattern as shown in FIG. 7A or FIG. 7B by performing edge extraction or feature distribution using pattern matching. FIG. 7A shows a pattern when the character block is written horizontally or when a vertically written document is photographed horizontally. FIG. 7B shows a pattern when the character block is written vertically or when a horizontally written document is photographed vertically.

  Blocks determined to be character blocks are recorded as character blocks in the table of Table 1. As described above, the character block detection of S1003 can be executed by the processing of steps S2001 to S2004 in FIG.

  Returning to the description of FIG. The character block detection unit 3102 determines whether a character block is detected (step S1004). If no character block is detected, the process proceeds to step S1012. If a character block is detected, the process proceeds to step S1005. Subsequently, the resolution determination unit 3103 determines the resolution of the character block. The resolution determination method will be described in detail with reference to the flowchart of FIG.

FIG. 8 is a flowchart for explaining the character block resolution determination process.
First, the resolution determination unit 3103 divides the character block into rectangles (step S3001).

  FIG. 9 is a diagram for explaining division of character blocks included in a document and determination of resolution. The document shown in FIG. 9A is a horizontally written document. Due to the perspective effect, the upper and lower sides of the original have different numbers of pixels that make up a character even with the same number of points in the same character block. Therefore, the resolution determination unit 3103 divides the character block 1 into two rectangular areas, upper and lower A and B.

  Returning to FIG. 8, the resolution determination unit 3103 calculates the number of character pixels in each rectangular area (step S3002). Specifically, the resolution determination unit 3103 determines the number of pixels of the character (number of character pixels) by calculating the number of pixels corresponding to the height and width of the character in each rectangular area.

When determining the number of pixels constituting a character, it may be difficult to actually determine an edge for determining the height of the character by defocusing. In this case, for example, the resolution determination unit 3103 calculates the height using general image processing such as edge extraction processing and binarization processing using a spatial filter. In particular, when the focal depth of the imaging apparatus 203 is shallow, the image is further out of focus depending on the angle between the imaging apparatus 203 and the document and the distance from the focal position. The distance from the focal position is the distance (number of pixels) between the pixel of interest and the focused pixel in the document image. In a state where characters are out of focus, the height indexing accuracy is reduced only by the spatial filter and edge extraction processing. Therefore, the edge extraction process and the binarization threshold may be executed according to the setting values shown in Table 2 below.

  The angle used in Table 2 can be acquired from the motion sensor 309. However, when a document pasted on the wall is photographed with a portable terminal, the angle between the portable terminal and the document changes by 90 degrees. Therefore, the portable terminal 101 displays a screen for selecting whether the original on the desk or the original pasted on the wall on the touch panel 201 before shooting. Judgment is made by adding 90 degrees.

  Returning to FIG. 8, the resolution determination unit 3103 determines whether the number of character pixels in each rectangular area calculated in step S3002 is greater than or equal to the threshold (step S3003). If the number of character pixels is greater than or equal to the threshold, the resolution determination unit 3103 determines that the character included in the rectangular area can be resolved, and the process advances to step S3005. If the number of character pixels is not greater than or equal to the threshold, the resolution determination unit 3103 determines that characters included in the rectangular area cannot be resolved, and the process advances to step S3004.

  The threshold value is a value determined in advance according to the performance of the imaging device 203 such as depth of field and resolution. Moreover, since the threshold value represents the number of pixels that can be used as a character, it varies depending on the purpose. For example, if printing is to be performed by the printer 104, it is necessary to have beauty as a character. Therefore, the threshold value is increased to obtain a large character. On the other hand, if the purpose is to be recognized by OCR or the like, the size of the threshold is adjusted to the performance of OCR. For example, the amount of resolution necessary for printing is generally 300 dpi. In this case, the number of pixels of characters is required to be 30 pixels or more, and the threshold value is 30 pixels. On the other hand, 150 dpi is generally required for OCR processing. In this case, the threshold value is 15 pixels.

  The determination of the resolution of the rectangular area will be described with reference to FIG. Rectangular areas A to C shown in FIG. 9A indicate rectangular areas in an image of a horizontally written document. In the case of horizontal writing, the number of pixels in the height direction of the character is set as the number of character pixels. The character written in the rectangular area A is assumed to be composed of 45 pixels when the height of the character on the document is 3.5 mm. This has the same resolving power as that obtained when shooting at a resolution of 326 dpi.

  Characters written in the rectangular area B are photographed at a lower resolution than the rectangular area A due to the perspective effect when the height of the characters on the document is 3.5 mm. For example, when the height of the character is composed of 30 pixels, the rectangular area B is captured at 217 dpi.

  The character block 2 is photographed at a lower resolution due to the perspective effect when the height of the character on the document is 3.5 mm. For example, when the character height is composed of 15 pixels, the resolution of the area of the character block 2 is 109 dpi. In other words, since the number of pixels constituting the same character size changes, it can be determined whether or not the character can be resolved by comparing the number of character pixels with a threshold value. However, when the characters on the document are printed at large points, the height of the characters is increased. For example, even in the character block 2, if the character size is 5.0 mm, it becomes 28 pixels, so that the character is easily resolved.

  In general, many commercially available document readers have a 300 dpi product, and are capable of sufficiently reading 7-point (2.5 mm) characters. When a 2.5 mm character is read at a resolution of 300 dpi, the height and width of the character are composed of 30 pixels. Considering this, the character in the rectangular area B can be resolved, but the character block 2 is difficult to resolve.

  FIG. 9B shows a rectangular area when the document is vertically written. In the case of vertical writing, the number of pixels in the width direction of the character is the number of character pixels. For example, it is assumed that the rectangular area A in FIG. 9B is composed of 45 pixels when the character width on the document is 3.5 mm. This has the same resolving power as that obtained when shooting at a resolution of 326 dpi. It is assumed that the character width of the rectangular area B is composed of 30 pixels when the character width on the document is 3.5 mm. In this case, the rectangular area B is captured at a resolution of 217 dpi. Here, when the number of pixels of the character is less than the threshold number of reference pixels (for example, less than 20 pixels), the resolution determination unit 3103 determines that the document cannot be resolved even if the document is properly captured in advance. In this case, the portable terminal displays, for example, a warning screen shown in FIG. 15 in order to call attention that the touch panel 201 cannot properly process. On the warning screen shown in FIG. 15, a pop-up for inquiring about a place that cannot be resolved and whether to continue processing is displayed.

  Returning to FIG. 8, the resolution determination unit 3103 returns the position of the rectangular area where the number of character pixels is not greater than or equal to the threshold, that is, the number of pixels is insufficient (step S3004). Specifically, the resolution determination unit 3103 returns the coordinates on the touch panel 201 of the rectangular area.

  Next, the resolution determination unit 3103 determines whether the processing has been completed for all rectangular areas of the character block (step S3005). If there is a rectangular area that has not been processed, the process returns to step S3003. When the process is completed for all rectangular areas of the character block, the process exits from this subroutine.

  Returning to the description of FIG. The recommended document frame generation unit 3104 determines whether the resolution of characters included in the document is sufficient (step S1006). Specifically, the recommended document frame generation unit 3104 determines whether all the character blocks are configured with more character pixels than the threshold based on the resolution determination result in step S1005. When all the character blocks are configured with the number of character pixels equal to or greater than the threshold value, the recommended document frame generation unit 3104 determines that the resolution of the characters included in the document is sufficient, that is, the characters can be resolved. If there is a character block composed of character pixels smaller than the threshold value, the recommended document frame generation unit 3104 determines that the resolution of characters included in the document is insufficient.

  When the resolution of the characters included in the document is sufficient, the CPU 301 acquires a main image that is an image to be recorded (step S1012). If the resolution of the characters included in the document is insufficient, the process advances to step S1007, and the recommended document frame generation unit 3104 generates a recommended document frame.

FIG. 10 is a flowchart illustrating an example of recommended document frame generation processing.
The recommended document frame generation unit 3104 reads the number of character pixels in all rectangular areas of all character blocks created in step S1003 of FIG. 4 (step S4001).

  Next, the recommended document frame generation unit 3104 calculates the necessary magnification of all the rectangular areas read in step S4001 (step S4002). The necessary magnification is the ratio of the threshold used in the determination process of step S3003 in FIG. 8 to the number of character pixels in each rectangular area.

In the case of horizontal writing, the required magnification is calculated by the following formula.
Required magnification = threshold [pixel] ÷ text height [pixel]
In the case of vertical writing, the required magnification is calculated by the following formula.
Required magnification = threshold [pixel] ÷ character width [pixel]

  Next, the recommended document frame generation unit 3104 creates a recommended document frame by correcting the document frame based on the required magnification (step S4003). Specifically, the recommended document frame generation unit 3104 adjusts so that the necessary magnification of all rectangular areas is 1 or less, that is, all rectangular areas have the number of character pixels greater than or equal to the threshold value. Create a recommended document frame. The magnification of 1 or less means a state in which the number of pixels of the character height and width is equal to or greater than the threshold number of pixels. When the photographer refers to the recommended document frame and shoots the document, the number of pixels constituting the characters of the photographed image increases and resolution is facilitated.

  FIG. 11A shows a screen example of the portable terminal in a state where the document frame is detected. Since the required magnification is 0.5 times, the rectangular area A shown in FIG. The rectangular area B can be resolved because the required magnification is 1.0. However, since the necessary magnification of the rectangular area C is 1.1 times, it is necessary to acquire a large image in order to resolve the character portion. In this case, in order to photograph the rectangular area C large, the recommended document frame generation unit 3104 adjusts the upper part of the document frame in FIG. A recommended document frame is generated without adjustment.

  FIG. 11B is a screen of the mobile terminal that displays a recommended document frame created by correcting the document frame of FIG. The recommended document frame generation unit 3104 changes the top two of the four vertices of the document frame, and newly recommends the vertex of the recommended document frame so that the entire rectangular area C is enlarged by 1.1 times. Is generated. In other words, the recommended document frame generation unit 3104 converts the document frame so that the region in the vicinity of the rectangular region in which the number of character pixels is determined to be smaller than the threshold among the regions surrounded by the document frame is enlarged and the recommended document is converted. Generate a frame.

  Returning to the description of FIG. The recommended document frame generation unit 3104 displays the recommended document frame on the touch panel 201 as shown in FIG. 12, for example (step S1008). The recommended frame generation unit 3104 displays a recommended document frame and marks (highlights) a region where the number of character pixels is lower than the threshold among the regions surrounded by the recommended document frame, thereby prompting the photographer to pay attention. . The photographer points the portable terminal 101 at the document with reference to the displayed recommended document frame. As a result, the captured image of the original is reacquired.

  Returning to FIG. 4, the document frame determination unit 3101 performs re-detection of the document frame based on the re-acquired captured image of the document (step S1002). The document frame re-detection method is the same as the document frame detection method in step S1002.

  The document frame comparison unit 3105 compares the document frame redetected in step S1009 with the recommended document frame (step S1010). Since the document frame is composed of trapezoidal vertices, the document frame comparison unit 3105 compares the vertex coordinates of the re-detected document frame and the recommended document frame.

  Next, the document frame comparison unit 3105 determines whether the re-detected document frame is larger than the recommended document frame based on the result of the comparison process in step S1010 (step S1011). If the re-detected document frame is not larger than the recommended document frame, the process proceeds to step S1009. As shown in FIG. 13, if the re-detected document frame is larger than the recommended document frame, the process proceeds to step 1012.

  Next, the CPU 301 acquires, as a main image, an image used in the document frame detection process in step S1009, that is, an image captured by the photographer with reference to the recommended document frame (step S1012). With the photographer referring to the recommended document frame and pointing the portable terminal 101 toward the document, the CPU 301 may switch the imaging device 203 to the high image quality mode and re-acquire the image. The high image quality mode is an operation mode in which the optical conditions of the image pickup apparatus and the processing at the time of digital conversion are switched to suppress noise and take a bright image.

  Next, the distortion correction processing unit 3106 corrects the distortion of the main image (step S1013), and the process ends. For example, the distortion correction processing unit 3106 corrects the distortion of the main image using projective transformation, but a correction method other than the projective transformation may be used. The corrected main image is printed by the printer 104 via the wireless router 102.

  According to the information processing apparatus of the first embodiment, it is possible to prevent characters included in an original photographed with a mobile terminal from being printed without being resolved.

(Example 2)
The information processing apparatus according to the first embodiment is configured to create a recommended document frame based on a document frame obtained by capturing a moving image frame. The information processing apparatus according to the second embodiment uses a document frame that is detected based on an image file that has already been shot and stored. In addition, the whole structure of Example 2, the structure of a portable terminal, and the functional block of a portable terminal are the same. Therefore, only differences from the first embodiment will be described.

FIG. 14 is a flowchart illustrating an example of the overall operation process of the mobile terminal according to the second embodiment.
First, the CPU 301 acquires an image from an image file of a document that has been captured in advance and stored in an HDD (not shown) or the like. For example, the CPU 301 displays a list of images (photos) included in the image file on the touch panel 201 and acquires an image selected by the photographer (step S5001). The CPU 301 reads the acquired image into the RAM 302.

  Next, the document frame determination unit 3101 detects a document frame from the image acquired in step S5001 and read into the RAM 302 (step S5002). The document frame detection method is the same as in the first embodiment.

  Steps S5003 to S5008 are the same as steps S1003 to S1008 in FIG. The photographer refers to the displayed recommended document frame (S5008) and points the portable terminal 101 toward the document, so that the captured image of the document is reacquired.

  In the second embodiment, the CPU 301 displays guidance for selecting whether or not to perform image distortion correction on the touch panel 201 (step S5009).

  FIG. 15 is a display example of a new document frame detected from a photographed image of a re-acquired document. As shown in the screen of FIG. 15, the CPU 301 prompts the photographer to select whether or not to perform image distortion correction by displaying a pop-up.

  The CPU 301 determines whether to perform distortion correction according to the user's selection (step S5010). If no distortion correction is performed, the process ends. In this case, printing on the printer 104 is also stopped. When performing distortion correction, the distortion correction processing unit 3106 performs distortion correction. The corrected image is printed by the printer 104 via the wireless router 102.

  According to the information processing apparatus of the second embodiment, it is possible to prevent characters included in a document image included in a file stored in advance from being printed without being resolved.

(Other examples)
The present invention can also be realized by executing the following processing. Software (computer program) that realizes the functions of the above-described embodiments is supplied to the system or apparatus via a network or various storage media, and the computer of the system or apparatus (or CPU, MPU, etc.) reads and executes the program. It is processing. In this case, the program and the storage medium storing the program constitute the present invention.

101 portable terminal 104 printer 105 server

Claims (12)

Detection means for detecting a document frame from a captured image;
A specifying means for specifying a character block from an area surrounded by the document frame;
Determining means for determining whether the character included in the character block can be resolved;
Generating means for generating a recommended document frame for enabling resolution of the character when it is determined that the character cannot be resolved;
An information processing apparatus comprising: display means for displaying the recommended document frame. The determination means includes
Dividing the character block into rectangular regions;
Calculating the number of character pixels in the rectangular area;
Determining whether the calculated number of character pixels is equal to or greater than a threshold;
The information processing apparatus according to claim 1, wherein when it is determined that the number of character pixels is not equal to or greater than a threshold value, it is determined that the character included in the character block cannot be resolved. When the character included in the character block is written horizontally, the determination means calculates the number of pixels in the height direction of the character corresponding to the rectangular area of the character block as the character pixel number, and includes the character block. The information processing apparatus according to claim 2, wherein when the character to be written is vertically written, the number of pixels in the width direction of the character corresponding to the rectangular area of the character block is calculated as the number of character pixels. The generating means converts the document frame so as to generate the recommended document frame so that the number of character pixels in all rectangular areas of the character block including the character determined to be unresolvable is equal to or greater than the threshold value. The information processing apparatus according to claim 2, wherein the information processing apparatus is an information processing apparatus. The generating unit calculates a ratio between the number of character pixels in a rectangular area determined not to be equal to or greater than the threshold and the threshold, and converts the document frame based on the calculated ratio to generate the recommended document frame. The information processing apparatus according to claim 2, wherein the information processing apparatus is an information processing apparatus. The generating means converts the document frame so that a region in the vicinity of a rectangular region in which the number of character pixels is determined not to be greater than or equal to the threshold among the regions surrounded by the document frame is converted to the recommended document. The information processing apparatus according to claim 2, wherein a frame is generated. The generation unit generates the recommended document frame by changing a position of a vertex in the vicinity of a rectangular area in which the number of character pixels is determined not to be greater than or equal to the threshold value among vertices of the document frame. The information processing apparatus according to any one of claims 2 to 6. The display unit highlights a character block including a character determined to be unresolvable in an area surrounded by the recommended document frame to be displayed. The information processing apparatus described in 1. The detection means includes
Detecting the document frame for a captured image for confirming the image,
A new document frame is detected from a captured image re-acquired after the recommended document frame is displayed;
9. The information processing according to claim 1, further comprising photographing means for photographing a main image that is an image to be recorded when the new document frame is larger than the recommended document frame. apparatus. The information processing apparatus according to claim 1, wherein the detection unit detects the document frame from a captured image included in a file stored in advance. A process of detecting a document frame from a photographed image;
Identifying a character block from an area surrounded by the document frame;
Determining whether a character included in the character block can be resolved;
When it is determined that the character cannot be resolved, generating a recommended document frame for resolving the character;
And a step of displaying the recommended manuscript frame.   A computer program for causing a computer to execute the control method of the information processing apparatus according to claim 11.

Images