JP2014123881A - Information processing device, information processing method, and computer program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To perform image quality determination which is less affected by image quality outside a document area of an image.SOLUTION: A document area is specified from an image, it is determined whether an image in the document area is suitable for printing on the basis of a feature of the image in the specified document area and print setting information, and when it is not determined that the image in the document area is suitable for printing, information indicating that the image in the document area is not suitable for printing is displayed.

Description

  The present invention relates to a technique for controlling shooting processing of an information processing apparatus having a shooting function.

  In recent years, mobile terminals having advanced information processing functions such as smartphones and tablet PCs have become widespread. These portable terminals include a camera and have a photographing function (camera function). It has been practiced to take a paper medium document using such a camera function of a portable terminal and store it as image data on the memory of the portable terminal. Along with this, there have been increased opportunities to use a combination of a portable terminal and a printer for copying an original, such as sending photographed image data to a printer for printing. In this way, even if the user does not have an MFP (Multi Function Peripheral) equipped with a scanner, if there is a portable terminal and an SFP (Single Function Peripheral) having a printing function, the document can be copied. It has become to.

  Unlike original document scanning in MFP, shooting a document using the camera function of a mobile terminal is different from the original's distance, tilt, focal length, exposure, shutter speed, camera shake, etc. Due to various factors, the image quality may not be stable. If the image quality is not satisfactory for the user, it is necessary to redo the shooting. It is troublesome for the user to determine whether or not to retake the image every time the image is taken, and it is difficult for the user to make a judgment if the image quality determination requires knowledge.

  In order to make it easy for the user to determine the image quality, Patent Document 1 uses a digital camera that performs notification for determining the image quality of a captured image using the amount of camera shake or the character size in the image and prompting the user to retake the image. Disclose. A user who is taking a picture using this digital camera determines whether to take a picture again according to the notification.

Japanese Patent No. 4366484

  Patent Document 1 discloses a technique for determining the image quality of the entire photographed image and prompting the user as to whether or not to recapture. For example, in the technique disclosed in Patent Document 1, when an object (such as a finger) that is accidentally reflected outside the original area of the photographed image is photographed out of focus, it is determined that the image quality is not good and the image is captured again. Is prompted by the user.

  However, a user who captures a document in an attempt to copy the document places importance on the image quality of the document area, not the entire captured image. For this reason, even if the image quality of the document area is desirable, if the image quality outside the document area is not good, if the user is prompted to retake it, the user may feel complicated.

  The present invention has been made in view of the above circumstances, and an object of the present invention is to make an image quality determination that is not easily affected by the image quality outside the document area of the image.

  An information processing apparatus according to the present invention includes an imaging unit that acquires an image by imaging a document, a setting unit that sets print setting information, and a specifying unit that specifies a document region from the image acquired by the imaging unit. Determining means for determining whether the image in the specified document area is suitable for printing based on the characteristics of the image in the specified document area and the set print setting information; and Display means for displaying information indicating that the image in the document area is not suitable for printing when the means does not determine that the image in the document area is suitable for printing. .

  Since the image quality determination that is not easily affected by the image quality outside the document area of the image is performed, the image quality determination in the document area can be improved.

System configuration diagram as one embodiment of the present invention Example of appearance of portable terminal 102 Example of configuration diagram of portable terminal 102 Example of configuration diagram of server 121 Example of configuration diagram of printer 103 Sequence Diagram Flow chart showing overall processing Diagram showing the document and document area The flowchart which shows the detailed process of the imaging | photography process of the original document of step S703 in Example 1. FIG. The figure which shows the example of the touch panel display at the time of original photography The figure which shows the example of the shape determination of a document area The figure which shows the example of the image quality judgment of a manuscript area The figure which shows the example of the image quality judgment in consideration of the original mode of the original area The figure which shows the example of the touch panel which prompts the user for the re-taking process The flowchart which shows the detailed process of the imaging | photography process of the original document of step S703 in Example 2. FIG. The flowchart which shows the detailed process of the imaging | photography process of the original document of step S703 in Example 3. FIG.

Example 1
Hereinafter, embodiments for carrying out the present invention will be described with reference to the drawings.

  FIG. 1 shows the overall configuration of a system used in this embodiment. A printer (printing apparatus) 103 and a wireless router 102 are connected to the LAN 110. The LAN 110 is connected to the Internet 120 and is also connected to the server 121 that provides the service via the Internet 120. The portable terminal 101, the printer 103, and the server 121 are connected to each other via the wireless router 102 and the LAN 110, and transmit / receive image data and various types of information. Here, the wireless router 102 and the printer 103 are connected to the LAN 110, but this is not a limitation. The wireless router 102 and the printer 103 may be connected to the server 121. Note that the portable terminal 101 functions as an information processing apparatus in the following embodiments.

<Configuration of mobile terminal>
FIG. 2 shows the appearance of the mobile terminal 101 used in this embodiment. There are various types of mobile terminals 101, but FIG. 2A shows the surface of the mobile terminal 101. A touch panel display 201 and operation buttons 202 are provided on the surface. FIG. 2B is the back surface of the mobile terminal 101. A camera 203 is disposed on the back surface. Note that this embodiment can be used in any terminal device having a camera function. In other words, it can be a smartphone or mobile phone with a camera function, or a digital camera with a communication function. The camera 203 is equipped with an autofocus mechanism (not shown) that automatically detects the in-focus state based on the contrast ratio, infrared transmission / reception timing, etc., and this measures the focal length and subject distance. can do. The autofocus mechanism used in this embodiment is an example of a method for measuring the focal length and the subject distance, and other methods that can measure the focal length and the subject distance may be used.

  FIG. 3 shows an internal configuration of the mobile terminal 101. However, this configuration diagram is an example of a configuration for carrying out the present embodiment. In FIG. 3, a CPU 301, a RAM 302, and a ROM 303 send and receive programs and data via a data bus 311. In addition, a storage unit 304, a data transmission / reception unit 305, an imaging unit 306, a display unit 307, an operation unit 308, an image processing unit 309, and a motion sensor 310 are connected to the data bus 311, and the CPU 301, RAM 302, and ROM 303 are also combined. They exchange programs and data with each other.

  The storage unit 304 is a flash memory, and stores image data and various programs.

  The data transmission / reception unit 305 includes a wireless LAN controller, and realizes data transmission / reception with the printer 103 and the server 121 via the wireless router 102.

  The imaging unit 306 is the above-described camera 203 and captures a photographed image by photographing a document. The acquired captured image data is transmitted to each unit as will be described later.

  The display unit 307 is a display that constitutes the touch panel display 201 described above, and displays various information such as a live view display when shooting a document using the camera function, and a notification based on the result of image quality determination of the present embodiment. Is displayed.

  The operation unit 308 is a touch panel and operation buttons 202 that constitute the touch panel display 201 described above, and receives operation from the user and transmits information on the operation to each unit.

  The motion sensor 310 includes a three-axis acceleration sensor, an electronic compass, and a three-axis angular velocity sensor, and can detect the attitude and movement of the mobile terminal 101 by using a known technique.

  The CPU 301 executes programs stored in the ROM 303 and the storage unit 304 to control the components in the portable terminal 101.

<Server configuration>
FIG. 4 is a configuration example of the server 121. The CPU 401 reads a control program stored in the ROM 403 and executes various control processes. The RAM 402 is used as a temporary storage area such as a main memory or work area for the CPU 401. A network I / F unit 404 connects the server 121 to the Internet 120 and transmits / receives various information to / from other devices. The HDD 405 stores image data and various programs.

  In FIG. 4, image data from the mobile terminal 101 received via the network I / F unit 404 is transmitted and received by the CPU 401, RAM 402, and ROM 403 via the data bus 410. When the CPU 401 executes the image processing program stored in the ROM 403 or the HDD 405, image processing for the image data is realized.

<Printer configuration>
FIG. 5 is a diagram illustrating the configuration of the printer 103 used in this embodiment. The printer 103 includes a controller unit 500, an operation unit 509, an external memory 510, and a printer unit 511. In the controller unit 500, the CPU 505 outputs an image signal to the printer unit 511 via the printer I / F unit 504 connected to the system bus 508 based on a control program or the like. The control program is stored in the ROM 507 and the external memory 510. The ROM 507 stores a control program for the CPU 505 and the like. The CPU 505 can perform communication processing with the mobile terminal 101 and the server 121 via the network I / F unit 501. Thus, information in the printer, shooting information on the mobile terminal, image processing information on the server 121, and the like can be transmitted to and received from the mobile terminal 101 and the server 121. A RAM 506 functions as a main memory, work area, and the like for the CPU 505.

  The input image data is sent to the controller unit 500 via the network I / F unit 501. Subsequently, the PDL (page description language) data processing unit 502 performs PDL interpretation and rendering, the image processing unit 503 performs image processing for print output, and the printer unit 511 prints the processed image data.

<Detailed description of this embodiment using flowchart>
FIG. 7 illustrates a process of extracting a document area in a photographed image by the mobile terminal 101, the server 121, and the printer 103 according to the present exemplary embodiment, and printing the image by scaling the extracted document area image to the size of the output document. It is a flowchart to do. Note that the processing in the portable terminal 101 is realized by the CPU 301 loading a processing program stored in the storage unit 304 into the RAM 302 and executing it. Similarly, the processing in the server 121 is realized by the CPU 401 loading a processing program stored in the HDD 405 into the RAM 402 and executing it. Further, the processing in the printer 103 is realized by the CPU 505 loading the processing program stored in the external memory 510 to the 506 and executing it.

  In step S <b> 701, when an instruction to start shooting is input from the user via the operation unit 308, the mobile terminal 101 is connected to the LAN 110 via the data transmission / reception unit 305, and a printer that prints shot image data is connected to the LAN 110. Search from. For example, when an IP address of a desired printer is input from the user via the operation unit 308, a method of searching for and finding a printer corresponding to the IP address can be considered. Further, for example, a search is made for available printers in the LAN 110, information on the plurality of found printers found is displayed on the display unit 307, and one of the plurality of printer information displayed on the operation unit 308 is displayed to the user. A method of finding a printer by selecting one is also conceivable.

  In step S702, when the printer 103 is found, the printer 103 notifies the portable terminal 101 of printer information. The printer information here means the paper size and paper type that can be used by the printer 103, the output resolution of the printer 103, whether the printer 103 can perform duplex printing, whether the printer 103 can perform Nin1 printing, or color printing by the printer 103. Information on the printing capability of the printer 103 is included. Based on the printer information, the user sets information (print setting information) related to settings for printing a captured image with the printer 103 via the operation unit 308. The set print setting information is stored in the RAM 302. By setting the print setting information based on the printer information in this way, it is possible to perform image print settings suitable for printing with the printer 103. The print setting information is information on the size and type of paper used for printing, the output resolution at the time of printing, the necessity of double-sided printing, the necessity of Nin1 printing, the necessity of color printing, and the like. In step S <b> 702, the CPU 301 of the mobile terminal 101 functions as a unit that sets print setting information in the RAM 302.

  In step S <b> 703, when the document is photographed by the imaging unit 306, the mobile terminal 101 extracts the vertex of the origin from the document area in the photographed image data. This is illustrated in FIG. FIG. 8 shows a captured image 801 and a document area 802. A document area 802 is a rectangular area surrounded by a line segment connecting vertices 803, 804, 805, and 806 of the extracted document area. The vertices 803 to 806 of the rectangular document area are extracted by a known method such as Douglas & Poker vertex extraction algorithm.

  Next, the CPU 301 determines whether the document area 802 in the photographed captured image 801 is an image suitable for printing. If the result of the determination is that it is not suitable for printing, the user is given a reason for taking it back and a countermeasure, and the user is prompted to take a taken image again. The detailed flow of step S703 described above will be described later with reference to FIG.

  In step S <b> 704, the mobile terminal 101 determines the output image size set by the user via the operation unit 308, the print image size derived from the print settings related to the magnification such as Nin1 printing, and the size of the document area in the captured image. Based on the above, the scaling parameter is calculated. Here, the scaling parameter to be calculated is a scaling parameter such that the document area in the photographed image is scaled to the output image size. For example, when the print setting is A4 size paper size and 1 in 1 printing, the output image size is A4 size, and the scaling parameter is a scaling factor such that the image in the document area is scaled to the output image size A4 size. It becomes a parameter. Also, for example, when the print setting is A4 paper size and 2 in 1 printing, the output image size is A5 size, and the scaling parameter is a scaling factor such that the image in the document area is scaled to the output image size A5 size. It becomes a parameter.

  It should be noted that the paper size may be switched from paper type series such as whether to use A series paper, B series paper, or inch series paper, based on the aspect ratio of the document area. Good.

  The scaling parameter is a projective transformation matrix, taking into account the case of distortion to a trapezoid. The projective transformation matrix can be calculated by a known method from the vertex information (vertices 803, 804, 805, 806) of the four points of the document area in the captured image and the coordinate information of the four corners of the output image. If priority is given to processing speed, an affine transformation matrix or a simple scaling factor may be calculated as a scaling parameter.

  In step S705, the mobile terminal 101 compresses the captured image data in the JPEG format or the like.

  In step S <b> 706, the mobile terminal 101 transmits print settings and scaling parameters to the server 121 via the data transmission / reception unit 305 together with the compressed captured image data.

  Note that in steps S705 and S706 described above, compression processing and transmission processing are performed on the captured image data. However, in step S <b> 705, the CPU 301 may extract image data of the document area in the captured image data, and perform compression processing and transmission processing using the extracted image data of the document area as captured image data.

  The subsequent processes in steps S707 to S716 are performed by the server 121.

  In step S <b> 707, the server 121 receives captured image data, print settings, and scaling parameters transmitted from the mobile terminal 101. The processing by the server 121 after the next step is performed based on print settings and scaling parameters.

  In step S708, the server 121 expands the compressed captured image data.

  In step S709, the server 121 performs a scaling process only on the document area of the captured image data using a scaling parameter such as the projective transformation matrix calculated in step S704. Since scaling processing is performed only on the document area, the captured image data after scaling is image data in the document area scaled to the output image size. That is, only the document area is scaled to the output image size. In other words, the server 121 extracts a document area from the photographed image, and scales the document area for printing. The scaling process may use other known scaling techniques such as affine transformation, cubic scaling, and linear scaling. In the subsequent processing, the captured image data is image data in the document area that has been scaled to the output image size.

  In step S <b> 710, the server 121 performs density unevenness correction on the captured image data, and corrects color unevenness due to the light source and shadow during document shooting. First, filter processing is performed so as to remove noise at the time of shooting, and then tone correction is performed so that white on paper can be reproduced by skipping colors on the background.

  In step S <b> 711, the server 121 performs image area determination, divides the captured image data into a character or line portion, and a portion that does not, and adds an attribute flag to each pixel.

  In step S <b> 712, the server 121 performs a filtering process on the pixels determined to have the character or line attribute, and corrects the character or line to be sharp.

  In step S <b> 713, the server 121 performs character monochromatic processing, reproduces pixels determined to be characters in a single color, and sharpens the characters. For example, the RGB values of pixels that reproduce a character are averaged, and the average value is applied as the character color.

  In step S714, the server 121 converts the captured image data from a camera-specific color space to a common RGB color space. Here, the conversion of the color space is performed by converting from the camera color space of the portable terminal 101 to a colorimetric common RGB color space such as sRGB by a 3 × 3 matrix operation defined in advance.

  In step S715, the server 121 compresses the captured image data (corrected image data) corrected in the previous steps S707 to S714 in the JPEG format or the like.

  In step S716, the server 121 transmits print settings to the printer 103 together with the corrected image data that has been compressed.

  The subsequent steps S717 to S723 are performed by the printer 103.

  In step S <b> 717, the printer 103 receives the corrected image data and print settings from the server 121. Thereafter, the printer 103 performs processing until the corrected image data is printed. The processing in steps S718 to S721 is performed by the image processing unit 503, and the received print setting is used as the processing parameter.

  In step S718, the printer 103 expands the corrected captured image data that has been compressed.

  In step S719, the printer 103 converts the expanded image data from the common RGB color space to the printer color space composed of CMYK color components suitable for the printer by interpolation using a color conversion LUT (Look Up Table). Perform conversion. The color conversion LUT here is a three-dimensional LUT obtained by dividing each of the three RGB components at an appropriate grid point interval, and each LUT entry has an 8-bit bit precision corresponding to the grid point of the LUT. The CMYK value of By combining a three-dimensional LUT and a known interpolation calculation, it is converted into image data composed of CMYK values.

  In step S720, the printer 103 performs gradation correction that corrects the density characteristics to match the gradation target using a one-dimensional LUT. When the gradation target is changed between the character part and the other part, the gradation target is changed here to perform correction.

  In step S721, the printer 103 converts the tone-corrected image data into image data having a pseudo halftone expression of 1 bit for each color of CMYK using a dither matrix. Here, the screen processing compares the numerical value on the dither matrix stored in the RAM 506 with the input image data, and outputs 1 if the numerical value of the input image data is large, and outputs 0 if the numerical value of the input image data is small. Process. In this embodiment, screen processing with 1-bit output is performed for ease of explanation, but the number of output bits is not limited to 1 bit. In addition, a plurality of dither matrices can be switched and used in accordance with an instruction from the CPU 505.

In step S722, the printer 103 prints the processed captured image data. In this way, the document photographed by the mobile terminal 101 is copied.
In step S723, the printer 103 sends a print end notification to the portable terminal 101 in order to notify the user that printing has ended.

  In step S724, the mobile terminal 101 receives the print end notification from the printer 103, displays the print processing notification on the display unit 307, and ends the process.

<Step S703 Detailed Flow of Shooting Process>
Next, document shooting in step S703 will be described with reference to FIG. This document photographing process is performed by the CPU 301 of the portable terminal 101, and the detailed operation of the CPU 301 will be described using the software block diagram of FIG. The software block diagram of FIG. 6 is a block diagram showing processing realized by the CPU 301 executing the processing program in the storage unit 304, and the CPU 301 functions as each processing unit shown in each block.

  In step S901, the CPU 301 operates as the imaging processing unit 601 in the software block diagram of FIG.

  First, the CPU 301 displays an image acquired by the camera 203 in real time on the touch panel display 201 as a preview image. The preview image is periodically updated by the CPU 301 and changes when the user moves the mobile terminal 101. This enables live view shooting.

  The CPU 301 determines whether there is a document in the preview image and, if there is a document, whether the preview image satisfies the shooting conditions. Whether or not there is a document is determined by detecting and specifying a document region, and the method is the same as the method in step S902 described later. The preview image satisfies the shooting conditions. The document area in the preview image has a certain area or more, and the deviation of the inner angle of the vertex of the document area from 90 degrees is within a certain range. The amount of camera shake is within a predetermined range. The respective threshold values used for determining whether or not each photographing condition is satisfied are not fixed, and the print setting information (output image determined from the paper size and Nin1 setting of the print setting) stored in the RAM 302 in step S702. Variable according to the size and output resolution). This is because the image quality suitable for printing differs depending on the print setting (output image size, output resolution, etc.).

  If the preview image satisfies the shooting conditions, the operation button 202 is activated to prompt shooting. Then, the CPU 301 waits for the user to press the operation button 202. FIG. 10A shows an example of this, and there is a document area in the preview image, the area of the document area is equal to or greater than a predetermined threshold value, and the amount of camera shake is equal to or less than the predetermined threshold value. And prompts the user to shoot.

  When the preview image does not satisfy the shooting condition, the user is prompted to shoot so that the preview image satisfies the shooting condition by displaying which shooting condition the preview image does not satisfy. FIG. 10B is an example of this, and there is a document, but since the document area is smaller than a predetermined threshold value, the photographing condition is not satisfied and photographing “impossible” is displayed. Then, the fact that the area of the document area is smaller than a predetermined threshold is emphasized by underlining the character string “NG”.

  When the user presses the operation button 202 after the shooting conditions are satisfied, the CPU 301 issues a shooting instruction to the imaging unit 306. It should be noted that the CPU 301 may automatically issue a shooting instruction even if the operation button 202 is not pressed when a condition for obtaining a document is satisfied.

  When a shooting instruction is issued, the camera 203 serving as the imaging unit 306 performs autofocus using an autofocus mechanism, and performs shooting (imaging) after adjusting the focal length. For this reason, there is a time lag between when a shooting instruction is issued and when actual shooting is performed. Therefore, the captured image acquired by the imaging unit 306 may be captured in a situation different from the situation in which the preview image satisfies the imaging conditions, for example, the situation in which the preview image does not satisfy the imaging conditions. . In addition, since the actual image quality such as the degree of blur is not determined until after shooting, there is a possibility that the image in the document area does not have an image quality suitable for printing. For this reason, the quality of the captured image, particularly the image of the original area, is evaluated in the subsequent steps.

  When shooting is completed, the CPU 301 transfers the shot image from the imaging unit 306 to the RAM 302, and the process proceeds to step S902. The data may be stored in the storage unit 304 and then transferred to the RAM 302.

  In step S902, the CPU 301 operates as the document area detection unit 602 in the software block diagram of FIG. That is, the CPU 301 analyzes the captured image in the RAM 302 and performs a process for detecting a document area in the captured image.

  In the preview image, the document area should have been detected and specified, but a change such as the movement of the mobile terminal 101 occurs after the shooting button is pressed until the actual image is shot. Therefore, the detection process of the document area is performed again. In this embodiment, it is assumed that the document area is a rectangular area, and the outermost rectangular area in the image is detected and specified as the document area. Specifically, first, in order to remove the influence of noise, the captured image is reduced to a resolution sufficient to detect the document area. The reduced photographed image is used only for detecting the document area. The captured images used in the subsequent steps are original-sized captured images.

  Next, edge detection is performed in order to detect an edge between the document and the table on which the document is placed. For the edge detection, for example, a known method such as Canny edge detection may be used. The edge detection threshold value may be a fixed threshold value or may be automatically calculated from the brightness of the image.

  If the edge of the document area can be normally detected, a closed area that is not included in another closed area among one or more closed areas included in the image is a document area candidate. The document area candidate may include a desk pattern or the like outside the document area. However, if the closed area is a document area, the closed area has an area of a certain size or more and should be the largest closed area in the image. Therefore, the CPU 301 detects the largest closed region in the image as a final document region candidate that has a certain area or more among one or more document region candidates. If the largest closed region in the image does not have a certain area or more, it is assumed that a region other than the document region has been detected, and the CPU 301 determines that the document region does not exist in the image. When the closed region of the document region candidate is detected, the closed region is approximated with a line segment, and the vertex is calculated. For the approximation, for example, a known method such as the above-described Douglas & Poker vertex extraction algorithm may be used.

  If the number of vertices is four as a result of the line segment approximation, the CPU 301 determines that four corners of the document are photographed and determines that the document area is detected, and is surrounded by the vertices. It is determined that the area is a document area. This is shown in FIG. 8 described above, and the detected vertices are vertices 803, 804, 805, and 806. Note that the vertex coordinates are positions in the coordinate system of the original photographed image, not the reduced photographed image, for use in subsequent steps. If the number of vertices is not four, it is determined that the document area does not exist in the image. Note that in the edge detection, if it is also considered that all edges of the document cannot be detected and are interrupted, the rectangular region of the document may be detected using Hough transform or the like.

  In step S903, the CPU 301 operates as the document area shape determination unit 603 in the software block diagram of FIG. 6, extracts a geometric feature of the image of the document area, and makes the image of the document area suitable for printing based on the feature. Determine whether or not. Here, in particular, the CPU 301 determines whether the document area has a shape suitable for printing. The determination is made based on geometric features such as the area of the document area and the internal angle. The determination is performed using a threshold value that is predetermined for the print setting. Note that the threshold value used for determining whether or not the shooting condition is satisfied is changed according to the print setting stored in the RAM 302 in step S702. For example, it is variable according to the output image size and output resolution determined from the paper size of the print setting and the Nin1 setting. This threshold value may provide a mechanism that allows the user to change on the touch panel display 201.

  An example of this determination process will be described with reference to FIG.

  A broken line frame represented by (a) in FIG. 11 is a captured image, and a solid line frame represented by (b) in FIG. 11 is a document area. 11A and 11B will be described with reference to the photographed image expressed in FIG. 11A and the document area expressed in FIG. 11B.

  11C, the document area does not exist in the photographed image, FIG. 11D shows the document area protruding from the photographed image, and FIG. 11E shows a document area other than the document such as a finger. Foreign objects are reflected. As shown in (c), (d), and (e) of FIG. 11, when the entire document area is not normally captured in the captured image, the entire document cannot be reproduced, and the captured image is suitable for printing. Not an image. In such a case, since the document area does not become a rectangular area, in the present invention, in step S902, the original area is not detected.

  Therefore, when the document area is not detected, the CPU 301 determines that the situation is as shown in FIGS. 11C, 11D, and 11E, and the document area does not have a shape suitable for printing. And judge.

  FIG. 11F shows a case where the area of the document area is smaller than a predetermined threshold. If the area of the document area is smaller than the predetermined threshold value, the document area image has a low resolution, so it is determined that the document area does not have a shape suitable for printing. For example, it is assumed that the paper size of the print setting is A4 size (210 mm × 297 mm), the Nin1 setting is a setting for printing with a layout of 1 in 1, and the minimum resolving power calculated from the output resolution is 200 DPI. In that case, the output image preferably has an area of about 3.87 million pixels (210 mm × 297 mm / 25.4 mm ^ 2 × 200 DPI ^ 2) or more. When the area of the rectangular area is smaller than this, the resolution of the document area is less than 200 DPI. Actually, when the document area is trapezoidal, the degree of image resolution differs between the back and the front. For example, the front image has a high resolution, but the back image has a low resolution. Therefore, when a resolution greater than the resolution obtained from the print settings (for example, 200 DPI in the above example) is obtained, it is simply obtained from the print settings in accordance with an allowable trapezoidal shape (allowable interior angle described later). A value larger than a threshold value (for example, 3.87 million pixels in the above example) needs to be set as the threshold value.

  FIG. 11G shows a case where the absolute value (angle difference value) of the difference between the internal angle of the document area and 90 degrees is larger than a predetermined threshold value. The fact that the angle of the inner angle is not 90 degrees means that the document and the camera 203 are not almost facing each other at the time of shooting, and the document is shot from an oblique direction. Therefore, the resolving power varies depending on the location of the document area in the photographed image. As the inner angle moves away from 90 degrees, the minimum resolving power in the document area becomes lower when the document area has a constant area. That is, it is necessary to increase the above-described area threshold value of the document area for obtaining a minimum resolution of a certain level or more as the inner angle moves away from 90 degrees.

  When the area threshold value is increased, the difference between the area threshold value and the photographed image area is reduced, so that the range of allowable distance between the document and the camera 203 is narrowed and it is difficult for the user to photograph. Also, the theoretical area threshold may exceed the captured image area in the first place.

  On the other hand, if the allowable angle difference value is made strict, it becomes necessary for the document and the camera 203 to face each other correctly at the time of shooting, which makes it difficult for the user to shoot. Therefore, it is necessary to determine the threshold value of the angle difference value in consideration of the balance with the photographed image area. For a document area having an angle difference value larger than the threshold value, the CPU 301 does not have a shape suitable for printing. Make a decision. The threshold value may be set such that the threshold value regarding the angle difference value increases as the area of the document area increases.

  FIG. 11H shows a case where the entire document area is in the captured image, the area of the document area is equal to or larger than a certain threshold value, and the angle difference value is equal to or smaller than the certain threshold value. In such a case, since the entire document area is photographed and the document area is photographed with a resolution higher than a certain level, the CPU 301 determines that the document area has a shape suitable for printing. .

  (I) in FIG. 11 is the same situation as (h) in FIG. 11 except that a hand is reflected outside the document area. When it is desired to print an image of the document area, the user places importance on the document area, and what appears outside the document area is relatively unimportant. Therefore, since it is only necessary to determine whether the captured image is suitable for printing using the shape of the document area, (i) in FIG. 11 also determines that the document area has a shape suitable for printing. . That is, in the captured image of FIG. 11 (i), since the hand is reflected in the captured image, it is considered that the shooting has failed when the entire captured image is regarded as important, and re-shooting is performed in the prior art. It is an image that will be prompted. However, for the user who wants to place importance on the image in the document area, even the image as shown in FIG. 11 (i) is sufficient as the quality of the document area used for printing. It is determined that the image is suitable for printing. As a result, it is possible to avoid requiring the user to perform an unnecessary retake.

  If it is determined by the determination described above that the document area does not have a shape suitable for printing, the process proceeds to step S906. If it is determined that the document area has a suitable shape, step S904 is performed. Proceed to Note that other document area geometric characteristics such as the ratio of the lengths of sides other than the area and the inner angle may be used. As described above, in this step S903, by using the shape of the document area, the first-stage re-shooting determination is performed, so that the average processing time is longer than when all the determinations are made and then the re-shooting determination is performed. It becomes possible to shorten. Note that the second-stage re-take determination is in step S905 described later.

  In step S904, the CPU 301 operates as the document area image quality / document mode recognition unit 604 in the software block diagram of FIG. That is, the CPU 301 recognizes the image quality / document mode of the document area in order to determine whether the image quality of the document area itself is suitable for printing.

  The image quality of the document area includes the brightness of the document area, the contrast value of the document area, the light amount unevenness of the document area, the degree of blur of the document area, and the like.

  As for the brightness of the document area, for example, the brightness value of the peak on the high brightness side of the histogram in the document area is set as the brightness of the background of the document, and this is used as the brightness value of the document.

  As for the contrast ratio of the document area, for example, the ratio of the peak luminance value on the high luminance side and the low luminance side of the histogram in the document area is set as the contrast value of the document area.

  For example, the original light amount unevenness is obtained by dividing the original region, detecting the background of the divided region, obtaining the brightness of the background of each divided region, and setting the dispersion value as the light amount unevenness value.

  As for the degree of blur of the document area, for example, a line-like edge in the document area is found, and the rate of change of the edge is used as the blur value. The line-like edge is assumed to be an edge of a character, a frame of a photographic region, or an artifact in the photo, and is expected to be a strong edge when it is not blurred. For this reason, since it is considered that the line-shaped edge is a weak edge because of the blur, the reciprocal of the edge change rate (edge strength) can be used as the blur value. Any other index may be used as long as it is a known index representing image quality.

  The document mode of the document area is a document type such as whether the document is a text-based document, a photograph-based document, or a document in which characters and photos are mixed. The document type is determined using, for example, the characteristics of each sampled small area. As features, the strength of the edge, the amount of the edge, the continuity of the edge, the periodicity of the edge, the dispersion of pixel values in a small region, and the like can be considered.

  First, small areas are extracted from a plurality of document areas, and it is determined whether each area is a character area, a photographic area, or a ground area without any object. For example, if the average edge strength of pixels within a small area, for example, a pixel having a certain or larger edge strength in the small area is equal to or higher than the first threshold and the periodicity of the edges is equal to or lower than the second threshold, It is determined that the area. Here, the reason for paying attention only to pixels having a certain level of edge strength is to prevent the influence of the underlying region. Of the areas that are not determined to be character areas, an area in which the average of the edge strengths of pixels having a certain level of edge strength is equal to or greater than the third threshold is determined to be a photographic area. Here, the third threshold is a value smaller than the first threshold. If the average edge strength is less than the third threshold (that is, if there is almost no amount of edges), it is determined that the region is the background region.

  If “number of areas determined as character area / (number of areas determined as character area + number of areas determined as photograph area)” is close to 1, the document mode is the character mode. Is determined.

  If it is close to 0, it is determined that the mode is a photo mode.

  When it is close to 0.5, it is determined that the character / photo mode is a mixture of characters and photos.

  Then, the CPU 301 sets the determined mode to the document mode.

  In step S905, the CPU 301 operates as the document area image quality determination unit 605 in the software block diagram of FIG. 6, extracts the image quality characteristics of the image of the document area, and determines whether the image of the document area is suitable for printing based on the characteristics. Determine if. Here, in particular, the CPU 301 determines whether or not the document area in the photographed image has an image quality suitable for printing based on the recognition result in step S904. In this determination, the CPU 301 analyzes the image of the document area in the photographed image, obtains the brightness value, contrast value, light intensity unevenness value, and blur value in the document area, and compares each value with the corresponding threshold value. Thus, it is determined whether or not the image in the document area has an image quality suitable for printing. The determination is performed using a predetermined threshold value. Note that the threshold value changes according to the print setting information (color / monochrome setting, etc.) stored in the RAM 302 in step S702. Moreover, this threshold value may provide the user with a mechanism that can be changed on the touch panel display 201. The image quality of the document area may be determined in consideration of the document mode as described later with reference to FIG. In this case, image quality suitable for the document mode is determined by using a different threshold value depending on the document mode.

  An example of this determination process will be described with reference to FIG. Also in FIG. 12, as shown in FIG. 12 (a), the broken line frame represented by (a) in FIG. 12 is a captured image, and the area surrounded by the solid line represented by (a) in FIG. Is a document area. The same applies to (b) to (g) of FIG.

  FIG. 12A shows a bright portion of the entire photographed image, and the brightness value of the entire photographed image is not less than a predetermined threshold and is not appropriate for printing, but the document area itself is dark and the brightness of the document. The thickness value is smaller than a predetermined threshold value.

  In FIG. 12B, the entire captured image has a bright portion and a dark portion, and the contrast value of the entire captured image is equal to or greater than a predetermined threshold, but the contrast between the document regions is not strong, and the contrast of the document regions. The value is smaller than a predetermined threshold value.

  In FIG. 12C, the entire captured image has a portion with no unevenness in the amount of light, and the unevenness in the amount of light other than the original area of the captured image is less than a predetermined threshold value. The light amount unevenness value of the area is larger than a predetermined threshold value. Note that brightness, contrast value, and light amount unevenness have a great influence on color printing in consideration of color reproduction. Therefore, when the print setting is color, a larger threshold is set than in the case of monochrome.

  In FIG. 12D, the area outside the document area is not blurred, but the document area is blurred, and the blur value of the document area is larger than a predetermined threshold value. Note that blurring has a greater influence on image quality as the output image size determined from the paper size and Nin1 setting in the print settings is larger, and as the output resolution is higher. Therefore, a smaller threshold is set as the output image size is larger. Further, a smaller threshold is set as the output resolution is higher.

  In the cases shown in FIGS. 12A to 12D, it is determined that the image in the document area does not hold the image quality suitable for printing. This is because the image quality determination is performed only on the document area.

  (E) in FIG. 12 is dark except for the image area in the photographed image and has a low contrast, but the document area itself is bright and has a light / dark difference, and the brightness value and contrast value of the document area are equal to or greater than a predetermined threshold value. Other indexes representing image quality also satisfy predetermined conditions.

  In FIG. 12F, the light amount unevenness is large except for the document region in the photographed image, but the document region itself has no light amount unevenness, and the light amount unevenness value of the document region is equal to or less than a predetermined threshold value, indicating other image quality. Also satisfy a predetermined condition.

  In FIG. 12G, the area other than the document area in the photographed image is blurred, but the document area itself is not blurred, the blur value of the document area is equal to or less than a predetermined threshold, and other indexes indicating image quality are also predetermined. Satisfy the condition of

  In the cases as shown in FIGS. 12E to 12G, it is determined that the image in the document area has an image quality suitable for printing. This is because the image quality determination is performed only on the document area.

  As described with reference to FIGS. 12A to 12G, for example, by using the brightness value, contrast value, light intensity unevenness value, blur value, and the image quality of the document area as a determination criterion, The influence of the image quality of the captured image can be suppressed, and it can be accurately determined whether the captured image has an image quality suitable for printing. As a result, it is possible to prevent the user from forcing re-taking that is not necessary or missing an image that needs to be re-taken.

  Next, determination processing using the document mode recognized in S904 and the image quality of the document area will be described. In general, when characters are blurred, the readability is strongly affected. Therefore, the character region is less resistant to the blur than the photo region. Therefore, if a threshold value is set for each document mode, it is possible to make a reshoot determination in consideration of the properties of each document. That is, when there is a first threshold value and a second threshold value for the blur value, and the second threshold value is larger, the blur value in the character mode is set as the first threshold value, and the blur value in the photo mode is set. Is the second threshold. An example of this will be described with reference to FIG.

  Also in FIG. 13, as shown in FIG. 13A, the broken-line frame expressed in FIG. 13A is a photographed image, and the area surrounded by the solid line expressed in FIG. Is a document area. The same applies to (b) to (e) of FIG.

  FIG. 13A shows a case where the document mode of the document area is the character mode and the blur value of the document area is between the first threshold value and the second threshold value.

  FIG. 13B shows a case where the document mode of the document area is the photo mode and the blur value of the document area exceeds a second predetermined threshold value.

  FIG. 13C shows a case where the document mode of the document area is the character mode and the blur value of the document area is equal to or less than the first threshold value.

  In FIG. 13D, the document mode of the document area is the photo mode, and the blur value of the document area is between the first threshold value and the second threshold value, which is about the same as the image in FIG. 13A. This is the case when you are out of focus.

  (E) in FIG. 13 represents the relationship between the blur value and the threshold value of the image regions in (a) to (d) in FIG. As is clear from FIG. 13E, it is determined that the image quality suitable for printing is not maintained in FIGS. 13A and 13B, and FIGS. 13C and 13D are suitable for printing. It is determined that the image quality is maintained. In this way, when a threshold value for each document mode is prepared, even a document region having a similar blur value can be determined more flexibly by considering the document mode. As a result, it is possible to further reduce the possibility of forcing the user to perform re-shooting that is not necessary or missing an image that needs to be re-taken. As described above, the first to third blur value threshold sets may be set according to print settings such as the output image size and output resolution.

  Specifically, it is possible to reduce the possibility of requesting the user who took the photographic document to re-shoot even though the photographic document has a sufficient blur value. Further, it is possible to reduce the possibility that the user who has taken the text document does not require re-shooting even though the text document has an insufficient blur value.

  If it is determined by the determination described above that the document area does not have an image quality suitable for printing, the process proceeds to step S906. If it is determined that the original area has a suitable shape, this flow is performed. And the process proceeds to step S704.

  In step S906, the CPU 301 operates as the re-taking UI display processing unit 606 in the software block diagram of FIG. That is, the CPU 301 displays a screen that recommends re-shooting on the touch panel display 201. In addition to simply recommending reshooting, the reason why it is determined that printing is not suitable in step S903 or step S905 and the countermeasures are displayed. As a result, the user knows how to re-take the document, and the convenience at the time of re-taking is improved. Note that the displayed information is not limited to the above-described reason and countermeasure information, but may be information indicating that the image in the document area is not suitable for printing. It is possible to determine whether or not to re-shoot. Such an example will be described with reference to FIG.

  In FIG. 14A, in step S903, it is determined that the document area does not have a shape suitable for printing because the document area is not in a shape suitable for printing (for example, a rectangle), and the process proceeds to step S906. It is a display of the case. Since the shape of the document area is inappropriate, a message to that effect and a coping method to capture the image so that the frame of the document fits in the captured image are displayed.

  In FIG. 14B, in step S903, it is determined that the document area does not have a shape suitable for printing because the shooting size (area) of the document area is too small and the resolution is insufficient when printing is performed. This is a display when the process proceeds to S906. Since the area of the image in the document area is smaller than the threshold value, a message to that effect and a coping method that captures a large image of the document are displayed.

  FIG. 14C shows a display when it is determined in step S905 that the document area does not have an image quality suitable for printing because the brightness of the document area is insufficient, and the process proceeds to step S906. Since the original area is not brighter than the appropriate value, a message to that effect and a coping method for photographing the original in a bright place are displayed.

  In FIG. 14D, it is determined in step S905 that the document area is a character document, and the document area has exceeded the allowable blur amount as a character document, and therefore it is determined that the document area does not have an image quality suitable for printing. This is a display when the process proceeds to step S906. Since the document area has exceeded the allowable amount of blur as a text document, a message to that effect and a countermeasure method for shooting with attention to camera shake are displayed.

  Thus, by displaying the reason for re-shooting and the coping method, the user can perform re-shooting smoothly. In addition, since the coping method is known, it is possible to prevent re-photographing from being performed many times.

  Note that the displayed information is not limited to the above-described reason and countermeasure information, but may be information indicating that the image in the document area is not suitable for printing, and this information is displayed. In this case, the user can determine whether or not to retake the document.

  In step S907, the CPU 301 operates as the reshoot determination unit 607 in the software block diagram of FIG. That is, the CPU 301 determines whether or not the user has agreed to retake on the touch panel display 201. For example, as shown in FIGS. 14A to 14D, whether or not the user agrees is determined by providing a “reshoot” button and a “cancel” button and pressing the “retake” button. do. If the “Cancel” button is pressed, it is determined that the user has not agreed.

  If the user agrees, the captured image stored in the RAM 302 is deleted and invalidated, and the process proceeds to step S901 to perform re-taking. If not agreed, this flow is terminated and the process proceeds to step S704.

  The above description is a detailed flow of the document photographing process in step S703. By using the flow of the present invention, the determination of reshooting is performed from the viewpoint of only the document area actually used for printing, so the possibility of forcing the user to perform reshooting that is not necessary can be reduced. In addition, since the threshold is relaxed to avoid unnecessary re-taking, the possibility of missing an image that needs to be re-taken can be reduced. Furthermore, by displaying the reason for re-taking and the coping method, it is possible to reduce the possibility that the user will re-take the photo many times.

(Example 2)
In the first embodiment, the image quality of the print area is determined after determining whether the shape of the document area is suitable for printing. In this embodiment, a method for determining the degree of blur due to camera shake without performing detailed analysis of the image of the document area as in S905 of the first embodiment will be described. That is, the blur of the image in the document area is determined by a method having a smaller processing amount than the processing of S905. In the second embodiment, only a portion having a difference from the first embodiment will be described.

  FIG. 15 is a detailed flowchart of the document photographing process in step S703 of FIG. The processing of this flow is executed when the CPU 301 executes a processing program stored in the storage unit 304. Since the differences from the first embodiment are steps S1501 to S1503, only that portion will be described. Note that the processing in steps S903 to S907 in FIG. 15 is the same as the processing in steps S903 to S907 in FIG. 9 described in the first embodiment.

  The processing in step S1501 is basically the same as that in step S901. The difference is that autofocus is performed by the autofocus function mounted on the camera 203, the preview image after the focal length is adjusted is transferred to the RAM 302, and held together with the photographed image. .

  In step S1502, the CPU 301 operates as the document area detection unit 602 in the software block diagram of FIG. That is, the CPU 301 performs a document area detection process in the captured image as in step S <b> 902 and a document area detection process in the preview image held in the RAM 302.

  In step S1503, the CPU 301 operates as the blur determination unit 608 in the software block diagram of FIG. That is, the absolute value of the difference between the position and area of the document area in the preview image immediately after autofocus and the position and area of the document area in the photographed image is the predetermined threshold value. It is determined whether the following is true.

  First, when the document area is not detected in the preview image or the photographed image, it is determined that the document area is not suitable for printing, and the process proceeds to step S906. If the document area can be detected, the coordinate system of the vertex coordinates of the preview area and the document area in the captured image is matched. For example, if the preview image is 500 × 375 pixels and the photographed image is 4000 × 3000 pixels, the photographed image is eight times larger than the preview image, so the vertex coordinates of the document area of the preview image are eight times larger. Then, the coordinate system of the preview image and the photographed image matches.

  Next, the area AREA_P of the document area in the preview image and the area AREA_I of the document area in the photographed image are calculated. The difference between AREA_P and AREA_I means that the distance between the document and the camera (the distance in the direction of the optical axis of the camera) has changed, and the scaling has changed as compared to immediately after autofocus. means. This is a major factor that causes image blurring. Therefore, when the difference between AREA_P and AREA_I exceeds a predetermined threshold value, the document area is blurred due to camera shake, and the document area is suitable for printing. If it is determined that there is not, the process proceeds to step S906.

  When the difference between AREA_P and AREA_I is less than or equal to a predetermined threshold, it is considered that the camera shake in the optical axis direction of the camera is within a certain range, and the deviation in the direction perpendicular to the optical axis direction of the camera is Make a decision. If the difference between the vertex coordinates of the preview image and the document area is regarded as a vertical deviation, and the deviation exceeds a predetermined threshold, the document area is blurred due to vertical camera shake and the document area is printed. It determines with it not being suitable, and progresses to step S906.

  Note that the vertical displacement is not a major factor causing blurring of the image because it is stationary at the moment of shooting and there is no blur if the distance from the original has not changed. Therefore, the threshold for vertical displacement is set to a loose value, and it is detected that the hand has moved significantly and is assumed to be moving at the time of shooting. In step S905, it is better to perform a more detailed image analysis to make the determination. If the vertical shift is less than or equal to the predetermined threshold, the process proceeds to step S903.

  The above description is a detailed flow of the document photographing process in step S703 in this embodiment. Note that the order of step S1503 and step S903 may be changed. By using the flow of this embodiment, it is possible to predict that blurring has occurred due to camera shake before image quality determination of the document area. This prediction makes it possible to determine that the image quality of the document area is not suitable for printing and enter the re-shooting flow, so that no processing time is required for determining the image quality. In general, since it takes more time to determine the image quality than to determine the amount of camera shake, it is possible to reduce the waiting time of the user when performing re-shooting due to camera shake. The threshold in the present embodiment may provide a mechanism that allows the user to change on the touch panel display 201.

(Example 3)
In the second embodiment, it is determined whether there is camera shake using the position and size of the vertex coordinates of the document area. In the present embodiment, a method of using the motion sensor 310 to determine the degree of blur due to camera shake without analyzing the image of the document area will be described. In the third embodiment, only a portion having a difference from the first embodiment will be described.

  FIG. 16 is a detailed flowchart of the document photographing process in step S703 of FIG. The processing of this flow is executed when the CPU 301 executes a processing program stored in the storage unit 304. Since the difference from the first embodiment is step S1601 and step S1602, only that portion will be described. Note that the processing in steps S902 to S907 in FIG. 16 is the same as the processing in steps S902 to S907 in FIG. 9 described in the first embodiment.

  The processing in step S1601 is basically the same as that in step S901. The differences include the three-dimensional movement of the mobile terminal 101 such as the movement and posture change of the mobile terminal 101 from immediately after performing autofocus by the autofocus mechanism mounted on the camera 203 that is the imaging unit 306 to shooting. This is a point where a simple trajectory is detected by the motion sensor 310. The detected three-dimensional trajectory is transferred to the RAM 302.

  In step S1602, the CPU 301 operates as the blur determination unit 608 in the software block diagram of FIG. In other words, the CPU 301 determines whether the movement of the mobile terminal 101 is within a predetermined range using the three-dimensional trajectory of the mobile terminal 101 at the time of shooting held in the RAM 302. That the movement of the portable terminal 101 is within a predetermined range means that the amount of camera shake is small and the image quality of the document area is suitable for printing. Specifically, the determination is performed using the positional relationship of the camera 203 between the time immediately after autofocusing and the moment of shooting and the movement of the moment of shooting.

  First, it is determined whether the optical axis deviation, which is a deviation between the optical axis direction of the camera 203 immediately after autofocusing and the direction of the optical axis direction of the camera 203 at the moment of shooting, is equal to or less than a predetermined threshold. If there is a certain amount of deviation in the direction of the optical axis, blurring will occur, and if the predetermined threshold is exceeded, it is determined that the document area is blurred and not suitable for printing, and the process proceeds to step S906.

  If the optical axis deviation is equal to or less than a predetermined threshold, then the position of the mobile terminal 101 immediately after autofocus and the position of the mobile terminal 101 at the moment of shooting are the camera 203 of the mobile terminal 101 immediately after autofocus. The amount of deviation from the optical axis direction is obtained. If the positional deviation in the optical axis direction becomes large, the document does not exist at the autofocused location, and blurring is likely to occur. Therefore, if the deviation exceeds a predetermined threshold, it is determined that the document area is blurred and not suitable for printing, and the process proceeds to step S906.

  When the optical axis position direction positional deviation is equal to or smaller than a predetermined threshold, it is determined whether the absolute value of the moving speed / acceleration of the mobile terminal 101 at the moment of shooting is equal to or smaller than the predetermined threshold. If the mobile terminal 101 has an absolute value of movement speed / acceleration exceeding a predetermined threshold at the moment of shooting, it is determined that the document area is blurred due to camera shake, and the process proceeds to step S906.

  If the absolute value of the moving speed / acceleration is equal to or less than the predetermined threshold, it is determined that the document area is suitable for printing, and the process proceeds to step S902.

  The above description is a detailed flow of the document photographing process in step S703 in this embodiment. By using the flow of this embodiment, it is possible to predict that blurring has occurred due to camera shake before judging the image quality of the document area, and to enter the re-shooting flow. Processing time is not required. By using a motion sensor, it is possible to determine whether blur has occurred due to camera shake at a higher speed than to determine whether a document area is blurred by image processing. As a result, it is possible to reduce the waiting time of the user when performing re-shooting due to camera shake. The threshold in the present embodiment may provide a mechanism that allows the user to change on the touch panel display 201.

Example 4
In the first embodiment, as in the processing flow described with reference to FIG. 9, it is determined whether the image quality in the document region is suitable for printing with respect to the captured image captured by the imaging unit 306, and the user re-photographs. A notification for prompting is displayed. More specifically, in the first embodiment, after the CPU 301 issues a shooting instruction to the imaging unit 306, it is determined whether the image quality in the document area is suitable for printing.

  In the first embodiment, in consideration of a time lag due to autofocus, a determination process is performed on a photographed image photographed after a photographing instruction. However, the above-described time lag can be substantially eliminated by using the imaging unit 306 that can perform high-speed autofocus or the imaging unit 306 that does not perform autofocus. Therefore, in this embodiment, before the CPU 301 issues a shooting instruction to the imaging unit 306 before the CPU 301 issues a shooting instruction to the user in step S901 in FIG. It is determined whether the image quality is suitable for printing (S902 to S905).

  When the CPU 301 determines that the image quality in the document area is suitable for printing, the CPU 301 transmits a shooting instruction to the imaging unit 306 by a user instruction or automatically, and shoots the document to obtain a captured image. . Further, when the CPU 301 does not determine that the image in the document area is suitable for printing, the display unit 307 displays information regarding the reason why the image in the document area is not determined to be suitable for printing. Like that. In this case, information indicating that the image in the document area is not suitable for printing may be displayed on the display unit 307.

  The photographed image obtained in this way is determined to have the image quality of the document area suitable for printing. Therefore, the photographed image is considered to satisfy the user's request, and the user is notified of re-shooting. You do n’t have to.

  The obtained captured image is subjected to the processing of S704 to S706 and transmitted to the server 121, as in the first embodiment. The printer 103 prints an image whose original area has been scaled.

(Other examples)
In the first to fourth embodiments described above, the mobile terminal 101 transmits captured image data to the server 121, the server 121 performs image processing on the captured image data, and the printer 103 performs captured image processing. Print the data. However, the mobile terminal 101 may perform image processing and directly transmit the captured image data after image processing to the printer 103. Alternatively, the captured image data captured by the mobile terminal 101 may be transmitted directly to the printer 103 and subjected to image processing in the printer 103 for printing.

  The present invention can also be realized by executing the following processing. In this process, software (computer program) for realizing the functions of the above-described embodiments is supplied to a system or apparatus via a network or various storage media, and the computer of the system or apparatus (or CPU, MPU, etc.) is programmed. Is read and executed.

Claims (17)

Imaging means for acquiring an image by imaging a document;
Setting means for setting print setting information;
Specifying means for specifying a document area from the image acquired by the imaging means;
Determining means for determining whether the image in the specified document area is suitable for printing based on the characteristics of the image in the specified document area and the set print setting information;
Display means for displaying information indicating that the image in the document area is not suitable for printing when the determination means does not determine that the image in the document area is suitable for printing;
An information processing apparatus comprising:   The display means displays information on the reason why the image in the document area was not determined to be suitable for printing when the determination means did not determine that the image in the document area was suitable for printing. The information processing apparatus according to claim 1.   The information processing apparatus according to claim 1, wherein the image pickup unit picks up the image of the original again when the determination unit does not determine that the image of the original area is suitable for printing. Means for obtaining a preview image of the document before the document is imaged by the imaging unit;
The determination unit determines whether the image of the document area is suitable for printing based on the characteristics of the image of the document area and the document area included in the preview image and the image acquired by the imaging unit. 4. The image processing apparatus according to claim 1, wherein a determination is made as to whether an image in the document area included in the image acquired by the imaging unit is suitable for printing based on a positional deviation from the document area included in the image. The information processing apparatus according to any one of claims. Detecting means for detecting a change in position and orientation of the imaging means;
The determination unit performs imaging based on a change in position and orientation detected by the detection unit before determining whether an image of the document area included in the image acquired by the imaging unit is suitable for printing. 4. The information processing apparatus according to claim 1, wherein it is determined whether an image of a document area included in the image acquired by a unit is suitable for printing. Imaging means for acquiring an image by imaging a document;
Setting means for setting print setting information;
Specifying means for specifying a document area from the image acquired by the imaging means;
Determining means for determining whether the image in the specified document area is suitable for printing based on the characteristics of the image in the specified document area and the set print setting information;
When the determination unit determines that the image in the document area is suitable for printing, the image capturing unit captures the image of the document and copies the document to the external device in order to copy the document. A transmission means for transmitting;
An information processing apparatus comprising:   If the determination means does not determine that the image in the document area is suitable for printing, at least information indicating that the image in the document area is not suitable for printing and the image in the document area are printed. The information processing apparatus according to claim 6, further comprising display means for displaying any one of the information on the reason why it is not determined to be suitable. The determination means includes
Based on the shape of the image of the document area included in the image acquired by the imaging unit and the set print setting information, the image of the document area included in the image acquired by the imaging unit is suitable for printing. The information processing apparatus according to claim 1, further comprising a shape determination unit that determines whether or not.   The information processing apparatus according to claim 8, wherein the print settings include settings for paper size, output resolution, and Nin1. The determination means includes
Image quality determination means for determining whether or not the image of the document area included in the image acquired by the imaging means is suitable for printing based on the image quality of the image of the document area included in the image acquired by the imaging means. The information processing apparatus according to claim 1, further comprising: The determination means includes
Shape determining means for determining whether or not the image of the document area included in the image acquired by the imaging means is suitable for printing based on the shape of the image of the document area included in the image acquired by the imaging means. When,
After the determination by the shape determination unit, an image of the document area included in the image acquired by the imaging unit is printed based on the image quality of the image of the document area included in the image acquired by the imaging unit. Image quality judging means for judging whether or not it is suitable,
The information processing apparatus according to claim 1, further comprising: Means for determining an original mode of an image in the original area by analyzing an image in the original area;
The image quality judging means is
Whether the image of the original area included in the image acquired by the imaging means is suitable for printing based on the image quality of the original area included in the image acquired by the imaging means and the determined original mode The information processing apparatus according to claim 10, wherein whether or not to determine is determined.   13. The image in the document area determined to be suitable for printing by the determination unit is printed by a printing unit according to the set print setting information. The information processing apparatus described.   14. The information processing apparatus according to claim 13, wherein the image in the document area determined to be suitable for printing by the determination unit is printed by being scaled to an output image size by the printing unit.   The program for functioning a computer as each means of any one of Claims 1 thru | or 14. An imaging process for acquiring an image by imaging a document;
A setting process for setting print setting information;
A specifying step of specifying a document area from the image acquired by the imaging step;
A determination step of determining whether the image in the specified document region is suitable for printing based on the characteristics of the image in the specified document region and the set print setting information;
A display step of displaying information indicating that the image in the document area is not suitable for printing when the determination step determines that the image in the document area is not suitable for printing;
An information processing method characterized by comprising: An imaging process for acquiring an image by imaging a document;
A setting process for setting print setting information;
A specifying step of specifying a document area from the image acquired by the imaging step;
A determination step of determining whether the image in the specified document region is suitable for printing based on the characteristics of the image in the specified document region and the set print setting information;
If it is determined in the determination step that the image in the document area is suitable for printing, the image capturing step captures the document and copies the document to the external device in order to copy the document. A transmission process to transmit;
An information processing method characterized by comprising: