JP2017163464A - Image correction device, image correction method, and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To automatically switch and execute inflection correction and rectangle correction according to how a page image is distorted without deteriorating usability.SOLUTION: A CPU 77 performs superimposed display of a page image and a mesh image to a display section having a display function and a touch input function, stretches and contracts, rotates, transfers, or deforms the mesh image in response to a touch operation to the display section by a user, performs inflection correction processing when a shape of the mesh image is inflected after edition subjected to stretch and contraction, rotation, transfer, or deformation, selects rectangle correction processing when the shape of the mesh image is a rectangular shape, selects trapezoidal correction processing when the shape of the mesh image is a trapezoidal shape, and corrects the shape of the page image according to the selected distortion correction processing.SELECTED DRAWING: Figure 9

Description

  The present invention relates to an image correction apparatus, an image correction method, and a program.

  2. Description of the Related Art Conventionally, a technique has been proposed in which a book is photographed with a camera while turning the page as it is without being cut, and digitized. When a book is opened in a natural state and photographed from above, the character strings and diagrams in the image are distorted due to the bending of the pages of the book, and it is difficult to read the character strings and diagrams in the image.

  Therefore, for example, in Patent Document 1, an application that performs image processing displays a mesh image on a distorted page image, manipulates the mesh image so as to match the curvature of the paper surface, and performs page processing based on the mesh image. Curve correction for correcting image distortion has been proposed. Further, Patent Document 2 proposes a rectangular correction that divides a page image into a plurality of regions and corrects the distortion of the page image so that each divided region has a correct shape (rectangular shape).

JP 2014-192901 A JP-A-2015-119431

  However, in the correction methods according to Patent Documents 1 and 2 described above, the user's operation method is different, and in the rectangular correction, the straight line is moved as a straight line and the trimming area is adjusted, whereas in the curvature correction, the correction is related to the curve To adjust the trimming area by changing the shape of the curve. As described above, since the respective operation methods are different, there is a problem that the usability is extremely deteriorated when both the correction methods are employed.

  In addition, the user selects for each book whether to perform curvature correction or rectangular correction. That is, when one book is digitized, this page cannot be distinguished from curvature correction and the rest as rectangular correction. Compared to curvature correction, rectangular correction is simpler and can be realized in a shorter time. Also, the curvature correction is more accurate than the rectangle correction. For this reason, there is a problem that if even a part of the curved correction is selected with high accuracy, it takes a long processing time for one book even if accuracy is not required for the remaining pages.

  On the other hand, if you want to process in a short time and select rectangular correction for some parts, even if there is a page with large distortion that requires curvature correction in other parts, it is sufficient because the accuracy is low. There is a problem that distortion cannot be removed. Of course, it is possible to switch the correction method for each page, but for that purpose, the user needs to select a correction method for each page and execute different operation methods. Will be damaged.

  SUMMARY OF THE INVENTION An object of the present invention is to selectively switch and execute a distortion correction process corresponding to a distortion method of a page image without impairing user convenience.

  The page image correction apparatus according to the present invention includes a display control unit that superimposes and displays an image to be processed and a mesh image on an input display unit having a display function and a touch input function, and a user's touch operation on the input display unit. Detecting means for detecting the image, editing means for expanding / contracting, rotating, moving or deforming the mesh image in accordance with the touch operation detected by the detecting means, and editing of the mesh image after editing by the editing means A selection unit that selects one distortion correction process among a plurality of distortion correction processes based on the shape; a correction unit that corrects the shape of the image to be processed by the distortion correction process selected by the selection unit; It is characterized by providing.

  The page image correction method according to the present invention is a page image correction method for correcting the shape of the processing target image based on the shape of the mesh image displayed superimposed on the processing target image. Detecting a user's touch operation on an input display unit that displays the target image and the mesh image, and expanding / contracting, rotating, moving, or deforming the mesh image in accordance with the detected touch operation Selecting one distortion correction process among a plurality of distortion correction processes based on the shape of the mesh image that has been stretched, rotated, moved, or deformed, and the selected distortion correction process to perform the process. Correcting the shape of the target image.

  The program according to the present invention provides a computer of a page image correction apparatus that corrects the shape of an image to be processed based on the shape of a mesh image displayed superimposed on the image to be processed, a display function and touch input. Display control means for superimposing and displaying the image to be processed and the mesh image on an input display section having a function, detection means for detecting a touch operation by the user to the input display section, touch detected by the detection means One of a plurality of distortion correction processes based on the shape of the mesh image edited by the editing means, an editing means for expanding / contracting, rotating, moving or deforming the mesh image according to an operation A selection means for selecting a process, and a correction method for correcting the shape of the image to be processed by the distortion correction process selected by the selection means. , Characterized in that to function as a.

  According to the present invention, it is possible to selectively switch and execute the distortion correction processing corresponding to the distortion method of the page image without impairing the convenience for the user.

1 is a perspective view showing a schematic configuration according to an example of a document camera system 1 of the present embodiment. It is a perspective view which shows schematic structure by the other example of the document camera system 1 of this embodiment. It is a block diagram which shows the schematic structure of the information processing terminal 7 by this embodiment. It is a conceptual diagram for demonstrating an example of the user interface with respect to a rectangle correction process. It is a conceptual diagram for demonstrating an example of the user interface with respect to a curvature correction process. It is a conceptual diagram for demonstrating the common point of a rectangle correction process and a curvature correction process. It is a schematic diagram showing a state of a mesh image 9b when five control points 80 are in a straight line but are not equally spaced. It is a flowchart for demonstrating the correction process in the information processing terminal 7 by this embodiment. It is a flowchart for demonstrating the correction process in the information processing terminal 7 by this embodiment. It is a schematic diagram for demonstrating the touch operation for deform | transforming the mesh image 9 by this embodiment. It is a schematic diagram for demonstrating the touch operation for operating individually the control point 80 of the mesh image 9 by this embodiment, and deform | transforming. It is a schematic diagram for demonstrating the touch operation for operating all the control points 80 of the mesh image 9 by this embodiment, and expanding and contracting simultaneously. It is a schematic diagram for demonstrating the touch operation for operating all the control points 80 of the mesh image 9 by this embodiment simultaneously, and translating. It is a schematic diagram for demonstrating the touch operation for operating and rotating all the control points 80 of the mesh image 9 by this embodiment simultaneously. It is a schematic diagram which shows a mode in the case of rotating a mesh image by the touch operation with three fingers by this embodiment.

A. First Embodiment Hereinafter, the best mode for carrying out the present invention will be described with reference to the drawings. However, although various technically preferable limitations for implementing the present invention are given to the embodiments described below, the scope of the invention is not limited to the following embodiments and illustrated examples.

  FIG. 1 is a perspective view showing a schematic configuration according to an example of a document camera system 1 of the present embodiment. In the following description, a case where the page P of the book B is turned from left to right will be described. As shown in FIG. 1, the document camera system 1 includes a document camera 2 as an imaging unit that images a page P of a book B, a page turning device 3 that turns the page P of a book B, a document camera 2, and a page turning. A personal computer 4 connected to the apparatus 3 for communication is provided.

  The document camera 2 is provided with a stand portion 21 and a camera 22 attached to the upper end of the stand portion 21. The stand unit 21 can be tilted in the front-rear direction and the left-right direction and can be expanded and contracted in the vertical direction so that the relative positional relationship between the book B and the camera 22 can be adjusted. The lens of the camera 22 faces downward so that the book B is within the angle of view. A position adjustment mechanism is provided at a joint portion between the camera 22 and the stand portion 21, so that the direction in which the lens of the camera 22 faces can be adjusted.

  The page turning device 3 holds the page P at the page turning source position of the page P of the book B on the holding table 6 and the holding table 6 that holds the opened book B, and the page P with respect to the page P at the page turning destination position. A turning device main body 30 for releasing the holding is provided.

  The holding table 6 includes a pair of holding plates 61 and 62 that can be folded by a hinge (not shown). Here, when the page P of the book B is turned from left to right, one of the pair of holding plates 61 and 62, the one holding plate 61 arranged on the left side is placed along the table, and on the right side. The other holding plate 62 to be placed is inclined and placed on the table so that it rises at a predetermined angle with respect to the one holding plate 61. On one holding plate 61, the page P that is the turning origin position of the book B is placed, and on the other holding plate 62, the page P that is the turning destination position of the book B is placed.

  As a result, the holding table 6 holds the book B so that the page P at the turning destination is more inclined than the page P at the turning original position in the direction in which the binding occurs. Since the pair of holding plates 61 and 62 can be folded by hinges, the angle formed by the pair of holding plates 61 and 62 can be adjusted, and the inclination angle of the page P at the turning point position with respect to the horizontal plane can be adjusted. It is adjustable.

  The turning device main body 30 includes an arm portion 34 that swings around a drive shaft 33, a suction portion 35 that is attached to the tip of the arm portion 34 and sucks against the page P of the book B, a drive shaft 33, and an arm portion. 34 and the pedestal part 38 that supports the suction part 35, and the blower part 5 that applies wind to the page P at the turning destination position by passing the wind above the page P at the turning origin position, and controls each part. And a control unit (not shown).

  As shown in FIG. 1, a rotating body 321 is attached to the tip of the drive shaft 33. An arm portion 34 is attached to the rotating body 321 along a plane orthogonal to the drive shaft 33. The arm portion 34 is, for example, a resin rectangular plate member. The arm portion 34 has a flat plate shape with a cross section perpendicular to the longitudinal direction. A suction part 35 is attached to the tip of the arm part 34 via a drive part 37 such as a motor. The adsorption part 35 is a substantially cylindrical adhesive part.

  FIG. 2 is a perspective view showing a schematic configuration according to another example of the document camera system 1 of the present embodiment. It should be noted that portions corresponding to those in FIG. The document camera system 1 shown in FIG. 2 differs only in that there is no personal computer 4 communicatively connected to the document camera 2, document camera 2, and page turning device 3 shown in FIG. The same. In the document camera system 1 shown in FIG. 2, instead of the document camera 2 and the personal computer 4, a smartphone or a tablet terminal (hereinafter referred to as an information processing terminal 7) including a camera unit is used. The information processing terminal 7 is installed on a stationary table 8 such as a table having a flat plate surface at an appropriate height, for example. The information processing terminal 7 is installed face down such that the display unit 71 faces upward and the imaging unit 72 faces downward so that the book B is within the angle of view. The imaging unit 72 is provided at the upper center of the information processing terminal 7 or at the upper corner (right end side when viewed from the display unit 71).

  1 and 2, in the page turning operation, first, the arm portion 34 is moved to the page turning origin position side of the page P on the holding plate 61 side, and the suction portion 35 is moved to the page turning origin position. The page P is adsorbed, and the arm 34 is moved to the turning position on the holding plate 62 side in the adsorbed state. The page P follows the forward movement of the arm portion 34 and moves to the turning destination position. Then, the suctioned page P is separated by rotating the suction unit 35 by the driving unit 37 at the turning point position. At this time, at a predetermined timing, the page 22 on the holding plate 61 side that has not been turned yet is imaged by the camera 22 or the imaging unit 72 of the information processing terminal 7. Thereafter, the arm portion 34 is moved to the turning original position side on the holding plate 61 side in the direction opposite to the forward movement. The adsorbing unit 35 adsorbs to a new page P (page after imaging) on the turning original position side. By repeating this reciprocating operation, the page P turning operation proceeds.

  Until the last page is reached, all pages on one side of the book B (for example, odd pages) are photographed by the camera 22, and the image data is numbered one by one (every image) and is stored in the personal computer 4 or the information processing terminal 7. The data is transferred to a CPU 77 (described later) and held in a predetermined storage unit or the like. The user reverses the book B and places it on the holding table 6 and performs the above-described page turning operation again. Thus, all the pages on one side of book B (for example, even-numbered pages) are photographed by camera 22 until the final page is reached, and are numbered one by one (every imaging) as described above, and personal computer 4 or information processing The data is transferred to a CPU 77 (described later) of the terminal 7 and held in a predetermined storage unit or the like. Thereafter, the odd pages and the even pages are rearranged alternately in the page order, and the scanned images of all the pages are collected.

  At this stage, the image data of page P is distorted in the entire page, images in the page, character strings, charts, etc. due to the deflection of the page of the book. The processing terminal 7 corrects the distortion of the page image by performing curvature correction, rectangle correction, or trapezoid correction on the page image by a predetermined application.

  In the present embodiment, when a mesh image is displayed on a distorted page image and the mesh image is deformed so as to match the distortion of the page image, a user operation (touch operation) on the mesh image is performed in one user interface. Etc.), the line segments that make up the mesh image can be matched to the distortion of the page image with a curve, or the line segments that make up the mesh image can be translated, rotated, or expanded or contracted while keeping the straight line. It is also possible to match the degree of distortion. For this reason, in this embodiment, the shape of the mesh image can be operated with a single user interface by a unified user operation without being aware of which correction method is applied. Further, even when the correction method is changed in units of pages as necessary, user convenience is not impaired.

  Further, in the present embodiment, when a curve is included in a line segment that forms a mesh image combined with a page image, curvature correction is applied, and when a line segment that forms a mesh image is a straight line, In order to apply rectangular correction or trapezoid correction, switching is performed in units of pages according to the line segment configuration of the mesh image. In other words, the user does not have to select whether to perform curvature correction, rectangular correction, or trapezoid correction after confirming the distortion of the page image, or to switch the user interface for each. . For this reason, in this embodiment, the shape of the mesh image is identified for each page, and the correction method (algorithm) for the corresponding page image is switched based on the shape of the mesh image. ) And a reduction in correction accuracy can be prevented.

  In the following, a case where an information processing terminal 7 such as a smartphone or a tablet terminal is used as an apparatus for performing image processing such as curvature correction, rectangle correction, or trapezoid correction will be described. Needless to say, although the configuration is different, it can be realized by the same software (application program).

  FIG. 3 is a block diagram showing a schematic configuration of the information processing terminal 7 according to the present embodiment. In FIG. 3, the information processing terminal 7 includes a communication unit 70, a display unit 71, an imaging unit 72, a ROM 73, a RAM 74, an operation unit (touch panel) 75, a recording medium 76, and a CPU 77. The communication unit 70 connects to a network such as the Internet using, for example, mobile communication, Bluetooth (registered trademark), and / or a wireless LAN (WiFi). The imaging unit 72 includes a lens block including an optical lens group and an imaging element such as a CCD or a CMOS, and captures an image entered from the lens block by the imaging element. In particular, in the present embodiment, the imaging unit 72 images the page P of the book B.

  The ROM 73 stores programs executed by the CPU 77 described later, various parameters necessary for operations, and the like. The RAM 74 stores data such as temporary data when a CPU 77 described later executes a program, various application programs, and various parameters necessary for executing the application. In particular, in the present embodiment, the RAM 74 stores captured images, corrected corrected images, and the like.

  The display unit 71 includes a liquid crystal display, an organic EL (Electro Luminescence) display, and the like, and displays icons associated with specific functions and applications, application screens, various menu screens, and the like. The operation unit (touch panel) 75 detects direct contact or proximity of a finger or a stylus (pen). The operation unit (touch panel) 75 may include mechanical switches such as a power button and a volume button. In particular, in the present embodiment, a page image captured on the display unit 71 and a control point on a mesh image displayed superimposed on the page image are individually moved from the operation unit (touch panel) 75 by a user's touch operation. Or a touch operation for transforming the shape of the mesh image to match the shape of the page image by inputting, contracting, rotating, or translating while maintaining the straight line portion of the mesh image. The display unit 71 may have a display function and a touch input function, and a touch for changing the shape of the mesh image to match the shape of the page image by a user's touch operation on the display unit 71. An operation may be input.

  The recording medium 76 stores various data such as captured image data. The CPU 77 controls the operation of each unit by executing the program stored in the ROM 73 described above. In particular, in the present embodiment, the CPU 77 executes the image processing program to deform the shape of the page image together with the mesh image so that the mesh image deformed by the user's touch operation has a correct shape (rectangular shape). Thus, a curvature correction process, a rectangular correction process, and a trapezoidal correction process for correcting deformation and curvature of the page image are executed.

  The curvature correction process is a process applied when the upper and lower line segments of the mesh image are curved and distorted. On the other hand, the rectangle correction process is a process applied when the mesh images are all composed of straight lines. The trapezoidal correction process is a process applied when the mesh image has a trapezoidal shape (the upper and lower line segments are straight lines and the left and right components are inclined with respect to the vertical line). Hereinafter, the curvature correction process, the rectangle correction process, and the trapezoid correction process will be described in detail.

(1-1) Rectangular correction processing (rectangular trimming)
FIG. 4 is a conceptual diagram for explaining an example of a user interface for the rectangle correction processing. FIG. 4 shows a state where the mesh image 9a is superimposed on the page image 10 and displayed on the display unit 71 as a user interface for the rectangular correction processing. The user touches and drags the outermost line of the mesh image 9a with one finger so that the side moves in a direction perpendicular to the side (for example, the direction of the thick arrow A above the side). Can be made. When one side moves, the side coupled with the side expands and contracts according to the movement, and maintains the shape of the rectangular parallelepiped as a whole.

  Further, the page image 10 may be tilted. In this case, since the mesh image 9a cannot be tilted by simply dragging each side of the mesh image 9a, the mesh image 9a cannot be moved to a desired position. Therefore, the “rotate” button 711 is used to rotate the entire page image 10 so that the inclination is not noticeable. Then, the mesh image 9a is moved again, and is superimposed on a position that substantially matches the page image 10.

  When the user determines that the rotation state of the mesh image 9 a and the page image 10 is appropriate, the user presses a “preview” button 712. Thus, parameters (mesh position, image rotation angle) are determined, and the page image 10 after the rectangle correction processing (rectangular trimming) is displayed. If the user determines that further correction is necessary, the same process is repeated.

(1-2) Curve correction processing (curve trimming)
FIG. 5 is a conceptual diagram for explaining an example of a user interface for the curvature correction processing. FIG. 5 shows a state where the mesh image 9 b is superimposed on the page image 10 and displayed on the display unit 71 as a user interface for the curvature correction processing. The plurality of control points 80 are control points of Bezier curves (upper line UL and lower line DL) constituting the mesh image 9b, and exist as a set of five points on the upper line UL and the lower line DL. When the user touches one of the control points 80 with one finger and drags the control point 80 so that the shape of the page image 10 matches the shape of the mesh image 9b, the mesh image is displayed as a change in the Bezier curve. It deforms so that the overall shape of 9b follows. That is, by operating the control point 80, the upper line UL and the lower line DL of the mesh image 9a can be curved.

  When the book B is photographed, the book B is bent due to the pulling of the closed portion of the book B or the thickness of the book B, and thus the above-described rectangular correction processing or the like results in a distorted image. Therefore, when the mesh image 9b is deformed in a curve, and the “preview” button 712 is pressed after aligning the boundary of the page image 10 and the curved line of the bent character, the page image 10 is matched with the curve of the mesh image 9b. The page image 10 is deformed so that the curve of the mesh image 9b is corrected to a straight line, and the page image 10 in which the deflection is corrected is generated. Note that the rotation button 711 in the user interface for the rectangle correction processing does not exist due to its characteristics.

(1-3) Close Parts of Both FIG. 6 is a conceptual diagram for explaining common points between the rectangle correction process and the curvature correction process. In the rectangle correction process, the shape of the mesh image 9a is adjusted by moving a straight line, whereas in the curve correction process, the control point 80 related to the curve is moved to adjust the shape of the mesh image 9b. The curvature correction process also has a function of changing the distortion of the page image, there is a difference in the algorithm, and the output page image is different.

  However, a similar page image may be obtained by applying two algorithms to a page image. That is, as shown in FIG. 6, the line segments constituting the mesh image 9b for the curvature correction process are all straight as in the mesh image 9a for the rectangle correction process. The conditions for this are as follows.

Condition 1: Five control points 80 are aligned in a straight line on the upper and lower sides.
Condition 2: The upper and lower end points (rectangular control points 80 in the figure) are in the same position with respect to the horizontal coordinates of the page image 10 (the right end control point 80 of the upper line UL and the right end control point 80 of the underline DL). And the control point 80 at the left end of the upper line UL and the control point 80 at the left end of the underline DL are the same).

  If both conditions 1 and 2 are satisfied, there is no difference between the processed page image 10 when the curve correction process (curve trimming) is performed and when the rectangle correction process (rectangular trimming) is performed. Therefore, even if the user performs an operation for the curvature correction process, the rectangle correction process may be applied.

  Further, in the curve correction process (curve trimming), if condition 1 is satisfied, all the lines become straight lines. Therefore, it is not necessary to perform a bending process for correcting the bending. Therefore, when only condition 1 is satisfied, it is possible to correct the page image 10 using the trapezoidal correction process described above, which takes more processing time than the rectangular correction process but takes less time than the curvature correction process. is there.

  As shown in FIG. 7, the five control points 80 are in a straight line. However, when the control points 80 are not equally spaced, the shape itself is not changed. That is, if the shape of the mesh image 9b is determined based on the above conditions 1 and 2, it is possible to determine which of the curvature correction process, the rectangle correction process, and the trapezoid correction process is applied. Hereinafter, the mesh image in the user interface according to the present embodiment will be described as a mesh image 9 that can be commonly used in any of the curvature correction process, the rectangle correction process, and the trapezoid correction process.

  8 and 9 are flowcharts for explaining the correction process in the information processing terminal 7 according to the present embodiment. In the information processing terminal 7, it is assumed that the page images 10 of all pages of the book B are already stored in the recording medium 76 as image data in a predetermined format in page order. After imaging the page of book B, the user instructs the information processing terminal 7 to execute image correction processing.

  First, the CPU 77 stores the first page image 10 of the book B from the recording medium 76 into the RAM 74, and outputs the page image 10 and the initial undistorted mesh image superimposed on the page image 10 to the display unit 71. (Step S10). Next, the CPU 77 determines whether or not the user has performed a touch operation on the operation unit (touch panel) 75 (step S12). If there is no touch operation by the user (NO in step S12), the CPU 77 returns to step S10 and continues displaying the page image 10 and the mesh image 9.

  On the other hand, when a user touch operation is performed on the operation unit (touch panel) 75 (YES in step S12), the CPU 77 determines whether the number of touched fingers is one or two, that is, 1 It is determined whether it is a touch operation with two fingers or a touch operation with two or more fingers (step S14).

  FIGS. 10A and 10B are schematic diagrams for explaining a touch operation for deforming the mesh image 9 according to the present embodiment. When the user wants to change the shape of the mesh image itself, that is, when the user wants to individually operate the control points on the mesh image, as shown in FIG. The control point 80 (all the control points in the figure are touched) may be touched, and the finger may be dragged (slid in the touched state) in the desired direction (individual operation). That is, in this individual operation, one control point 80 is operated (moved) from among a plurality of control points 80 (5 points above and below the mesh image in the present embodiment) on the mesh image formed by Bezier curves. Will do.

  As shown in FIG. 10A, the user expands and contracts the shape of the mesh image 9 displayed on the page image 10 to match the shape of the page image 10 by a touch operation. When the user moves the control point 80 so that the shape of the page image 10 matches the shape of the mesh image 9 by a touch operation, the shape of the mesh image 9 follows the movement of the control point 80 in real time. It is designed to deform.

  The uppermost line UL and the lowermost line DL at the bottom of the mesh image 9 are drawn with a quartic Bezier curve. By moving the five control points 80 of the upper line UL and the five control points 80 of the lower line DL, the upper line UL and the lower line DL can be freely moved. The CPU 77 uses a predetermined mesh generation algorithm based on the upper line UL and the lower line DL, and derives the width of the horizontal and vertical lines sandwiched between the upper line UL and the lower line DL, the curve curve, etc. in real time. The shape of the mesh image 9 is deformed.

  In this embodiment, the control points 80 for deforming the mesh image 9 are a total of 10 points in the upper and lower directions. However, the present invention is not limited to this, and each intersection of a plurality of regions constituting the mesh image 9 may be used as the control point. . In addition, without being limited to the upper line UL and the lower line DL, the user can perform a touch operation on a location that the user wants to deform so that the surrounding intersection including the touched location functions as a control point for the deformation. May be. In this case, in order to show that it is effective as a control point, the dot at the corresponding intersection may be enlarged or the color may be changed.

  On the other hand, when the user wants to extend, rotate, or translate while maintaining the straight part of the mesh image without changing the shape of the mesh image, that is, an operation that moves all the control points on the mesh image at the same time. If desired, as shown in FIG. 10B, a predetermined operation may be performed by touching the mesh image 9 with two or more fingers (overall operation). In other words, in this overall operation, all control points 80 ((● in the figure are all control points; in the example shown in the figure) are simultaneously operated (without changing the relative positions of each other) (expansion / contraction, rotation, translation). Will do.

  Here, as described above, the screen image of the display unit 71 displays a mesh image 9 composed of Bezier curves together with a captured page image (not shown). The Bezier curve (fourth order) constituting the mesh image 9 has five control points on the top and bottom. Note that the mesh image 9 itself is a user interface formed from a curve, and is not directly related to the curve correction process or the trapezoid correction process described later. That is, in the curvature correction process or the trapezoidal correction process, since the control point 80 is a movement target, it is necessary to specify the control point 80 to be moved from the position of the finger by the touch operation.

  Therefore, for example, as shown in FIG. 10B, it is assumed that there are five control points 80. First, rectangular coordinates including all five control points 80 are derived in advance. The derivation of the rectangle is the control point having the smallest abscissa (x coordinate) among the five points (control point 80b in the example shown), the maximum control point (control point 80a in the example shown), and the ordinate (y coordinate). ) Is the minimum control point (control point 80b in the illustrated example) and the maximum control point (control point 80a in the illustrated example), and the upper, lower, left, and right sides may be configured based on the minimum and maximum amounts. . In the example shown in the figure, the rectangular shape matches the outline of the mesh image 9, but the present invention is not limited to this.

  Next, the center point CP shown in FIG. 10B on the geometrical positions of two or more fingers by the touch operation is measured. When the center point CP is in the rectangle, it is determined that all the control points 80 in the rectangle are expanded / contracted, rotated, or translated at the same time (without changing their relative positions).

  As a measuring method of the center point CP, the coordinates of all positions of the finger of the touch operation are measured, and the average value of each of the x-coordinate and the y-coordinate may be taken with these values. In the example shown in FIG. 10B, a touch operation is performed with two fingers. For example, if the touch position T1 (x1, y1) of the first finger and the touch position T2 (x2, y2) of the second finger, the coordinates of the center point CP of the two fingers are ((x1 + x2) / 2 , (Y1 + y2) / 2). In this way, it is determined whether the center point CP is in the rectangle from the coordinates of the center point CP of the two fingers and the coordinates of the rectangle including all the five control points 80. Can do.

  Then, as shown in FIG. 10A, when the touch operation is one finger (NO in step S14), the CPU 77 displays on the mesh image constituted by the Bezier curve by the touch operation of one finger. The control point 80 is determined to be an individual operation, and a deformation process for moving the corresponding control point according to the movement of one touched finger is executed (step S16).

  FIG. 11 is a schematic diagram for explaining a touch operation for individually operating and deforming the control points 80 of the mesh image 9 according to the present embodiment. That is, as shown in FIG. 11, the user moves the finger in a direction in which the user wants to deform the mesh image so that the shape of the mesh image matches the shape of the page image 10 while touching the control point 80 with the finger. The CPU 77 moves the control point 80 on the mesh image constituted by the Bezier curve according to the moving direction and moving amount of the touching finger. The shape of the mesh image is deformed in real time in conjunction with the movement of the control point 80.

  Although details will be described later, in order to increase, decrease, or partially adjust the degree of curvature correction while the user confirms such movement of the control point 80 in real time, the mesh image is used. The shape can be changed by moving the nine control points 80 by a touch operation of one finger.

  On the other hand, when the touch operation is two or more fingers (YES in step S14), the CPU 77 detects the operation (movement) of the touching finger (step S18), and what kind of operation is performed? Is determined (step S20). In the present embodiment, in the case of a touch operation with two or more fingers, the movement of the distance between the two fingers (expansion / contraction), or the two fingers are maintained while the relative positions of the two fingers are maintained. Whether to rotate (arc) around the center point as a rotation axis (rotation) or to move two fingers (parallel movement) while maintaining the relative position of the two fingers Judging.

Here, the expansion / contraction, rotation, and parallel movement will be described.
FIGS. 12A to 12C are schematic diagrams for explaining a touch operation for simultaneously operating and expanding and contracting all the control points 80 of the mesh image 9 according to the present embodiment. Expansion and contraction is an operation in which each control point moves away from or approaches a certain point. According to this expansion / contraction operation, the mesh image 9 composed of Bezier curves can be enlarged or reduced while maintaining its shape. For touch devices such as smartphones and tablets, this is an operation method called pinch-in or pinch-out. The coordinates of the touch positions T1 and T2 by the two fingers and the coordinates of the center point CP of the two fingers are measured for several frames from the start of the touch. Then, it is assumed that the center point CP does not move within the allowable range, and the touch positions T1 and T2 by the two fingers move by moving away from or approaching the radial direction within the allowable range from the center point CP, respectively. When detecting a vector, that is, a movement vector in which the touch position T1 and the touch position T2 move in directions opposite to each other, it is determined that the user's touch operation is an expansion / contraction operation.

  FIGS. 13A and 13B are schematic diagrams for explaining a touch operation for simultaneously operating and rotating all the control points 80 of the mesh image 9 according to the present embodiment. Rotation is an operation in which all points rotate by the same angle around a certain point. According to this rotation operation, the mesh image 9 composed of Bezier curves is only rotated while maintaining its shape. That is, if the line is straight from the beginning, the operation is such that the inclination changes while the straight line is maintained. The operation method will be briefly described. Touching the mesh image 9 with two fingers, the touch position T1 of one finger is moved in a certain direction and the touch position T2 of the other finger is traced in the opposite direction. For example, if the index finger and the middle finger are touched on the operation unit (touch panel) 75 and the finger is rotated around the center point of the two fingers, the rotation operation is performed. The coordinates of the touch positions T1 and T2 of the two fingers and the coordinates of the center point CP of the two fingers are measured for several frames from the start of the touch. Then, if the center point CP is not moved within the allowable range, and if it is detected that the movement vectors of the touch positions T1 and T2 by the two fingers are in opposite directions, the user's touch operation is a rotation operation. It is judged that.

  FIGS. 14A and 14B are schematic diagrams for explaining a touch operation for simultaneously operating and controlling all the control points 80 of the mesh image 9 according to the present embodiment. The parallel movement is an operation in which all control points 80 move in the same direction and the same movement amount. The operation method will be briefly described. When the mesh image 9 composed of Bezier curves is touched with two fingers and the touch positions T1 and T2 are shifted in the same direction, all the control points 80 corresponding to the movement amount are obtained. Move in the same direction. For example, when the index finger and the middle finger are touched on the operation unit (touch panel) 75 and the screen is traced so as to move the entire hand, the translation operation is performed. The coordinates of the touch positions T1 and T2 of the two fingers and the coordinates of the center point CP of the two fingers are measured for several frames from the touch start. When it is detected that the moved vectors are the same within a certain allowable range, it is determined that the user's touch operation is a parallel movement operation.

  Returning to the flowchart shown in FIG. 8, for example, as shown in FIG. 12A, when the operation is to change the distance between two fingers (extension / contraction) (extension / contraction in step S <b> 20), the CPU 77. Performs an expansion / contraction process that expands / contracts without changing the relative position of the control point 80 of the mesh image formed by the Bezier curve according to the movement amount of the two touched fingers (step S22). For example, as shown in FIG. 12B, when the touch position is moved so that the distance between the touch positions T <b> 1 and T <b> 2 of the two fingers is widened, the mesh image indicates the amount of movement of the two fingers. Accordingly, the control points 80 are expanded (enlarged) without changing the relative positions thereof. On the other hand, as shown in FIG. 12 (c), when the touch position is moved so that the distance between the touch positions T1 and T2 of the two fingers is narrowed, the mesh image becomes the amount of movement of the two fingers. Accordingly, the control point 80 is reduced without changing the relative position.

  More specifically, in the calculation of expansion / contraction, first, the coordinates M (xm, ym) of the center point CP of the two touch positions T1, T2 are recorded. Using only one coordinate of the touch position, the coordinate A (xa, ya) of the previous frame and the coordinate B (xb, yb) of the current frame are used. As the distances RA and rb of MA and MB, the expansion / contraction rate c / rb / ra between the frames is assumed. For a certain control point X, the vector MX is multiplied by c to be MX '. The end point X 'of this vector is the coordinate of the control point after expansion and contraction. If this is applied to all the control points 80 for each frame and the display is updated, the mesh image 9 can be expanded and contracted.

  Thus, the mesh image 9 can be freely expanded and contracted while maintaining the shape of the straight line portion of the mesh image 9. As a result, the mesh image 9 can be easily matched with the linear region in the page image 10.

  On the other hand, as shown in FIG. 13 (a), the two fingers rotate (circulate) around the center point thereof as a rotation axis while maintaining the relative position of the two fingers (rotation). In the case of (rotation of step S20), the CPU 77 uses a Bezier curve in the rotation direction of the two touching fingers and the rotation amount (rotation angle) corresponding to the rotation amount (rotation angle) of the finger. Without changing the relative position of the configured control point 80, a rotation process for rotating the entire mesh image 9 around the center point CP of the two fingers as the rotation axis is executed (step S24). For example, as shown in FIG. 13B, when two fingers rotate counterclockwise in a touched state, the mesh image 9 corresponds to the amount of movement in the rotation direction of the two fingers. It will rotate without changing the relative position of each control point 80.

More specifically, there are two types of rotation derivation methods.
(1) Derivation method in the case of rotating with the center point CP of the two touch positions T1 and T2 as the rotation axis (2) The amount of movement for each frame of the two touch positions T1 and T2 is calculated, Derivation method when rotating around the axis

  The above (2) is accurate, but some correction is necessary because the rotation axis changes every frame and becomes unstable (slightly vibrates). The above (1) is slightly different from the actual but stable. Moreover, although not accurate, it seems that the rotation axis between fingers is not so different from the center point CP. There is actually no sense of incongruity. Therefore, the example (1) will be described.

First, the coordinates M (xm, ym) of the center point CP are recorded. If the information processing terminal 7 has a function of detecting the rotation angle, the angle may be used. Otherwise, using one of the touched coordinates as the point A (xa, ya) of the previous frame coordinate and the coordinate B (xb, yb) of the current frame, vector MA and vector MB MA · MB = | MA || MB | cosθ
Thus, cos θ and sin θ are respectively obtained (θ is a rotation angle).

Next, the control point 80 is rotated.
One of the control points 80 is assumed to be X (x, y). Since it is desired to convert the coordinates of the control point 80 using the rotation matrix, the coordinates X are converted so that the coordinates M of the center point CP, which is the rotation axis, are moved to the origin.
X ′ = X−M = (x−xm, y−ym) = (x ′, y ′)

Multiply this X 'by the rotation matrix,
x ″ = x′cos θ−y′sin θ
y ″ = x′sin θ + y′cos θ
Get.

Since this coordinate system is a coordinate system in which the center point is the origin, it is restored.
(X ″ + xm, y ″ + ym)
This coordinate is the coordinate of the control point 80 after rotation. This is changed for every control point 80 for each frame, and the display of the mesh image 9 is updated to rotate.

Regarding rotation, a touch operation with three fingers is also conceivable.
In an actual usage example, two upper and lower Bezier curves constituting the mesh image 9 are used, and the two Bezier curves are rotated. That is, the center point CP when touched by three fingers is set as the rotation axis, and the above-described calculation is applied to all control points 80 (upper five points, lower five points, ten points in total) of the upper and lower Bezier curves. In this way, the mesh image 9 can be rotated while the entire shape is maintained.

  FIGS. 15A and 15B are schematic diagrams illustrating a state in which a mesh image is rotated by a touch operation with three fingers according to the present embodiment. In the case of three fingers, as in the case of two fingers, as shown in FIG. 15 (a), the relative touch positions T1, T2, and T3 of the three fingers are maintained. When two fingers are rotated (circular) around the center point CP as a rotation axis, the center points of the three fingers are rotated in the direction of rotation of the three fingers touching, as shown in FIG. The entire mesh image 9 is rotated by the rotation amount (rotation angle) corresponding to the rotation amount (rotation angle) of the finger with CP as the rotation axis.

  Thus, the mesh image can be freely rotated while maintaining the shape of the straight line portion of the mesh image. As a result, the mesh image can be easily matched with a linear region in the page image 10.

On the other hand, as shown in FIG. 14A, when the movement of two fingers (parallel movement) is performed while the relative touch positions T1 and T2 of the two fingers are maintained ( In step S20, the CPU 77 changes the relative position of each control point 80 of the mesh image formed by the Bezier curve in the direction of movement of the two touching fingers and in accordance with the amount of movement of the fingers. The parallel movement process for translating the entire mesh image 9 is executed (step S26). For example, as shown in FIG. 14 (b), when two fingers move downward in the drawing state, the mesh image 9 indicates the amount of finger movement in the direction of movement of the two fingers. Accordingly, the control points 80 are moved in parallel without changing the relative positions.
More specifically, the translation is derived by calculating the movement vector of the center point CP from the previous frame and the current frame, and adding the movement vector to each of the coordinates of the five control points 80 to obtain a new control point 80. The display of the control point 80 is also updated.

  Thus, the mesh image can be freely translated while maintaining the shape of the straight line portion of the mesh image. As a result, the mesh image can be easily matched with a linear region in the page image 10.

  In step S16, S22, S24, or S26 described above, the CPU 77 deforms, expands, contracts, rotates, or translates the mesh image 9 (control point 80) according to the user's touch operation, and then processes the mesh image. 9 is superimposed on the page image 10 and output to the display unit 71 (step S28). Next, the CPU 77 determines whether or not to proceed to the next process, that is, the curvature correction process or the trapezoidal correction process for the page image 10 based on the mesh image 9 (step S30).

  If the user visually recognizes the mesh image 9 displayed in step S28 and determines that the mesh image 9 does not match the page image 10 yet and the shape, size, angle, and position of the mesh image 9 need to be corrected. For example, an instruction to return to the deformation process of the mesh image 9 is input by a touch operation or the like. On the other hand, if the mesh image 9 matches the page image 10 with sufficient accuracy, an instruction to proceed to the next process is input by, for example, a touch operation. Alternatively, if the user determines that the mesh image 9 does not yet match the page image 10 and that the shape, size, angle, and position of the mesh image 9 need to be corrected, no instruction is given. If the page image 10 matches with sufficient accuracy, an instruction to proceed to the next process may be input.

  When an instruction to return to the deformation process of the mesh image 9 is input or when an instruction to proceed to the next process is not input (NO in step S30), the CPU 77 returns to step S12 and returns to the mesh image 9 described above. Repeat the process to correct. When viewed from the user, by performing a predetermined touch operation on the mesh image 9 displayed on the display unit 71, the shape of the mesh image 9 can be continuously deformed without interruption, or the mesh image 9 It is possible to change the size, the angle, and the position.

  When the user visually recognizes the mesh image 9 displayed in step S28, determines that the mesh image 9 matches the page image 10 with sufficient accuracy, and inputs an instruction to proceed to the next process ( Based on the current shape of the mesh image 9, the CPU 77 selects a curvature correction process, a rectangular correction process, or a trapezoid correction process for the page image 10 so that the page image 10 has a substantially rectangular shape. (Step S32). More specifically, the CPU 77 performs a curvature correction process on the page image 10 when the mesh image 9 has a curved shape, and converts the page image 10 into a page image 10 when the mesh image 9 has a rectangular shape. On the other hand, the rectangle correction process is executed, and when the mesh image 9 has a trapezoidal shape, the keystone correction process is executed on the page image 10.

  More specifically, the CPU 77 first determines whether or not both sets of control points of the uppermost line UL and the lowermost line DL of the mesh image 9 are aligned, that is, the upper line UL and the lower line DL. Are both straight lines (step S40 in FIG. 9). When the upper line UL and the lower line DL are both straight lines (YES in step S40), the CPU 77 determines that the left and right end points of the upper line UL and the left and right end points of the lower line DL are in the horizontal position and straight line (upper line). It is determined whether the lengths of UL and underline DL are equal and the straight lines (upper line UL and underline DL) are parallel (step S42).

  Then, when the positions of the start points of the upper line UL and the lower line DL in the mesh image 9 in the horizontal direction and the lengths of the straight lines are equal and the straight lines (the upper line UL and the lower line DL) are parallel (YES in step S42). The CPU 77 performs rectangular correction processing (rectangular trimming) on the page image 10 based on the correction parameters derived from the mesh image 9 because the mesh image 9 (page image 10) has a rectangular shape (cuboid). Execute (Step S44). Thereafter, the process returns to the flowchart of FIG.

  On the other hand, although the upper line UL and the lower line DL are both straight lines (YES in step S40), “the positions of the start points of the upper line UL and the lower line DL in the horizontal direction in the mesh image 9 and the lengths of the straight lines are equal, and the straight lines When (the upper line UL and the lower line DL) are not parallel ”(NO in step S42), the CPU 77 derives from the mesh image 9 because the mesh image 9 (page image 10) has a trapezoidal shape (or a quadrangle). Based on the correction parameters to be performed, a keystone correction process (trapezoidal trimming) is performed on the page image 10 (step S46). In the trapezoidal correction process, a process generally called perspective transformation is performed. Thereafter, the process returns to the flowchart of FIG.

  On the other hand, when neither the upper line UL nor the lower line DL is a straight line, that is, at least one of them is not a straight line (NO in step S40), the CPU 77 determines that the mesh image 9 (page image 10) has a curved shape. Based on the correction parameter derived from the mesh image 9, a curvature correction process (curve trimming) is performed on the page image 10 (step S48). Thereafter, the process returns to the flowchart of FIG.

  In this way, the positional relationship with respect to the set of control points in the upper line UL and the lower line DL of the mesh image 9 is examined, the shape of the mesh image 9 (page image 10) is determined from the positional relationship, and the mesh By selecting and processing a distortion correction processing algorithm suitable for the shape of the image 9 (page image 10), the processing time can be shortened without complicating the user's operation.

  Next, returning to FIG. 8, the CPU 77 again outputs the processed page image 10 and the mesh image 9 superimposed on the page image 10 to the display unit 71 (step S34). In step S30, the CPU 77 compares the processed mesh image 9 with the page image 10 using image recognition, and meets a predetermined condition (whether the mutual deviation is within an allowable range). If so, it may be determined that the mesh image 9 matches the page image 10 with sufficient accuracy, and the process may automatically proceed to the next process.

  Next, the CPU 77 determines whether to end the processing or to deform the mesh image 9 for the other page images 10 (step S36). The user inputs an instruction to end the process or to edit the mesh image 9 for the other page images 10. If an instruction to edit the mesh image is input to another page image 10 (NO in step S36), the process returns to step S10, and another page image 10 (for example, the next page image 10) is input. For the page image 10) of the page specified by the user or the user, the processing from step S10 onward is repeated. On the other hand, when an instruction to end the process is input (YES in step S36), the process ends.

  The correction process described above is a process for the page image 10 of one page, but basically the same correction process is executed for the first page image 10 and the last page image 10. The CPU 77 derives correction parameters for the intermediate page image 10 based on the shape of the mesh image 9 for the first page image 10 and the shape of the mesh image 9 for the last page image 10. Specifically, first, the amount of movement of each control point 80 from the first page P to the last page P is derived, and the amount of movement is derived by linearly interpolating correction parameters for each page based on the number of pages. . Thereby, the correction process is executed on the first page image 10, the intermediate page image 10, and the last page image 10 based on the correction parameters for the respective pages.

  Further, in the above-described embodiment, the page image 10 is alone described above, for example, when the intermediate page is forgotten to be shot or there is a problem in the shooting state, and the intermediate page is retaken. The correction parameter may be derived by editing the mesh image 9. In this case, even if the first page image 10 or the mesh image 9 having a shape different from that of the last page image 10 is applied to the intermediate page image 10, correction processing according to the shape of the mesh image 9 is performed. Is applied selectively.

  According to the present embodiment described above, the mesh image 9 is expanded / contracted, rotated, moved, or deformed in accordance with the touch operation by the user on the touch panel 75, and based on the shape of the edited mesh image 9. Since one distortion correction process is selected from the plurality of distortion correction processes and the shape of the page image 10 to be processed is corrected according to the selected distortion correction process, the user's convenience is impaired. The distortion correction processing corresponding to the distortion method of the page image 10 can be selectively switched and executed.

  As a result, since the distortion correction processing corresponding to each page image 10 is selectively switched for the page image 10 for one book B, high accuracy is obtained for the page image 10 that requires accuracy. Distortion processing (curvature correction processing), and for page images 10 that do not require much accuracy, distortion processing (rectangular correction processing or trapezoid correction processing) with high processing speed can be used without reducing accuracy. Reduction of processing time can be expected.

  Further, according to the above-described embodiment, the curvature correction process is selected when each of the upper or lower Bezier control points constituting the edited mesh image 9 does not form a straight line. Without sacrificing the convenience of the user, it is possible to switch to and execute the curvature correction processing corresponding to the distortion method of the page image 10.

  Further, according to the above-described embodiment, each of the control points of the upper and lower Bezier curves constituting the edited mesh image 9 forms a straight line, and the upper left and lower left ends of the Bezier curve, and Since the rectangular correction processing is selected when the horizontal positions of the upper and lower right end control points are the same, the rectangle corresponding to the distortion method of the page image 10 without impairing the user's convenience. It can be executed by switching to the correction process.

  Further, according to this embodiment described above, each of the control points of the upper and lower Bezier curves constituting the edited mesh image 9 forms a straight line, and the outer shape of the edited mesh image 9 is trapezoidal. Since the trapezoid correction process is selected in some cases, the trapezoid correction process corresponding to the distortion method of the page image 10 can be switched and executed without impairing the convenience for the user.

  Moreover, according to this embodiment mentioned above, when touch operation is individual operation with respect to any one control point 80 among the several control points 80 on the upper or lower Bezier curve which comprises the mesh image 9. FIG. In addition, by moving one control point 80 designated by the touch operation according to the touch operation, the shape of the mesh image 9 is deformed, and the touch operation is an overall operation for simultaneously operating the plurality of control points 80. In this case, the mesh image 9 can be expanded, contracted, rotated, or moved by moving the plurality of control points 80 in accordance with the touch operation under the same conditions, so that it is easier without sacrificing user convenience. The shape of the mesh image 9 can be matched with the shape of the page image 10 by operation.

  Further, according to the above-described embodiment, the individual operation is a touch operation on the touch panel 75 with one finger, and the entire operation is a touch operation on the touch panel 75 with two or more fingers. Since the simultaneous touch is performed, the shape of the mesh image 9 can be matched with the shape of the page image 10 by an easier operation without impairing the convenience of the user.

  Further, according to the present embodiment described above, one control point 80 touched with one finger is moved based on the moving direction and moving amount when one finger is dragged on the touch panel 75. Thus, the mesh image 9 is deformed, and the mesh image 9 is moved by moving the plurality of control points 80 based on the moving direction and the moving amount when two or more fingers are dragged on the touch panel 75. The shape of the mesh image 9 can be matched with the shape of the page image 10 by an easier operation without impairing the convenience of the user.

As mentioned above, although several embodiment of this invention was described, this invention is not limited to these, The invention described in the claim, and its equal range are included.
Below, the invention described in the claims of the present application is appended.

(Appendix 1)
The invention according to appendix 1 includes a display control unit that superimposes and displays an image to be processed and a mesh image on an input display unit having a display function and a touch input function; Detecting means for performing, editing means for expanding / contracting, rotating, moving, or deforming the mesh image in accordance with the touch operation detected by the detecting means, and the shape of the mesh image after editing by the editing means. A selection unit that selects one of the plurality of distortion correction processes, and a correction unit that corrects the shape of the image to be processed by the distortion correction process selected by the selection unit. This is a page image correction apparatus characterized by the above.

(Appendix 2)
The invention according to attachment 2 is characterized in that the selection means selects the curvature correction processing when each of the control points of the upper or lower Bezier curve constituting the edited mesh image does not form a straight line. The page image correction apparatus according to Supplementary Note 1, wherein the page image correction apparatus is characterized.

(Appendix 3)
According to the third aspect of the present invention, the selection means is configured so that each of the control points of the upper and lower Bezier curves constituting the edited mesh image forms a straight line, and the upper and lower sides of the Bezier curve. The page image correction apparatus according to appendix 1 or 2, wherein the rectangular correction processing is selected when the horizontal positions of the control points at the left end and the upper and lower right ends are the same.

(Appendix 4)
In the invention according to appendix 4, the selection means is such that each of the control points of the upper and lower Bezier curves constituting the edited mesh image is a straight line, and the outer shape of the edited mesh image is The page image correction apparatus according to any one of appendices 1 to 3, wherein a trapezoidal correction process is selected in the case of a trapezoid.

(Appendix 5)
The invention according to appendix 5 is the page image correction device according to appendix 4, wherein the trapezoidal correction processing includes perspective transformation.

(Appendix 6)
The invention according to appendix 6 is that the editing unit is configured such that the touch operation detected by the detecting unit is any one of a plurality of control points on an upper or lower Bezier curve constituting the mesh image. When the control point is an individual operation, the shape of the mesh image is deformed by moving one control point designated by the touch operation according to the touch operation, and is detected by the detection unit. When the touch operation is an overall operation for simultaneously operating the plurality of control points, the mesh image is expanded, contracted, rotated, or moved by moving the plurality of control points according to the touch operation under the same conditions. The page image correction device according to any one of appendices 1 to 5, wherein

(Appendix 7)
In the invention according to appendix 7, the individual operation is a touch operation on the input display unit with one finger, and the overall operation is a touch operation on the input display unit with two or more fingers. The page image correction apparatus according to appendix 6, wherein simultaneous page touch is performed. It is.

(Appendix 8)
In the invention according to attachment 8, the editing unit is touched with the one finger based on a moving direction and a moving amount when the one finger is dragged on the input display unit. The shape of the mesh image is deformed by moving one control point, and the plurality of control points are based on a moving direction and a moving amount when the two or more fingers are dragged on the input display unit. The page image correction apparatus according to appendix 7, wherein the shape of the mesh image is expanded, contracted, rotated, or moved by moving.

(Appendix 9)
The invention according to appendix 9 is a page image correction method for correcting the shape of the image to be processed based on the shape of the mesh image displayed superimposed on the image to be processed. Detecting a touch operation by a user to an input display unit that displays an image and the mesh image; and expanding, contracting, rotating, moving, or deforming the mesh image according to the detected touch operation; Based on the shape of the mesh image that has been stretched, rotated, moved, or deformed, a step of selecting one distortion correction process from among a plurality of distortion correction processes, and the selected distortion correction process, And a step of correcting the shape of the image.

(Appendix 10)
The invention according to appendix 10 includes a computer of a page image correction apparatus that corrects the shape of the image to be processed based on the shape of the mesh image that is displayed superimposed on the image to be processed. Display control means for displaying the image to be processed and the mesh image superimposed on an input display section having an input function, detection means for detecting a touch operation by the user to the input display section, detected by the detection means One of a plurality of distortion correction processes based on the editing means for expanding / contracting, rotating, moving, or deforming the shape of the mesh image according to a touch operation, and the shape of the mesh image after editing by the editing means A selection unit that selects a correction process; a correction unit that corrects the shape of the image to be processed by the distortion correction process selected by the selection unit; Is a program for causing the to function.

1. Document camera system 2. Document camera (imaging means)
3 Page turning device 9 Mesh image 10 Page image 30 Turning device body 321 Rotating body 33 Drive shaft 34 Arm portion 35 Suction portion 37 Drive portion 38 Base portion 4 Personal computer 5 Blower portion 6 Holding stand 61, 62 Holding plate 7 Information processing terminal 8 Stationary table 70 Communication unit 71 Display unit 72 Imaging unit 73 ROM
74 RAM
75 Operation unit (touch panel)
76 Recording medium 77 CPU
80, 80a, 80b Control point B book P page

Claims (10)

Display control means for displaying an image to be processed and a mesh image superimposed on an input display unit having a display function and a touch input function;
Detecting means for detecting a touch operation by the user to the input display unit;
Editing means for expanding, contracting, rotating, moving, or deforming the mesh image in accordance with the touch operation detected by the detecting means;
Selection means for selecting one distortion correction process among a plurality of distortion correction processes based on the shape of the mesh image after editing by the editing means;
Correction means for correcting the shape of the image to be processed by the distortion correction processing selected by the selection means;
A page image correction apparatus comprising: The selection means selects a curvature correction process when each of the upper or lower Bezier curve control points constituting the edited mesh image does not form a straight line;
The page image correction apparatus according to claim 1. The selection means is configured such that each of the upper and lower Bezier curve control points constituting the edited mesh image forms a straight line, and the upper and lower left ends of the Bezier curve, and the upper and lower sides. Select the rectangle correction process when the horizontal position of the rightmost control point is the same,
The page image correction apparatus according to claim 1, wherein the page image correction apparatus is provided. The selection means performs a trapezoidal correction process when each of the control points of the upper and lower Bezier curves constituting the edited mesh image is a straight line and the outer shape of the edited mesh image is a trapezoid. Select
The page image correction device according to any one of claims 1 to 3. The trapezoidal correction process includes perspective transformation,
The page image correction apparatus according to claim 4. The editing means includes
When the touch operation detected by the detection means is an individual operation on any one of the control points on the upper or lower Bezier curve constituting the mesh image, the touch operation The shape of the mesh image is deformed by moving one control point specified by, according to the touch operation,
When the touch operation detected by the detection means is an overall operation for simultaneously operating the plurality of control points, the mesh image is obtained by moving the plurality of control points according to the touch operation under the same conditions. Extend, rotate, or move,
The page image correction apparatus according to any one of claims 1 to 5, wherein In the individual operation, the touch operation on the input display unit is a touch with one finger, and the overall operation is a touch operation on the input display unit with a simultaneous touch with two or more fingers.
The page image correction apparatus according to claim 6. The editing means includes
The shape of the mesh image by moving one control point touched with the one finger based on a moving direction and a moving amount when the one finger is dragged on the input display unit. Transform
The shape of the mesh image is expanded, contracted, rotated, or moved by moving the plurality of control points based on the moving direction and moving amount when the two or more fingers are dragged on the input display unit. To
The page image correction apparatus according to claim 7. A page image correction method for correcting the shape of the processing target image based on the shape of the mesh image displayed superimposed on the processing target image,
Detecting a touch operation by a user to an input display unit that displays the image to be processed and the mesh image;
Stretching, rotating, moving, or deforming the mesh image in accordance with the detected touch operation;
Selecting one distortion correction process among a plurality of distortion correction processes based on the shape of the mesh image that has been stretched, rotated, moved, or deformed;
Correcting the shape of the image to be processed by the selected distortion correction processing;
A page image correction method comprising: A computer of a page image correction apparatus that corrects the shape of the image to be processed based on the shape of the mesh image displayed superimposed on the image to be processed.
Display control means for superimposing and displaying the image to be processed and the mesh image on an input display unit having a display function and a touch input function;
Detecting means for detecting a touch operation by the user to the input display unit;
Editing means for expanding / contracting, rotating, moving, or deforming the mesh image according to the touch operation detected by the detecting means,
Selection means for selecting one distortion correction process among a plurality of distortion correction processes based on the shape of the mesh image after editing by the editing means;
Correction means for correcting the shape of the image to be processed by the distortion correction processing selected by the selection means;
A program characterized by functioning as

Images