JP2017151579A - program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a program of a mobile terminal for allowing a user to appropriately specify an area to be subjected to keystone correction, in a captured image generated by a camera.SOLUTION: A terminal program causes a mobile terminal to execute: acquisition processing of acquiring image data generated by a camera having imaged an object (S12: Yes); extension processing (S13) of generating an extended image by adding an additional pixel to an outer edge of the image; first display processing(S14) of displaying the extended image; first receiving processing (S15) of receiving touch operation for specifying a shape of a trapezoidal image in the extended image; correction processing (S17) of applying keystone correction to the trapezoidal image of the specified shape, to generate a rectangle correction image; and output instruction processing (S20/S21) of outputting corrected image data.SELECTED DRAWING: Figure 2

Description

  The present invention relates to a portable terminal program for correcting a keystone of an image generated by a camera.

  Cited Document 1 describes a mobile terminal program for correcting a trapezoid of a captured image generated by a camera. More specifically, the program described in the cited document 1 causes a captured image generated by a camera that captured a subject to be displayed on a display, and allows the user to specify the position of the apex of a substantially rectangular area surrounding the subject in the captured image. Then, the image of the specified substantially rectangular area is trimmed, and the trimmed image is trapezoidally corrected.

JP 2013-218547 A

  However, when the subject is zoomed and photographed, a part of the subject may not be included in the photographed image. If the user tries to designate the position of the vertex of the substantially rectangular area in such a captured image, the position inside the substantially rectangular area may be designated as the vertex from the original position. As a result, when such an image of the substantially rectangular area is trimmed and trapezoidally corrected, a part of the subject included in the captured image may be further lost.

  The present invention has been made in view of the above circumstances, and an object of the present invention is to provide a portable terminal program that allows a user to appropriately specify a region to be trapezoidally corrected in a captured image generated by a camera. Is to provide.

  The program described in the present specification can be read by a portable terminal including a camera, a display, and an input interface having a touch sensor. The program expands the captured image by acquiring additional image data obtained by acquiring the captured image data generated by the camera that captured the object, and an outer edge of the captured image indicated by the captured image data. In the extended process for generating extended image data indicating the extended image, the first display process for displaying the extended image on the display, and the trapezoidal image in the extended image displayed in the first display process. The portable terminal is caused to execute a first reception process for receiving a designated touch operation through the input interface. The trapezoidal image is an image that shows the target object represented in a trapezoid in the extended image by photographing the rectangular target object from an oblique direction. The program includes a correction process for generating corrected image data indicating a rectangular correction image by correcting the trapezoidal image having the shape specified in the first reception process, and an output instruction for outputting the corrected image data. The processing is executed by the portable terminal.

  According to the above configuration, the user can designate a trapezoidal image to be trapezoidally corrected in an extended image in which additional pixels are added to the outer edge of the captured image. That is, even when a part of the subject is not included in the photographed image, the user can appropriately specify the area to be corrected for trapezoid.

FIG. 1 is a schematic diagram of a system 100 according to the present embodiment. FIG. 2 is a flowchart of the print instruction process. FIG. 3 is a flowchart of the trapezoidal expansion process. 4A and 4B are examples of captured images, where FIG. 4A shows an example in which four straight lines are detected, and FIG. 4B shows an example in which three straight lines are detected. 5A and 5B show examples of extended images, where FIG. 5A shows an example in which an additional pixel 85 is added to the captured image 80, and FIG. 5B shows an example in which an additional pixel 95 is added to the captured image 90. 6A and 6B are display examples of the display 53, where FIG. 6A shows a trapezoid designation screen and FIG. 6B shows a preview screen.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings as appropriate. The embodiment described below is merely an example of the present invention, and it is needless to say that the embodiment of the present invention can be changed as appropriate without departing from the gist of the present invention. For example, the execution order of each process described below can be changed as appropriate without departing from the scope of the present invention.

  FIG. 1 is a schematic diagram of a system 100 according to the present embodiment. A system 100 shown in FIG. 1 includes a multifunction machine 10 and a portable terminal 50. Further, the system 100 may include a printer single-function device instead of the multifunction device 10. The multi-function device 10 and the portable terminal 50 are configured to be able to communicate with each other through the communication network 101. Although the specific example of the communication network 101 is not specifically limited, For example, the internet, wired LAN, wireless LAN, or these combination may be sufficient.

  As shown in FIG. 1, the multifunction machine 10 mainly includes a printer 11, a scanner 12, a display 23, an input I / F 24, a communication I / F 25, a CPU 31, a memory 32, and a communication bus 33. Prepare for. Each component constituting the multi-function device 10 is connected to each other through a communication bus 33.

  The printer 11 executes a printing operation for recording an image indicated by the image data on a sheet. As a recording method of the printer 11, a known method such as an ink jet method or an electrophotographic method can be employed. The scanner 12 executes a scanning operation for reading an image recorded on a document and generating image data.

  The display 23 is a liquid crystal display, an organic EL display, or the like, and includes a display surface for displaying various information.

  The input I / F 24 is a user interface that receives an input operation by a user. Specifically, the input I / F 24 includes a button, and outputs various operation signals associated with the pressed button to the CPU 31. Further, the input I / F 24 may include a film-like touch sensor superimposed on the display surface of the display 23. An operation for designating an object displayed on the display surface of the display 23 and an operation for inputting a character string or a numeric string are examples of user operations. The “object” is, for example, a character string, icon, button, link or the like displayed on the display 23.

  The input I / F 24 implemented as a touch sensor outputs position information indicating the position on the display surface touched by the user. Note that “touch” in this specification includes all operations for bringing an input medium into contact with a display surface. In addition, even if the input medium is not touching the display surface, “hover” or “floating touch” that brings the input medium close to a position where the distance to the display surface is very small is based on the concept of “touch” described above. May be included. Furthermore, the input medium may be a user's finger or a touch pen. The user operation of tapping the position of the icon displayed on the display 23 is an example of a specifying operation for specifying the icon.

  The communication I / F 25 is an interface that can communicate with an external device via the communication network 101. That is, the multifunction machine 10 outputs various information to the external device through the communication I / F 25 and receives various information from the external device through the communication I / F 25. Although the specific communication procedure of communication I / F25 is not specifically limited, For example, Wi-Fi (registered trademark of Wi-Fi Alliance) is employable.

  The CPU 31 controls the overall operation of the multifunction machine 10. The CPU 31 acquires and executes various programs, which will be described later, from the memory 32 based on various information output from the input I / F 24, various information acquired from an external device through the communication I / F 25, and the like. The CPU 31 and the memory 32 constitute an example of a control unit.

  The memory 32 has a program storage area 32A and a data storage area 32B. The area 32A stores an OS 34 and a device program 35. The device program 35 may be a single program or an aggregate of a plurality of programs. The area 32B stores data or information necessary for executing the device program 35. The memory 32 is configured by, for example, a RAM, a ROM, an EEPROM, an HDD, a portable storage medium such as a USB memory attached to or detached from the multifunction machine 10, a buffer provided in the CPU 31, or a combination thereof.

  The memory 32 may be a computer readable storage medium. A computer-readable storage medium is a non-transitory medium. Non-transitory media include recording media such as CD-ROM and DVD-ROM in addition to the above examples. A non-transitory medium is also a tangible medium. On the other hand, an electrical signal that carries a program downloaded from a server on the Internet is a computer-readable signal medium that is a kind of computer-readable medium, but a non-transitory computer-readable storage. Not included in the media. The same applies to the memory 62 of the portable terminal 50 described later.

  As shown in FIG. 1, the portable terminal 50 mainly includes a camera 52, a display 53, an input I / F 54, a communication I / F 55, a CPU 61, a memory 62, and a communication bus 63. The display 53, input I / F 54, communication I / F 55, CPU 61, memory 62, and communication bus 63 included in the portable terminal 50 are the display 23, input I / F 24, communication I / F 25, CPU 31 included in the multifunction device 10. Since the configuration is the same as that of the memory 32 and the communication bus 33, description thereof is omitted. The CPU 61 and the memory 62 are an example of a control unit.

  The mobile terminal 50 is, for example, a mobile phone, a smartphone, or a tablet terminal. More specifically, the display 53 of the mobile terminal 50 desirably has a display screen size of 12 inches or less, more preferably 8 inches or less. The input I / F 54 of the portable terminal 50 is preferably a touch sensor superimposed on the display screen of the display 53. In the area 62A of the memory 62, an OS 64 and a terminal program 65 are stored.

  The camera 52 shoots a subject and generates image data (hereinafter sometimes referred to as “captured image data”). The operation of the camera 52 is controlled by, for example, a camera program (not shown) stored in the area 62A. That is, the camera program causes the display 53 to display an image indicated by the captured image data and a [capture] icon. The image displayed on the display 53 is changed following the change in the direction of the camera 52. Then, the camera program stores the captured image data in the area 62B in response to accepting a user operation for designating the [capture] icon through the input I / F 54.

[Operation of System 100]
With reference to FIG.2 and FIG.3, operation | movement of the system 100 which concerns on this embodiment is demonstrated.

  The flowchart in this specification basically shows the processing of the CPUs 31 and 61 in accordance with the instructions described in the program. That is, processes such as “determination”, “extraction”, “selection”, and “control” in the following description represent processes of the CPUs 31 and 61. The processing by the CPU 61 includes hardware control via the OS 64. Further, “data” in this specification is represented by a bit string readable by a computer. Data having substantially the same semantic content but different formats are handled as the same data. The same applies to “information” in this specification.

  First, the terminal program 65 of the portable terminal 50 activates a camera program (not shown) (S11). Next, the camera program stores the photographed image data in the area 62 </ b> B in response to accepting a user operation for designating the [capture] icon through the input I / F 54, and stores the stored photographed image data in the terminal program 65. provide. For example, the terminal program 65 may execute an API for starting the camera program as a so-called plug-in. The camera program may include the path of the captured image data stored in the area 62B in the return value of the API. And the terminal program 65 should just read picked-up image data based on the path | pass included in a return value (S12: Yes).

  In the present embodiment, it is assumed that the user of the mobile terminal 50 has taken a rectangular whiteboard from the oblique direction with the camera 52. As a result, the terminal program 65 acquires the captured image data indicating the captured image 80 illustrated in FIG. 4A or the captured image data indicating the captured image 90 illustrated in FIG. And A whiteboard is an example of an object. However, a specific example of the object is not limited to a white board, and may be a substantially rectangular or square object such as a blackboard, a poster, or a display, and it is desirable that characters or designs are drawn on the surface.

  The captured image 80 is a rectangular image surrounded by contour lines 81A, 81B, 81C, 81D as shown in FIG. The captured image 90 is a rectangular image surrounded by contour lines 91A, 91B, 91C, and 91D, as shown in FIG. 4B. The captured images 80 and 90 include at least a part of the trapezoidal images 84 and 94. The trapezoidal images 84 and 94 are images showing whiteboards that are trapezoidal in the captured images 80 and 90 when the whiteboard is captured from an oblique direction.

  Next, the terminal program 65 executes image expansion processing (S13). The image expansion process is a process for generating an extended image by adding pixels to the captured image indicated by the captured image data. That is, the extended image is an image obtained by extending a captured image. The details of the image expansion process will be described with reference to FIG.

  First, the terminal program 65 detects a straight line longer than a predetermined length included in the photographed image using a known algorithm such as Hough transform (S31). Further, the terminal program 65 determines the number of straight lines detected in S31 (S32). The process of S31 is an example of a detection process, and the process of S32 is an example of a first determination process.

  As an example, the terminal program 65 detects four straight lines 82A, 82B, 82C, and 82D in the captured image 80 shown in FIG. As another example, the terminal program 65 detects three straight lines 92A, 92B, and 92C in the captured image 90 shown in FIG. However, the number of straight lines detected in S31 is not limited to the example of FIG. 4 and may be two or less.

  Next, in response to determining that the three straight lines 92A to 92C have been detected (S32: 3), for example, as shown in FIG. One of 91A to 91D is determined as the fourth straight line (S33). The process of S33 is an example of a determination process. On the other hand, as shown in FIG. 4A, the terminal program 65 skips the process of S33 in response to determining that four straight lines 82A to 82D have been detected (S32: 4).

  For example, in S33, the terminal program 65 determines that two straight lines 92A and 92C that are nearly parallel among the three straight lines 92A to 92C detected in S31 are paired straight lines. The pair of straight lines 92 </ b> A and 92 </ b> C are straight lines that can form opposite sides of the trapezoidal image 94. Then, the terminal program 65 determines the contour line 91D farthest from the straight line 92B having no pair of straight lines out of the four contour lines 91A to 91D of the captured image 90 as the fourth straight line.

  Next, the terminal program 65 specifies the position of the intersection of the four straight lines detected in S31, or the position of the intersection of the three straight lines detected in S31 and one straight line determined in S33. Then, the terminal program 65 estimates the specified position of the intersection as the position of the vertex of the trapezoidal images 84 and 94 (S34). The process of S34 is an example of an estimation process. As an example, the terminal program 65 estimates the position of the intersection of each straight line as the vertices 83A, 83B, 83C, and 83D of the trapezoidal image 84 in the captured image 80 shown in FIG. As another example, the terminal program 65 estimates the position of the intersection of each straight line as the vertices 93A, 93B, 93C, and 93D of the trapezoidal image 94 in the captured image 90 shown in FIG. As shown in FIG. 4, some of the vertices estimated in S <b> 34 may be located outside the captured images 80 and 90.

  Next, the terminal program 65 determines whether or not the positions of the four vertices estimated in S34 are in the captured image (S35). Then, in response to determining that the position of at least one vertex is outside the captured image (S35: No), the terminal program 65 generates extended image data indicating the extended image (S36). On the other hand, the terminal program 65 skips the process of S36 in response to determining that the positions of all the vertices are within the captured image (S35: Yes). The process of S35 is an example of a second determination process, and the process of S36 is an example of an extension process.

  In S36, the terminal program 65 generates extended image data by adding additional pixels to the outer edge of the captured image. As an example, the terminal program 65 generates an extended image 86 in which an additional pixel 85 shown in FIG. 5A is added to the outer edge of the captured image 80 shown in FIG. 4A along the outline 81C. As another example, the terminal program 65 adds an L-shaped additional pixel 95 shown in FIG. 5A along the contour lines 91C and 91D to the outer edge of the captured image 90 shown in FIG. 4B. The extended image 96 is generated. The extended images 86 and 96 include the entire trapezoidal images 84 and 94.

  That is, it is desirable that the additional pixels 85 and 95 are provided along the contour lines 81C and 91C closest to the vertices 83C and 93C located outside the captured images 80 and 90. Further, it is desirable that the additional pixel 95 is provided along the outline 91D determined as a straight line in S33. However, the shapes of the additional pixels 85 and 95 are not limited to the example of FIG. As an example, the additional pixels 85 and 95 may not be provided in the entire area of the contour lines 81C, 91C, and 91D. That is, the extended images 86 and 96 do not have to be rectangular. As another example, the terminal program 65 may generate an extended image 88 to which a frame-type additional pixel 87 surrounding the captured image 80 is added, as shown in FIG.

  Further, the widths of the additional pixels 85 and 95 are appropriately set depending on, for example, the positions of the vertices 83C and 93C located outside the captured images 80 and 90 and the number of straight lines detected in S31. First, the terminal program 65 sets the widths of the additional pixels 85 and 95 so that the vertices 83C and 93C positioned outside the captured images 80 and 90 are positioned inside the contour lines of the extended images 86 and 96. Good. Further, the terminal program 65 may increase the widths of the additional pixels 85 and 95 as the number of straight lines detected in S31 is smaller.

  On the other hand, in response to determining that two or less straight lines are detected in S31 (S32: 2 or less), the terminal program 65 replaces S33 to S36, for example, as shown in FIG. By adding a frame-shaped additional pixel 87 surrounding the captured image 80, extended image data indicating the extended image 88 is generated (S37). The process of S37 is an example of an extension process. In the present embodiment, the following processing will be described on the assumption that a frame-type additional pixel 87 is added to the captured image 80 in S36. When the process of S36 is skipped (S35: Yes), the captured image 80 is used instead of the extended image 88 in the following process.

  Returning to FIG. 2, the terminal program 65 displays the trapezoid designation screen shown in FIG. 6A on the display 53 (S14). The trapezoid designation screen includes an extended image 88, guide images 111, 112, 113, and 114, and a [Correction] icon 115. The guide images 111 to 114 are images indicating the positions of the vertices 83A to 83D estimated in S34, and are displayed so as to be superimposed on the extended image 88. The [Correction] icon 115 corresponds to an instruction to correct a trapezoidal image. Then, the terminal program 65 receives a user operation for the trapezoid designation screen through the input I / F 54 (S15). The process of S14 is an example of a first display process, and the process of S15 is an example of a first reception process.

  Note that the terminal program 65 may cause the extended image 88 in which the non-background pixel value is set to the additional pixel 87 to be displayed on the trapezoid designation screen in S14. For example, the terminal program 65 copies the extended image data generated in S36 and S37, sets a non-background pixel value to the additional pixel 87 included in the copied extended image data, and extends the extended image 88 indicated by the extended image data. May be displayed on the trapezoid designation screen. The non-background pixel value is a pixel value different from the pixel value of the background pixel of the captured image. The pixel value of the background pixel indicates, for example, the mode value of a histogram of pixel values included in the captured image. The terminal program 65 may display the guide images 111 to 114 at predetermined positions on the extended image 88 when two or less straight lines are detected in S31 (S32: 2 or less).

  Next, the terminal program 65 changes the position of the guide image 113 on the trapezoid designation screen, for example, in response to receiving a drag operation on the position of the guide image 113 through the input I / F 54 (S15: guide image) (S15: Guide image). S16 & S14). More specifically, the terminal program 65 changes the position of the guide image 113 following the input medium in response to the input medium touched at the position of the guide image 113 being moved on the display surface. Then, the terminal program 65 determines the position where the input medium is separated, that is, the end position of the drag operation, as a new position of the guide image 113.

  The guide image 113 can be moved to an arbitrary position in the extended image 88 of the trapezoid designation screen, that is, an arbitrary position in the captured image 80 and the additional pixel 87. On the other hand, the guide image 113 cannot be moved outside the extended image 88 on the trapezoid designation screen. The same applies to the drag operation on the guide images 111, 112, and 114. The drag operation is an example of a touch operation that changes the positions of the guide images 111 to 114 and a touch operation that specifies the shape of the trapezoidal image 84. Note that the touch operation for designating the shape of the trapezoidal image is not limited to the drag operation on the guide images 111 to 114. For example, the terminal program 65 may change the position and angle of the side in response to receiving a drag operation for the position of the side constituting the trapezoidal image 84 through the input I / F 54.

  Next, in response to the designation of the [Correction] icon 115 being received through the input I / F 54 (S15: Correction icon), the terminal program 65 displays a trapezoidal image 84 having the positions of the guide images 111 to 114 as vertices. Extracted from the extended image 88. Then, the terminal program 65 corrects the extracted trapezoidal image 84 to trapezoidal correction, and generates corrected image data indicating a rectangular corrected image (S17). The process of S17 is an example of a correction process.

  When at least one of the guide images 111 to 114 is arranged at the position of the additional pixel 87, the terminal program 65 extracts the trapezoidal image 84 including the pixel of the captured image 80 and the additional pixel 87. In addition, the terminal program 65 generates corrected image data by interpolating pixels such that the upper side and the shorter side are parallel to the trapezoidal image 84 having apexes at the positions of the guide images 111 to 114, for example. . The keystone correction is realized by a known algorithm such as projective transformation.

  Next, the terminal program 65 displays the preview screen shown in FIG. 6B on the display 53 (S18). The preview screen includes the corrected image 121 generated in S <b> 17, a “print” icon 122, and a “save” icon 123. The “print” icon 122 corresponds to an instruction for causing the multifunction machine 10 to perform a printing operation on the corrected image data. The [Save] icon 123 corresponds to an instruction to store the corrected image data in the area 62B. Then, the terminal program 65 receives a user operation for the preview screen through the input I / F 54 (S19). The process of S18 is an example of a second display process, and the process of S19 is an example of a second reception process.

  Then, the terminal program 65 transmits the print instruction information to the multi-function device 10 through the communication I / F 55 in response to accepting the designation of the [print] icon 122 through the input I / F 54 (S19: print icon) (S20). ). The print instruction information is information for instructing a printing operation for the corrected image data, and includes the corrected image data. On the other hand, although not shown, the device program 35 of the multifunction machine 10 receives print instruction information from the portable terminal 50 through the communication I / F 25. Then, the apparatus program 35 causes the printer 11 to execute a printing operation according to the received print instruction information.

  On the other hand, the terminal program 65 stores the corrected image data in the area 62B in response to accepting the designation of the [Save] icon 123 through the input I / F 54 (S19: Save icon) (S21). Note that when the corrected image data output in S20 and S21 includes an additional pixel, the terminal program 65 may set the background pixel value of the captured image in the additional pixel. The designation of the [Print] icon 122 or the [Save] icon 123 is an example of an instruction operation. The processes of S20 and S21 are an example of an output instruction process.

[Effects of Embodiment]
According to the above-described embodiment, the user can designate a trapezoidal image to be trapezoidally corrected in an extended image in which additional pixels are added to the outer edge of the captured image. That is, even when a part of the subject is not included in the photographed image, the user can appropriately specify the area to be corrected for trapezoid.

  As in the example of FIG. 5, the captured images 80 and 90 can be displayed relatively large on the trapezoid designation screen by selectively adding the additional pixels 85 and 95 to the necessary portions. On the other hand, as shown in FIG. 6A, by adding a frame-shaped additional pixel 87 to the captured image 80, the centers of the captured image 80 and the extended image 88 coincide with each other. The image 88 can be displayed on the trapezoid designation screen. Further, since the captured images 80 and 90 are expanded only when the vertices 83C and 93C of the trapezoidal images 84 and 94 are located outside the captured images 80 and 90, unnecessary addition of additional pixels is suppressed.

  Further, according to the above embodiment, the positions of the vertices 83A to 83D of the trapezoidal image 84 are estimated, and the guide images 111 to 114 indicating the estimated positions of the vertices 83A to 83D are displayed on the extended image 88, The position of the displayed guide images 111 to 114 is changed by the user as necessary. Thereby, the user operation for designating the trapezoidal image 84 can be simplified. Further, it is possible to output the corrected image data after allowing the user to confirm whether or not an appropriate corrected image 121 has been generated through the preview screen.

  Further, according to the above-described embodiment, by setting a non-background pixel value for the additional pixel 87 on the trapezoid designation screen, the user can distinguish the pixel included in the captured image 80 from the additional pixel 87. Accordingly, it is possible to prompt the user to confirm whether or not the guide image 113 superimposed on the additional pixel 87 is displayed at an appropriate position. On the other hand, by setting the background pixel value to the additional pixel included in the corrected image data, the additional pixel can be made inconspicuous in the corrected image indicated by the output corrected image data.

  Further, in the multi-function device 10 and the portable terminal 50 according to the above-described embodiment, the CPU 31 and 61 execute various programs stored in the areas 32A and 62A of the memories 32 and 62, thereby executing the control unit of the present invention. An example in which each process is realized has been described. However, the configuration of the control unit is not limited to this, and part or all of the configuration may be realized by hardware such as an integrated circuit.

  Furthermore, the present invention may be realized not only as the multifunction device 10 and the portable terminal 50 but also as a program that causes the multifunction device 10 and the portable terminal 50 to execute processing. Then, the program may be provided by being recorded on a non-transitory recording medium. The non-transitory recording medium may include a storage unit mounted on a server that can be connected to the MFP 10 and the portable terminal 50 through the communication network 101 in addition to a CD-ROM, a DVD-ROM, and the like. And the program memorize | stored in the memory | storage part of the server may be distributed through the communication networks 101, such as the internet, as the information or signal which shows the said program.

10: MFP 11 ... Printer 23, 53 ... Display 24, 54 ... Input I / F
25, 55 ... Communication I / F
31, 61 ... CPU
32, 62 ... Memory 35 ... Device program 50 ... Portable terminal 65 ... Terminal program

Claims (10)

A program readable by a mobile terminal comprising a camera, a display, and an input interface having a touch sensor,
The program
An acquisition process for acquiring captured image data generated by the camera that captured the object;
An expansion process for generating extended image data indicating an extended image obtained by extending the captured image by adding an additional pixel to the outer edge of the captured image indicated by the captured image data;
First display processing for displaying the extended image on the display;
In the extended image displayed in the first display process, the mobile terminal is caused to execute a first reception process for receiving a touch operation for designating a shape of a trapezoidal image through the input interface, and the trapezoidal image is rectangular. It is an image showing the object represented in a trapezoid in the extended image by the object of
The program
A correction process for generating corrected image data indicating a rectangular correction image by correcting the trapezoidal image of the shape specified in the first reception process;
A program for causing the portable terminal to execute an output instruction process for outputting the corrected image data. The program
Causing the portable terminal to perform an estimation process for estimating a position of a vertex of the trapezoidal image included in the captured image;
In the first display process, a guide image indicating the position of the vertex estimated in the estimation process is displayed on the display so as to overlap the extended image.
In the first accepting process, it is possible to accept at least a touch operation for changing the position of the guide image displayed superimposed on the additional pixel through the input interface;
The program according to claim 1, wherein, in the correction process, the trapezoidal image having the apex at the position of the guide image changed in the first reception process is corrected.   In the first reception process, the program performs a drag operation on the position of the guide image displayed over the additional pixel, and changes the position of the vertex corresponding to the guide image to the end position of the drag operation. The program according to claim 2, which is accepted as an operation. The program
A detection process for detecting a straight line longer than a predetermined length included in the captured image;
Causing the portable terminal to execute a first determination process for determining the number of straight lines detected in the detection process;
The estimation process when the first determination process determines that four straight lines have been detected, wherein the position of each intersection of the straight lines detected by the detection process is estimated as the position of the vertex of the trapezoidal image. 3. The program according to 3. The program
In response to determining in the first determination process that three straight lines have been detected, the portable terminal is caused to execute a determination process for determining one of the contour lines of the captured image as a fourth straight line,
5. The program according to claim 4, wherein, in the estimation process, a position of each intersection of the straight line detected in the detection process and the straight line determined in the determination process is estimated as a position of a vertex of the trapezoidal image. The program
Causing the portable terminal to execute a second determination process for determining whether or not the positions of all the vertices estimated in the estimation process are within the captured image;
In response to the determination in the second determination process that the positions of all the vertices are within the captured image, the portable terminal is not caused to execute the expansion process, and the captured image is displayed in the display in the first display process. Displayed on the
The extension for adding the additional pixel to the photographed image so that the vertex is included in the extension image in response to the determination in the second determination process that the position of at least one vertex is outside the photographed image The program according to claim 4 or 5, which causes the portable terminal to execute processing.   In response to determining in the first determination process that two or less straight lines have been detected, the estimation process is not performed by the mobile terminal, and the captured image is included in the captured image so as to surround the captured image. The program according to any one of claims 4 to 6, which causes the portable terminal to execute the extension processing for adding an additional pixel.   In the first display process, the program causes the display to display the extended image in which a non-background pixel value is set for the additional pixel, and the non-background pixel value is a pixel value of a background pixel of the captured image. The program according to claim 1, wherein the pixel values are different.   The program outputs the corrected image data in which a background pixel value is set for the additional pixel in the output instruction process, and the background pixel value is a pixel value of a background pixel of the photographed image. 8. The program according to any one of 8. The program
A second display process for displaying the corrected image on the display;
A second receiving process for receiving an instruction operation for instructing output of the corrected image data through the input interface;
The program according to any one of claims 1 to 9, which causes the portable terminal to execute the output instruction process in response to receiving the instruction operation in the second reception process.

Images