JP2006025037A - Image processing system - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a system for generating a still image from a motion image photographed with a portable terminal. <P>SOLUTION: A cellular phone 10 selects a reference frame image F0 and frame images F1-F3 before and after the reference frame image, and transmits the selected images to a server 100. The server 100 executes sharpening processing on the basis of deviation in pixels between the reference frame image F0 and the other frame images F1-F3 to create a high-definition still image 200, attaches a sample indication 210 indicating that the image 200 is a sample, and transmits the image to the cellular phone 10. The cellular phone 10 transmits a print instruction to the server 100 in response to a print instruction from a user. The server 100 transmits the high-definition still image 200 from which the sample indication 210 is removed, to a designated printer PRT. The printer PRT executes printing. In this way, a high-definition still image can be easily acquired from a motion image photographed with the cellular phone 10. <P>COPYRIGHT: (C)2006,JPO&NCIPI

Description

  The present invention relates to an image processing system via a network.

  2. Description of the Related Art In recent years, image processing apparatuses that create high-resolution images from moving images composed of a plurality of images that are continuous in time series are becoming popular. For example, the image processing apparatus creates a high-resolution still image by interpolating pixels based on a pixel shift between a reference image of a plurality of images and images before and after the reference image.

Japanese Patent Laid-Open No. 11-146118 JP 2003-108343 A JP 2003-296060 A Japanese Patent Laid-Open No. 2002-215772

  In recent years, mobile terminals such as mobile phones and PDAs are rapidly spreading. Many of these portable terminals have a camera function, and an increasing number of terminals are capable of shooting not only still images but also moving images. Therefore, there is a demand for a technique for creating a high-resolution still image from a moving image captured by a mobile terminal.

  Image processing for creating a high-resolution still image from a moving image is a very burdensome process. However, since the mobile phone has low arithmetic processing capability from the viewpoint of size reduction and weight reduction, there is a problem that it is difficult to perform image processing for creating a high-resolution still image from a moving image captured by a mobile terminal. there were.

  The present invention has been made in view of such a problem, and an object of the present invention is to provide a technique for acquiring a high-resolution still image even in an image processing apparatus with a simple configuration and low calculation capability.

In order to solve at least a part of the problems described above, the present invention has the following configuration as the first configuration. That is,
In an image processing system in which an image processing apparatus and a terminal are connected via a network,
The terminal
An acquisition unit for acquiring a plurality of images continuous in time series;
A receiving unit for receiving an image after image processing from the image processing device;
A display unit for displaying the plurality of images and the image after the image processing;
A selection unit that selects at least two images from the displayed plurality of images;
A transmission unit that transmits the selected image to the image processing apparatus;
The image processing apparatus includes:
A receiver for receiving the transmitted image;
An image creation unit that creates a high-resolution still image having a resolution higher than the resolution of the image for at least a part of the image using the image;
The gist of the present invention is to include a transmission unit that transmits the high-resolution still image to the terminal.

  If such a configuration is adopted, the terminal can easily display a high-resolution still image based on a plurality of time-series continuous images acquired by the terminal. Therefore, the user can easily confirm a high-resolution still image.

As a second configuration of the present invention,
On the terminal,
An acquisition unit for acquiring a plurality of images continuous in time series;
A receiving unit for receiving an image after image processing from the image processing device;
A display unit for displaying the plurality of images and the image after the image processing;
A selection unit that selects at least two images from the displayed plurality of images;
The gist of the present invention is to include a transmission unit that transmits the selected image to the image processing apparatus.

  In this way, the terminal can flexibly select two or more images from which a high-resolution still image is created from a plurality of images that are continuous in time series, and transmit the selected image to the image processing apparatus.

  The terminal of the present invention may further include an input receiving unit that receives an input of a reference image serving as a reference for the selection, and the selection unit may select the two or more images based on the reference image. . In this way, the user can arbitrarily select an image serving as a reference for creating a high-resolution still image.

  In the terminal of the present invention, the input reception unit receives input of two images that are arbitrarily selected as the reference image, and the selection unit includes an image existing between the two images, and the two images. An image may be selected as the two or more images. In this way, the user can flexibly select the range and number of images used to create a high-resolution still image.

As a second configuration of the present invention,
In the image processing apparatus,
A receiving unit for receiving a plurality of time-series images transmitted from the terminal;
An image creation unit that creates the high-resolution still image having a resolution higher than the resolution of the image for at least a part of the image using the plurality of images;
The gist of the present invention is to include a transmission unit that transmits the high-resolution still image to the terminal.

  In this way, the user can easily acquire a high-resolution still image without performing image processing by himself / herself by transmitting a plurality of images to the image processing apparatus.

  In the image processing apparatus of the present invention, the image creation unit may further add an identification symbol for distinguishing the high-resolution still image and the plurality of images on the high-resolution still image. . In this way, the user who has confirmed the high-resolution still image can easily understand that the high-resolution still image is a sample.

  In the present invention, the various aspects described above can be applied by appropriately combining or omitting some of them. In addition to the configuration as the above-described image processing system, terminal, and image processing apparatus, the present invention provides an image transmission method by the terminal, an image processing method by the image processing apparatus, a computer program for causing the terminal to transmit an image, and an image The present invention can also be configured as a computer program for causing a processing device to execute image processing, a recording medium on which the computer program is recorded so as to be readable by a computer, and the like. In any configuration, the above-described aspects can be appropriately applied. As a computer-readable recording medium, various media such as a flexible disk, a CD-ROM, a DVD-ROM, a magneto-optical disk, an IC card, and a hard disk can be used.

Hereinafter, embodiments of the present invention will be described in the following order based on examples.
A. Example:
A1. System configuration:
A2. Function block
A3. High-resolution still image creation process:
B. Variations:

A. Example:
A1. System configuration:
FIG. 1 is an explanatory diagram illustrating an image processing system 1000 according to an embodiment of the present invention. The image processing system 1000 includes a mobile phone 10, a server 100, and a printer PRT, and is connected via the Internet INT. The mobile phone 10 includes an operation unit 30, a display unit 31, and a CCD camera (not shown) arranged on the back surface.

  The CCD camera provided in the mobile phone 10 has a function of taking still images and moving images. The operation unit 30 is used for making and receiving calls, taking images, and operating applications built in the mobile phone 10. The display unit 31 displays various information in accordance with the operation. The mobile phone 10 corresponds to a “terminal” in the claims of the present invention, and the server 100 corresponds to an “image processing apparatus”. The moving image corresponds to “a plurality of still images continuous in time series” in the claims of the present invention. Hereinafter, in this embodiment, the plurality of still images are referred to as frame images. The image processing system 1000 according to this embodiment operates as follows.

  The mobile phone 10 transmits a part of the plurality of frame images (frame images F0 to F3) constituting the captured moving image to the server 100. The server 100 creates a high-resolution still image using the frame images F0 to F3 and transmits it to the printer PRT. The printer PRT prints the received high resolution still image.

  The printer PRT prints document data, image data, etc. received via the Internet INT. A printer number assigned uniquely is assigned to the printer PRT. The server 100 manages the printer number and the printer IP address in association with each other. As shown in the figure, a sticker 50 having a printer number is affixed to the printer PRT. In this embodiment, the printer number of the printer PRT is “1020”.

  The mobile phone 10 receives a selection input from a user for a reference frame image F0 that serves as a reference for creating a high-resolution still image among frame images constituting a moving image taken by the mobile phone. The reference frame image F0 is hatched in the figure. The mobile phone 10 transmits the reference frame image F0 and the frame images F1 to F3 before and after the reference frame image together to the server 100 via the Internet INT. Since the frame images F1 to F3 are frame images to be compared with the reference frame image, the frame images F1 to F3 are hereinafter referred to as target frame images F1 to F3.

  When the server 100 receives the frame images F0 to F3, the server 100 executes the sharpening process based on the pixel shift between the reference frame image F0 and the other frame images F1 to F3, and creates the high-resolution still image 200. Transmit to the mobile phone 10. At this time, since the server 100 indicates that the high-resolution still image 200 to be transmitted is a sample version, a sample display 210 that is an image in which characters “SAMPLE” are drawn is attached to the center of the image. The sharpening process will be described later.

  The user of the mobile phone 10 confirms the high-resolution still image 200, and when printing is desired, inputs the printer number of the printer PRT that prints the high-resolution still image 200 and issues a print instruction. In this embodiment, the user directly inputs the printer number from the operation unit 30 in the printer number input area displayed on the display unit 31. As an input method, for example, a still image obtained by photographing the sticker 50 on which the printer number is described may be transmitted to the server 100 together with a print instruction.

  The mobile phone 10 transmits the input printer number and print instruction to the server 100. Upon receiving the print instruction, the server 100 identifies the printer PRT based on the printer number, and transmits the high-resolution still image 200 from which the sample display 210 is deleted to the printer PRT. The printer PRT prints a high-resolution still image 200.

A2. Functional Block FIG. 2 is an explanatory diagram illustrating functional blocks of the mobile phone 10 in this embodiment. The mobile phone 10 includes a main control unit 11, a communication unit 12, an input reception unit 13, a photographing processing unit 14, an application 20, and an image storage unit 16. The application 20 includes a frame image selection unit 21, a print instruction unit 22, and a display control unit 23, and is an application downloaded in advance via a network. Each functional block is configured in software and controlled by the main control unit 11. Each functional block may be configured in hardware.

  The communication unit 12 is a so-called network device, and has a function of exchanging information with other devices such as the server 100 via the Internet INT. The input receiving unit 13 has a function of receiving user input via the operation unit 30 and notifying other functional blocks.

  The photographing processing unit 14 has a function of photographing a moving image and a still image via a CCD camera. The imaging processing unit 14 stores the captured moving image and still image in the image storage unit 16.

  The display control unit 23 has a function of displaying, on the display unit 31, operation guidance, processing contents, frame images, and the like from when the application 20 is activated until it is terminated.

  The frame image selection unit 21 receives a selection input of the reference frame image from the user, selects a total of five frame images of the reference frame image F0 and the target frame images F1, F2, and F3, and the frame image F0. -F3 is transmitted to the server 100.

  The print instruction unit 22 transmits a print instruction for the high-resolution still image 200 to the server 100 together with the printer number input by the user.

  FIG. 3 is an explanatory diagram illustrating functional blocks of the server 100 in the present embodiment. The server 100 includes a main control unit 110, a communication unit 120, an area identification processing unit 121, an image processing unit 122, a print processing unit 125, a high-resolution still image storage unit 130, and a printer management information storage unit 140. It is composed of Each functional block is configured by software and controlled by the main control unit 110. Each functional block may be configured by hardware.

  The communication unit 120 is a so-called network device, and has a function of exchanging information with other devices such as the mobile phone 10 and the printer PRT via the Internet INT. For example, the selected frame image 310 is received from the mobile phone 10 or the high-resolution still image 200 is output to the printer PRT.

  The area specification processing unit 121 has a function of specifying a target area to be increased in resolution in the reference frame image F0 when the image processing unit 122 creates the high-resolution still image 200 from the acquired selected frame image 310. In this embodiment, the entire region of the reference frame image F0 is specified as the target region and the overall resolution is set to the high resolution. For example, the reference frame is divided into blocks, and the target frame images F1 to F3 and the reference frame Based on a pixel shift from the image F0, a block area determined to have the highest effect of the sharpening process may be specified as the target area.

  The image processing unit 122 includes a sharpening processing unit 123 and a stamp applying unit 124. The sharpening processing unit 123 creates a high-resolution still image 200 based on the selected frame image 310 transmitted from the mobile phone 10. The image processing unit 122 performs pixel interpolation based on the pixel shift between the reference frame image F0 and the target frame images F1 to F3, generates a high-resolution still image 200, and stores it in the high-resolution still image storage unit 130. To do.

  The stamp attaching unit 124 has a function of pasting a sample display 210 indicating that the high resolution still image 200 is a sample image to the high resolution still image 200.

  The printer management information storage unit 140 stores a printer number uniquely assigned to a plurality of printers connected to the Internet INT, and a printer IP address corresponding to the printer number. As illustrated, for example, the IP address of the printer with the printer number “1020” is “192.168.121.100”.

  The print processing unit 125 transmits the high-resolution still image 200 to the designated printer based on the print instruction from the mobile phone 10. Specifically, based on the printer number included in the print instruction from the mobile phone 10, the printer management information storage unit 140 is referred to specify the IP address of the printer to be printed, and the data is in a format that the printer can interpret. Convert and send print data. In this embodiment, it is assumed that “1020” is designated as the printer number of the printer to be printed in the print instruction.

A3. Printing process:
FIG. 4 is a timing chart for explaining high-resolution still image creation / print processing according to the present embodiment.

  The mobile phone 10 captures a moving image (step Sa10). Next, the cellular phone 10 accepts the user's selection input of the reference frame image, selects the reference frame image F0 and the target frame images F1 to F3 (step Sa11), and transmits them to the server 100 (step Sa12). FIG. 5 schematically shows a process corresponding to step Sa11.

  FIG. 5A is an explanatory diagram illustrating the display unit 31 of the mobile phone 10 in this embodiment. The display unit 31 illustrates a state where the moving image MV is being reproduced. The user presses the selection button 32 when a frame desired to create a high-resolution still image is displayed during playback of the moving image MV. The frame image selection unit 21 detects pressing of the selection button 32 via the input reception unit 13, and selects the frame image displayed at that time as the reference frame image F0.

  FIG. 5B is an explanatory diagram schematically showing frame image selection processing in the present embodiment. As shown in the figure, the moving image MV is composed of a plurality of frame images 300 continuous in time series. The time t changes in the direction of the arrow from the left side to the right side of the figure. When the cellular phone 10 detects the selection of the reference frame image F0, the cellular phone 10 selects the frames before and after the selection as the target frame images F1 to F3.

  The mobile phone 10 transmits to the server 100 a selected frame image 310 that is a combination of the reference frame image F0 and the target frame images F1 to F3.

  Returning to FIG. The server 100 performs a sharpening process based on the received selected frame image 310 and creates a high-resolution still image 200 (step Sa20). 6 to 11 are explanatory diagrams for explaining the sharpening process.

  FIG. 6 is a flowchart for explaining the sharpening process in this embodiment. This process is a process executed by the sharpening processing unit 123 of the server 100 and is started when the selected frame image 310 is received from the mobile phone 10.

  The server 100 acquires the selected frame image 310 from the mobile phone 10 via the network (step S100).

  Next, the server 100 calculates the shift amount of the pixels existing in the corresponding positions in the reference frame image F0 and the target frame images F1 to F3, and corrects the shift based on the shift amount. A correction amount is estimated (step S101).

  Specifically, the server 100 first sequentially calculates a “deviation amount” indicating how much each of the three target frame images F1 to F3 is deviated from the reference frame image F0 for all pixels. . The “displacement” in the present embodiment is not caused by the movement of the photographing object itself or the movement of the installation location of the video camera, but the camera work called so-called pan or the change of the orientation of the video camera like camera shake. Is due only to. That is, it is assumed that all the pixels are shifted by the same amount between different frame images. The deviation is represented by a combination of “translational deviation” indicating horizontal and vertical deviation and “rotational deviation”. When the above-described shift amount is calculated for each pixel for the target frame images F1 to F3, a correction amount for correcting the shift is estimated based on the shift amount. The correction amount is represented by a combination of a “translation correction amount” and a “rotation correction amount” in the same manner as the pixel shift amount. Details of the correction amount estimation process are shown below.

  As shown in FIG. 7, it is assumed that the coordinates (x1, y1) of the reference frame information correspond to the coordinates (x2, y2) of the target frame information. The translation correction amount is (u, v), and the rotation correction amount is δ with the center of the frame image as the origin. Since it is assumed that the focal length is not changed at the time of shooting, the following equation is used as a coordinate conversion equation on the assumption that only translation and rotation are converted.

式（２）

式（１ａ）

式（２ａ）

Formula (1)

Formula (2)

Formula (1a)

Formula (2a)

  Since the time difference between the reference frame image and the target frame image is very small, u, v, and δ are minute amounts. Here, when δ is a minute amount, cos δ≈1 and sin δ≈δ, so the above equation can be replaced as follows.

式（４）

式（３ａ）

式（４ａ）

Formula (3)

Formula (4)

Formula (3a)

Formula (4a)

  Then, u, v, and δ in the equations (3) and (4) (or (3a) and (4a)) are estimated by the method of least squares. The estimation of the correction amount is based on a gradient method (gradient method) that estimates the pixel position in units smaller than one pixel using, for example, the luminance of each pixel between frame information.

  Here, as shown in the upper part of FIG. 8, the luminance of each pixel of the reference frame information is expressed as Z1 (ix, iy), and the luminance of each pixel of the target frame information is expressed as Z2 (ix ′, iy ′). . First, assuming that the coordinates (ix ′, iy ′) of the target frame information are between the coordinates (ix to ix + 1, iy to iy + 1) of the reference frame information, a method of obtaining the coordinates (ix ′, iy ′) by the gradient method. Will be explained.

  As shown in the middle of the figure, assuming that the position in the x-axis direction of the coordinates (ix ′, iy ′) is ix + Δx, and Px = Z1 (ix + 1, iy) −Z1 (ix, iy), Px · Δx = Z2 ( It is sufficient to obtain Δx such that ix ′, iy ′) − Z1 (ix, iy). In practice, Δx is calculated for each pixel, and the average is taken as a whole. Here, if simply expressed as Z1 = Z1 (ix, iy) and Z2 = Z2 (ix ′, iy ′), Δx that satisfies the following expression may be calculated.

Formula (5)

  Further, as shown in the lower part of the figure, if the position in the y-axis direction of the coordinates (ix ′, iy ′) is iy + Δy, and Py = Z1 (ix, iy + 1) −Z1 (ix, iy), then Py · Δy = What is necessary is just to obtain | require (DELTA) y which becomes Z2 (ix ', iy')-Z1 (ix, iy). Here, if expressed simply by Z1 = Z1 (ix, iy) and Z2 = Z2 (ix ′, iy ′), Δy may be calculated such that the following expression is satisfied.

Formula (6)

Therefore, in consideration of both x and y directions, Δx and Δy that minimize S ^{2 in the} following equation may be obtained by the method of least squares.

  The method for obtaining the translation correction amount on the assumption that the frame image is translated in the x-axis direction and the y-axis direction by the gradient method has been described above. Next, the rotation correction amount will be described. As shown in FIG. 9, when the distance from the origin O of the coordinates (x, y) of the reference frame information is r and the rotation angle from the x axis is θ, r and θ can be obtained by the following equations.

式（９）

Formula (8)

Formula (9)

  Here, assuming that the translational deviation is corrected, the origins of the reference frame image and the target frame image are matched, and the target frame image is rotated δ from the coordinates (x, y) to the coordinates (x ′, y ′). Assuming that, the amount of movement in the x-axis direction and the amount of movement in the y-axis direction due to this rotation are obtained by the following equations.

式（１１）

Formula (10)

Formula (11)

  Therefore, Δx and Δy in the above equation (7) are expressed by the following equations using the translation correction amounts u and v and the rotation correction amount δ.

式（１３）

Formula (12)

Formula (13)

  Substituting these into equation (7) gives the following equation:

Formula (14)

That is, when the coordinates of the reference frame information are (ix, iy) and the coordinate values and gradation data (luminance values) of all the pixels of the reference frame information are substituted into the equation (14), S ² is minimized. , V, δ may be obtained by the method of least squares.

  Returning to FIG. Next, the server 100 combines the frame images F0 to F3 using the estimated correction amount (step S102). Specifically, the server 100 corrects the pixel positions of the target frame images F1 to F3 using the correction amount (step S103). The pixel position correction means that the pixel position of each pixel in the target frame images F1 to F3 is moved by the vertical and horizontal correction amounts based on the translation correction amount, and rotated by the rotation correction amount. Indicates replacement with a position. As a result of such correction processing, the reference frame image F0 and the target frame images F1 to F3 coincide at the overlapping portion.

  Next, the server 100 determines the nearest pixel (step S104). FIG. 10 is a schematic diagram for explaining the nearest pixel determination process.

  FIG. 10 is a schematic diagram for explaining the nearest pixel determination process in the present embodiment. This figure is described taking the translation correction amount as an example. This is a process executed by the server 100. This figure enlarges a part of the matched image of the reference frame image F0 and the target frame images F1 to F3, and each of the frame images F0 to F3 and the high-resolution still image 200 obtained from the frame images F0 to F3. Each pixel position is shown. In the figure, black circles represent pixels of the high-resolution still image 200, and white squares represent pixels of the reference frame image F0. In addition, each pixel of the target frame images F1 to F3 corrected in step S102 is represented by different hatching. In the present embodiment, the high-resolution still image 200 is increased in resolution to a pixel density 1.5 times higher than that of the reference frame image F0. Therefore, as illustrated, the distance between the pixels of the high-resolution still image 200 is 2/3 of the distance between the pixels of the reference frame image F0. Among the pixels of the high-resolution still image 200 such as the pixel G (k) (k is an integer representing the pixel position) in the figure, the pixels that are every two pixels in the vertical and horizontal directions are the pixels of the reference frame image F0. The pixel positions will match.

  Description will be made by paying attention to the j-th pixel G (j) of the high-resolution still image 200. Hereinafter, the pixel G (j) is referred to as a target pixel. The server 100 calculates a distance L0 between the target pixel G (j) and a pixel that is closest to the target pixel G (j) in the pixels of the reference frame image F0. Further, the server 100 calculates the distances L1, L2, and L3 with the pixels that are closest to the target pixel G (j) in the pixels of the target frame images F1, F2, and F3 after the correction processing in step S102.

  Next, the server 100 compares the values of the distances L <b> 0 to L <b> 3 and determines a pixel that exists at a distance closest to the target pixel G (j). Hereinafter, such a pixel is referred to as a “nearest neighbor pixel”. As shown in the figure, since the pixel on the target frame F3 having the distance L3 is the pixel closest to the target pixel G (j), the server 100 determines this pixel as the nearest pixel. Hereinafter, it is assumed that the nearest pixel to the target pixel G (j) is the i-th pixel of the target frame F3 and is represented as the nearest pixel g (F3, i).

  The server 100 sequentially executes the above processing for all the pixels of the high-resolution still image 200, and determines the nearest pixel for each pixel.

  Returning to FIG. 6, the description will be continued. Next, the server 100 interpolates pixel data of each pixel in the high resolution still image 200 from pixel data of other pixels (step S104). In this embodiment, the server 100 uses the pixel data of the nearest pixel determined in step S103 and the pixel data in the target frame image to which the nearest pixel belongs. FIG. 11 is an explanatory diagram for explaining pixel interpolation processing.

  FIG. 11 is a schematic diagram for explaining pixel interpolation processing in the present embodiment. In this figure, the process in the case where it is determined in step S103 that the nearest pixel for the target pixel G (j) is the nearest pixel g (F3, i) is illustrated.

  In addition to the nearest pixel g (F3, i), the server 100, together with the nearest pixel g (F3.i), the pixel g (F3, j) and the pixel g (F3.k) surrounding the target pixel G (j). ), Pixel data of the target pixel G (j) is obtained by the bi-linear method using the respective pixel data of the pixel g (F3, l). Not only the bi-linear method but also a method such as the bi-cubic method and the nearest neighbor method may be used.

  The server 100 finishes the sharpening process as described above, and stores the obtained high-resolution still image 200 in the high-resolution still image storage unit 130.

  Returning to FIG. The server 100 attaches a sample display 210 indicating that the high-resolution still image 200 is a sample near the center of the high-resolution still image 200 (step Sa21). Such processing is shown in FIG.

  FIG. 12 is an explanatory diagram illustrating a high-resolution still image 200 after image processing according to the present embodiment. As shown in the figure, the server 100 superimposes the sample display 210 on the high-resolution still image 200 that has been increased in resolution by the sharpening process and synthesizes it. The server 100 is a sample by a method such as changing some of the pixels of the high-resolution still image 200 to become “SAMPLE” of the sample display 210 instead of superimposing the sample display 210 and combining them. May be represented.

Returning to FIG. Next, the server 100 transmits the high-resolution still image 200 with the sample display 210 attached thereto to the mobile phone 10 (step Sa22).

  The user confirms the high resolution still image 200 to which the sample display 210 is pasted. If the user desires to print the high resolution still image 200 from which the sample display 210 is deleted, the user inputs the printer number of the printer PRT, and the server A print instruction is sent to 100.

  The cellular phone 10 receives the input of the printer number from the user, designates the printer to execute printing (step Sa13), and transmits a print instruction to the server 100 together with the printer number (step Sa14). FIG. 13 shows an example of a screen for designating a printer number.

  FIG. 13 is an explanatory diagram showing an example of a screen for performing printer designation in this embodiment. The display content of the display part 31 displayed in step Sa13 of FIG. 4 is shown. The display unit 31 displays a printer number input area 40 and a print button 41. The user operates the operation unit 30 to input a printer number in the printer number input area 40 and presses the print button 41. The mobile phone 10 detects that the print button 41 has been pressed, and transmits a print instruction to the server 100 together with the input printer number.

  Returning to FIG. Upon receiving the print instruction, the server 100 identifies the IP address of the designated printer PRT based on the printer number, and converts the high-resolution still image 200 from which the sample display 210 is deleted into a data format that can be printed by the printer PRT. Transmit (step Sa23).

  The printer PRT prints the data of the received high-resolution still image 200 (step Sa30).

  According to the image processing system 1000 of the embodiment described above, the user can easily acquire the high-resolution still image 200 from the moving image captured by the mobile phone 10. The user can check the high-resolution still image 200 to which the sample display 210 is attached before printing, and can determine whether or not to execute printing.

  In this embodiment, the user designates and prints the printer after confirming the high-resolution still image 200. However, the present invention is not limited to this. For example, if the user desires high-resolution image data itself instead of being printed, the server 100 may send the image from which the sample display 210 is deleted to the mobile phone 10 again. Good. In this way, the mobile phone 10 can easily obtain the high-resolution still image 200 even if it cannot create the high-resolution still image 200 by itself. In such a case, for example, the server 100 may transmit a key for deleting the sample display 210 to the terminal. The server 100 may transmit an identification number that uniquely identifies the high-resolution still image 200 and a password to the terminal. In such a case, the user can achieve this by using the identification number and password to access and use the high-resolution still image 200 stored on the network.

  Further, for example, in the image processing system 1000 of this embodiment, the sample display 210 may be deleted only when the user pays a fee after confirming the high-resolution still image 200 to which the sample display 210 is attached. . For example, charging may be realized by allowing a user to enter a credit card number or to make a contract to be paid together with a mobile phone usage fee.

B. Variations:
Although various embodiments of the present invention have been described above, it is needless to say that the present invention is not limited to these embodiments and can take various configurations without departing from the spirit of the present invention. For example, it is good also as taking the following structures.

B1. Modification 1:
In the embodiment described above, the frame image selection unit 21 of the mobile phone 10 receives a selection input of the reference frame F0 from the user, and selects the target frame images F1 to F3 based on the reference frame F0. Not limited to this, as shown in FIG. 14, a frame image to be transmitted to the server 100 may be selected.

  FIG. 14 is an explanatory diagram schematically illustrating a frame image selection process in the modification. As shown in FIG. 14A, a frame image range that is a reference for creating a high-resolution still image may be selected from the frame images 300. That is, the cellular phone 10 receives selection input of the two frame images F10 and F11 from the user, and selects the frame images F10 and F11 and the frame images F12 to 14 existing between the frame images F10 and F11. It is good also as transmitting to the server 100. In this way, the user can select the first and last frame images to be transferred to the server 100, and convenience is improved.

  In addition, for example, the mobile phone 10 receives a selection input of a first frame image F20 and a last frame image F21 in an arbitrary range from a user, and the frame image and a frame image included therebetween, that is, More frame images than the number necessary for the sharpening process are transmitted to the server 100. The server 100 performs processing using the frame image optimal for the sharpening process from the received frame image. In this way, the server 100 can execute the process without using a frame image inappropriate for the sharpening process, and the resolution can be improved. As such an optimal frame image, for example, luminance information of each frame image is analyzed, and only the frame images that can be determined to have the same or similar shooting conditions by determining the same or similar shooting conditions between the frame images should be used. It is good.

  In addition, for example, the mobile phone 10 transmits more frame images necessary for the sharpening process to the server 100 as described above, and the server 100 creates a plurality of high-resolution still images from the frame images, and It may be transmitted to the telephone 10. The user may be able to arbitrarily select one or more images from the plurality of transmitted high-resolution still images.

  Also, as shown in FIG. 14B, the mobile phone 10 includes a selection frame image 320 composed of frame images F30 to F33 and a selection frame image composed of frame images F40 to F43 among the frame images 300. 330 and the selected frame image 340 including the frame images F50 to F53 may be selected at a time and delivered to the server 100. In this case, the server 100 may create a high-resolution still image for each of the selected frame images 320, 330, and 340 and transmit all of them to the mobile phone 10. In this way, the user can confirm a plurality of high-resolution still images from one moving image. Therefore, for example, the server 100 can be instructed to print only one desired high-resolution still image from a plurality of high-resolution still images, or can be instructed to print all the high-resolution still images. You can also improve convenience.

B2. Modification 2:
In the above-described embodiment, the sharpening process is applied to the entire area of the reference frame F0, but the present invention is not limited to this. As shown in FIG. 15, the sharpening process may be applied to a part of the reference frame F0. FIG. 15 is an explanatory diagram for explaining the region specifying process in the modification. FIG. 15A illustrates a state where the reference frame F0 is divided into 4 × 4 16 blocks. It is assumed that the target frame images F1 to F3 are also divided into blocks like the reference frame image F0. In each block of the target frame images F1 to F3, a ratio of effective pixels used for the sharpening process may be calculated, and a block having a high ratio may be set as a target for the sharpening process. For example, when the block C30 indicated by hatching in the figure is a block having a high effective pixel ratio, as shown in FIG. 15B, only in the area corresponding to the block C30 of the high-resolution still image 200, A sharpening process may be applied. In this way, the effect of the sharpening process can be confirmed by the user without increasing the resolution of the entire area of the reference frame image F0.

B3. Modification 3:
FIG. 16 is an explanatory diagram exemplifying a region where the resolution is increased in the modified example. As shown in FIG. 16A, when a person is photographed in the reference frame image 500, the area around the face of the person may be specified as an area for increasing the resolution. Specifically, for example, based on the distribution of color information of all pixels of the reference frame image 500, an area having color information close to skin color may be specified as a target area for high resolution. When a person is photographed in the reference frame image 500, the person is often an important part of the image. With such a configuration, the important part of the image is efficiently sharpened. Can be applied.

  As shown in FIG. 16B, half of the image may be specified as a target area for high resolution. In this way, it is not necessary to specify in advance which area is to be increased in resolution, so that the amount of calculation can be reduced and the load can be reduced. In addition, the user can clearly understand the effect of the sharpening process by comparing the left and right regions, which is preferable.

  As shown in FIG. 16C, a predetermined area may be specified as a high resolution target area. In this modification, the center area of the image is the target area for higher resolution. Since what is photographed in the center of the image is often an important part of such an image, the present invention can be applied efficiently by adopting such a configuration.

  As shown in each modification of FIG. 16, when the resolution of a partial area of the image is increased, only the area with the increased resolution may be displayed to the user.

It is explanatory drawing which illustrates the image processing system in the Example of this invention. It is explanatory drawing which illustrates the functional block of the mobile telephone in a present Example. It is explanatory drawing which illustrates the functional block of the server in a present Example. 6 is a timing chart for explaining a high-resolution still image creation / printing process according to the present exemplary embodiment. It is explanatory drawing which showed typically the frame image selection process in a present Example. It is a flowchart explaining the sharpening process in a present Example. It is a figure which shows typically a mode that the translation correction amount is estimated by the pattern matching method in a present Example. It is a figure which shows typically a mode that the translation correction amount by the gradient method in a present Example is estimated. It is a figure which shows typically the rotation correction amount of the pixel in a present Example. It is a schematic diagram explaining the nearest pixel determination process in a present Example. It is a schematic diagram explaining the pixel interpolation process in a present Example. It is explanatory drawing which illustrates the high-resolution still image after the image processing in a present Example. It is explanatory drawing which shows the example of a screen of the display part which performs the printer designation | designated in a present Example. It is explanatory drawing explaining the frame selection process in a modification. It is explanatory drawing explaining the area | region identification process in a modification. It is explanatory drawing which illustrates the area | region which performs high resolution in a modification.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1000 ... Image processing system 10 ... Mobile phone 11 ... Main control part 12 ... Communication part 13 ... Input reception part 14 ... Shooting processing part 16 ... Image storage part 20 .. Application 21 ... Frame image selection unit 22 ... Print instruction unit 23 ... Display control unit 30 ... Operation unit 31 ... Display unit 32 ... Select button 40 ... Printer number input area 41 ... Print button 50 ... Sticker 100 ... Server 300 ... Frame image 310 ... Selected frame image 110 ... Main control unit 120 ... Communication unit 121 ... Area identification processing unit 122 ... Image processing unit 123 ... Sharpening processing unit 124 ... Stamp giving unit 125 ... Print processing unit 130 ... High-resolution still image storage unit 140 ... Printer management information storage unit 200 High resolution still image 210 ... Sample display 320 ... Selected frame image 330 ... Selected frame image 340 ... Selected frame Image 500 ... the reference frame image

Claims (11)

An image processing system in which an image processing apparatus and a terminal are connected via a network,
The terminal
An acquisition unit for acquiring a plurality of images continuous in time series;
A receiving unit for receiving an image after image processing from the image processing device;
A display unit for displaying the plurality of images and the image after the image processing;
A selection unit that selects at least two images from the displayed plurality of images;
A transmission unit that transmits the selected image to the image processing apparatus;
The image processing apparatus includes:
A receiver for receiving the transmitted image;
An image creation unit that creates a high-resolution still image having a resolution higher than the resolution of the image for at least a part of the image using the image;
An image processing system comprising: a transmission unit that transmits the high-resolution still image to the terminal. A terminal,
A display for displaying an image;
An acquisition unit for acquiring a plurality of images continuous in time series;
A receiving unit for receiving an image after image processing from the image processing device;
A display unit for displaying the plurality of images and the image after the image processing;
A selection unit that selects at least two images from the displayed plurality of images;
A terminal comprising: a transmission unit that transmits the selected image to the image processing apparatus. The terminal according to claim 2, wherein
Furthermore,
An input receiving unit that receives an input of a reference image serving as a reference for the selection;
The selection unit is a terminal that selects the two or more images based on the reference image. The terminal according to claim 3, wherein
The input receiving unit receives input of two arbitrarily selected images arbitrarily selected as the reference image,
The selection unit is a terminal that selects an image existing between the arbitrary selection images and the arbitrary selection image as the two or more images. An image processing apparatus,
A receiving unit for receiving a plurality of time-series images transmitted from the terminal;
An image creation unit that creates a high-resolution still image having a resolution higher than the resolution of the image for at least a part of the image using the plurality of images;
An image processing apparatus comprising: a transmission unit that transmits the high-resolution still image to the terminal. The image processing apparatus according to claim 5, wherein
The image creation unit further includes:
An image processing apparatus for providing an identification symbol for distinguishing the high-resolution still image and the plurality of images on the high-resolution still image. An image transmission method executed by the terminal,
(A) acquiring a plurality of images consecutive in time series;
(B) receiving an image after image processing from the image processing device;
(C) displaying the plurality of images and the image after the image processing;
(D) selecting at least two or more images from the displayed plurality of images;
(E) An image transmission method comprising: transmitting the selected image to the image processing apparatus. An image generation method executed by an image processing apparatus,
(A) receiving a plurality of time-sequential images transmitted from the terminal;
(B) using the plurality of images, creating a high-resolution still image having a higher resolution than the resolution of the image for at least a portion of the image;
And (c) transmitting the high-resolution still image to the other terminal. A computer program for causing a computer to execute transmission of an image to another device,
A function for acquiring a plurality of continuous images in time series,
A function of receiving an image after image processing from the image processing device;
A function of displaying the plurality of images and the image after the image processing;
A function of selecting at least two images from the displayed plurality of images;
A computer program for causing a computer to execute a function of transmitting the selected image to the image processing apparatus. A computer program for causing a computer to execute image creation,
A function of receiving a plurality of time-sequential images transmitted from other terminals,
A function of using the plurality of images to create a high-resolution still image having a resolution higher than the resolution of the image for at least a part of the image;
A computer program for causing a computer to execute a function of transmitting the high-resolution still image to the other terminal.   The recording medium which recorded the computer program of Claim 9 or Claim 10 so that computer reading was possible.

Images