JP2004207936A - Image processing apparatus - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an image processing apparatus capable of creating a composite processing image data print by designating a desired moving picture part from moving picture data and utilizing a network. <P>SOLUTION: In this image processing apparatus, a client personal computer 2 cuts the required frames out of a MOTION JPEG moving picture, converts the frames into an MPEG file, compresses the file into an image file, prepares instruction information of an image composition method and print order information, and uploads the obtained image file, the instruction information and the print order information to a moving picture print server 30 connected to the client personal computer 2 via a network. The moving picture print server 30 applies image composition processing to the image file, the instruction information and the print order information to print out the resulting image. The moving picture print server 30 executes the image composition processing with a high added value to obtain a print subjected to image composition processing with a high added value for a user. <P>COPYRIGHT: (C)2004,JPO&NCIPI

Description

[0001]
BACKGROUND OF THE INVENTION
The present invention relates to an image processing apparatus that processes and prints moving image data via a network.
[0002]
[Prior art]
In recent years, not only video cameras but also digital cameras can shoot and record moving images. From the recorded and recorded video, cut out the part you like or want to watch carefully, and perform image composition processing such as multiple exposure and disassembly photography, and still images obtained by combining these images Sometimes you want to print an image.
[0003]
Since still images subjected to these image synthesis processes can obtain prints rich in visual effects, there is a strong demand for obtaining high-quality synthesized prints. In order to obtain high-quality prints, it is advantageous to use a photo print service as described in Patent Document 1 below, rather than a home printer. However, a moving image and a composition method are designated and recorded in a photo shop or print service shop. A method of ordering prints by bringing media was used.
[0004]
Patent Document 2 below proposes an image processing method that obtains the same effect as multiple exposure possible with a silver halide camera or the like when capturing a moving image with a digital camera using a CCD image sensor. It discloses a method that can be implemented in a local environment such as a personal computer.
[0005]
The following non-patent document 1 discloses a system (referred to as “FDi net service”) provided by the present applicant and capable of ordering prints using a network. When placing a print order with this system, the user designates an image that he / she wants to print on his / her personal computer, selects a photo shop that receives the completed print, and transmits it to the “FDi Net Service” server via the network. The user goes to the photo store to receive the completed print. However, this system can specify the print size and the number of prints when ordering online, but it is possible to obtain a composite still image print such as multiple exposure and disassembled photo in the form desired by the user based on the normal moving image original image. could not.
[0006]
[Patent Document 1]
JP 2001-319216 A
[0007]
[Patent Document 2]
JP 2001-28726 A
[0008]
[Non-Patent Document 1]
"FDinet Service" homepage <URL: http: //www.fdinet.fujifilm.co.jp/>
[0009]
[Problems to be solved by the invention]
In consideration of the above circumstances, the present invention provides an image processing apparatus that can specify a desired moving image portion from moving image data and create a combined image data or a combined image data print using a network. With the goal.
[0010]
An object of the present invention is to provide an image processing apparatus capable of uploading a moving image from a client-side terminal via a network and creating composite processed image data or a composite processed image data print.
[0011]
The present invention displays a moving image in a server placed on the network on a client-side terminal, designates a composition method, transmits composition method designation data to the server, and transmits desired composition processing image data or composition processing. An object of the present invention is to provide an image processing apparatus capable of creating an image data print.
[0012]
[Means for Solving the Problems]
The present invention of claim 1 is an image processing apparatus that processes moving image data and is connected to a server via a network,
Reading means to read video data,
A designation unit for designating information on a portion to be synthesized and an image synthesis method information from the movie file of the movie data read by the reading unit;
A conversion unit that cuts out the portion of the designated image to be combined from the video file and converts it into an upload file;
The image processing apparatus includes an upload unit that uploads the file converted into the upload file and the instruction information on the image composition method to the server.
[0013]
The present invention of claim 1 is an image processing apparatus on the client side, wherein the designation means designates a portion to be synthesized from a moving image file of the moving image data read by the reading means and information on the image composition method, and the conversion The specified portion to be combined with the image is cut out from the moving image file and converted into an upload file, and the file converted into the upload file by the upload unit and the instruction information of the image combining method are uploaded to the server.
[0014]
The present invention of claim 2 is an image processing apparatus having a server function connected to a client side image processing apparatus via a network,
Receiving means for receiving an upload file transmitted from the image processing apparatus on the client side and image synthesis method instruction information for the file;
Generating means for generating a still image composite image from the received upload file and the instruction information of the image composition method;
And an image processing apparatus including transmission means for transmitting the generated still image composite image to a client-side image processing apparatus.
[0015]
The present invention of claim 2 is an image processing apparatus on the server side, wherein the receiving unit receives the upload file transmitted from the client side image processing apparatus and the instruction information of the image composition method for the file, and generates the file A still image synthesized image is generated from the received upload file and image synthesis method instruction information by the means, and the generated still image synthesized image is transmitted to the client-side image processing apparatus by the transmitting means. Thereafter, the client-side image processing apparatus determines whether the still image composite image is good or bad by displaying and displaying the transmitted still image composite image.
[0016]
In the image processing apparatus according to claim 1, the conversion unit may use a method of converting into an image recording format file based on a frame difference as a method of converting into an upload file.
[0017]
Further, in the image processing device according to any one of claims 1 to 3, the image composition method may be a method of combining and synthesizing a subject of interest.
[0018]
The image processing apparatus according to any one of claims 1 to 4, wherein the image composition method secures position information of a subject of interest at the time of composition and locates the subject of interest based on the position information. Can be included.
[0019]
The present invention of claim 6 is an image processing apparatus having a server function connected to a client side image processing apparatus via a network,
Storage means for storing video data;
Clipping means for cutting out a designated frame designated by a client-side image processing device from a moving image file of moving image data stored in the storage means;
Receiving means for receiving information on a designated frame designated by the image processing apparatus on the client side and instruction information on an image synthesis method for synthesizing the image of the designated frame;
Synthesizing means for synthesizing a still image from the designated frame clipped by the clipping means and the instruction information of the image synthesizing method;
The image processing apparatus comprises transmission / reception means for transmitting / receiving data of moving image data, still image data, and print order data of a synthesized still image.
[0020]
According to the present invention of claim 6, the receiving means receives information of a designated frame designated by a client-side image processing apparatus, and the designated frame is extracted from a moving picture file of moving picture data stored in the storing means. Cut out by the cutout means, the receiving means receives the instruction information of the image synthesis method for synthesizing the image of the designated frame, and the synthesis means is stationary from the designated frame cut out by the cutout means and the instruction information of the image synthesis method Synthesize the image.
[0021]
The present invention of claim 7 is an image processing apparatus that processes moving image data and is connected to a server via a network,
Receiving means for receiving video data from the server;
Playback means for playing back the moving image data received by the receiving means;
Generating means for generating designation frame information for designating a frame to be cut out from the movie file of the movie data and instruction information for instructing an image synthesis method for synthesizing an image of the designated frame;
The image processing apparatus comprises: transmission means for transmitting the designated frame information and instruction information and print order data of the synthesized image to the server.
[0022]
According to the seventh aspect of the present invention, the specified frame information for receiving the moving image data from the server by the receiving means, reproducing the moving image data by the reproducing means, and specifying the frame to be cut out from the moving image file of the moving image data by the generating means. Then, instruction information for instructing an image composition method for compositing the image of the designated frame is generated, and the designated frame information and the instruction information and print order data of the image to be synthesized by the server are transmitted to the server by the transmission means.
[0023]
7. The image processing apparatus according to claim 6, further comprising: a data receiving unit that receives the still image combined with the combining unit and print order data; and a printing unit that prints the still image received by the data receiving unit. Can have.
[0024]
The image processing apparatus according to claim 6, wherein the synthesizing unit can use an image recording format synthesizing method based on a frame difference.
[0025]
Further, in the image processing apparatus according to claim 6 or 7, the image composition method may be a method of combining and synthesizing a subject of interest.
[0026]
The image processing apparatus according to claim 6, wherein the image composition method can include a method of securing position information of a subject of interest at the time of composition and specifying the subject of interest based on the position information. .
[0027]
DETAILED DESCRIPTION OF THE INVENTION
Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings.
[0028]
FIG. 1 is a diagram showing a schematic overall configuration of a moving image print ordering system 1 according to the first embodiment.
[0029]
A user wants to print a composite of a moving image captured by a digital camera in a desired state. Therefore, the moving image to be combined is uploaded to the moving image / print server 30, the uploaded image is combined by the moving image / print server 30, printed, and the user goes to the photo store.
[0030]
More specifically, the client personal computer 2 (that is, the user personal computer) and the moving image / print server 30 are connected via the Internet. A digital camera 4 is connected to the client personal computer 2 via a USB cable 6. What is printed by the moving image / print server 30 is sent to the photo shop via the transport route.
[0031]
The captured video data of the digital camera 4 is transmitted to the client personal computer 2 via the USB cable 6, and the part of the video data read from the video data read by the client personal computer 2 and the information of the image composition method are designated by the keyboard or the mouse. Then, the client PC 2 cuts out the designated portion to be combined from the moving image file, converts it into an upload file, and uploads the upload file and the image combining method instruction information to the moving image / print server 30.
[0032]
The video / print server 30 receives the upload file and the image synthesis method instruction information transmitted from the client personal computer 2, generates a still image composite image, and connects the generated still image composite image to the client personal computer 2 via the Internet network. Send through. Thereafter, the client personal computer 2 determines the state of the still image composite image by displaying the transmitted still image composite image and viewing it. If it is OK, the formal order information is transmitted to the video / print server 30, and the video is displayed. The print server 30 prints a still image composite image.
[0033]
FIGS. 2A to 2C are diagrams showing still image composite images when a predetermined range of a moving image is cut out and the cut out frames are combined in the form of a decomposed photo, multiple exposure, and overlay composition, respectively. It is. The disassembled photograph is, for example, a method in which a moving image obtained by shooting a golf swing is divided into still images of instantaneous moments over time and displayed in a list in order. The multiple exposure is to display the same subject photographed under various conditions in one frame. Overlay composition is to superimpose frames shot separately and display them in one frame. The user ultimately wants to obtain such a still image composite image print.
[0034]
FIG. 3 is a block diagram showing a hardware configuration of the client personal computer 2. As shown in FIG. 3, the client personal computer 2 includes a central processing unit (CPU) 10 that mainly controls the operation of each component, and a main memory 12 that stores a control program for the device and serves as a work area when the program is executed. An operating system (OS) of the client personal computer 2, various application software including device drivers of peripheral devices connected to the client personal computer 2, a hard disk device 14 storing user images, and the like, and a CD-ROM drive 16 A display memory 18 for temporarily storing display data, a monitor device 20 such as a CRT monitor or a liquid crystal monitor for displaying images, characters and the like by image data and character data from the display memory 18, a keyboard 22, and a position Mouse 24 as input device and detection of mouse 24 status The mouse controller 26 that outputs signals such as the position of the mouse pointer on the monitor device 20 and the state of the mouse 24 to the CPU 10 and a USB (Universal Serial Bus) cable that can be connected to the digital camera 4 and input image data and the like. 6, an interface 27 that connects to the Internet 6, a communication interface 8 that connects to the Internet, and a bus 28 that connects the above components.
[0035]
In the personal computer 2 having the above-described configuration, each component is well-known, and a detailed description of each component is omitted.
[0036]
FIG. 4 is a block diagram showing a hardware configuration of the moving image / print server 30. As shown in FIG. 4, the moving image / print server 30 includes a hard disk device 32, a CPU 34, a working memory 36, a moving image processing unit 38, a printing unit 40, a communication interface 42 for connecting to the Internet network, a still image storage memory 44, A still image combining unit 46 and a combined still image sending unit 48 are connected via a bus 50.
[0037]
The hard disk device 32 stores various application software including an image composition program such as a decomposed photograph, multiple exposure, and overlay composition according to the present invention. The work memory 36 is a memory for temporarily storing data and performing various work. The moving image processing unit 38 expands the transmitted moving image data such as MPEG and performs various image processing. The print unit 40 prints the synthesized still image. The still image storage memory 44 stores still image data processed by the moving image processing unit 38 and decomposed into still images for each frame. The still image synthesizing unit 46 performs image synthesis such as decomposition photography, multiple exposure, and superposition synthesis by an image synthesis program stored in the hard disk device 32. The synthesized still image sending unit 48 generates sending data for sending the synthesized still image to the client personal computer 2 or the like via the communication interface 42.
[0038]
The operation of the first embodiment will be described. A description will be given from the case where the user actually operates the client personal computer 2.
[0039]
FIG. 5 is a flowchart showing a flow in which the user designates information about a portion to be combined and an image combining method from a moving image file of moving image data stored in the hard disk device 14.
[0040]
First, a target video file is read (step 100). At this point, the moving image file is in the MOTION JPEG format. The read moving image file is displayed on the monitor 20 (step 101). The monitor 20 displays a frame designation screen 92. The frame designation screen 92 is a film roll display, and the moving image frame to be noticed is slid by the left and right arrow buttons (step 102), and the corresponding moving image frame is displayed largely on the upper screen. In the lower part, two frames are displayed in a reduced manner, including the large frame.
[0041]
To specify the frame of the section to be combined, specify the start frame with the IN button (step 104), slide the moving picture frame again with the arrow button (step 106), and specify the end frame with the OUT button (step). 108).
[0042]
Thereafter, a synthesis method is designated (step 110). There is a compositing method button on the upper right side of the frame designation screen 92, and when this button is clicked, three kinds of compositing methods are displayed: disassembled photo, multiple exposure, and subject addition. The user clicks and selects a desired synthesis method from the three types of synthesis methods.
[0043]
Then, order information is designated (step 112). There is an order button in the center on the right side of the frame designation screen 92. When this button is clicked, a print order data print order data input screen 94 is displayed. On the print order data input screen 94, the number of prints, the address of the orderer, the name, the telephone number, and the name of the store that receives the print are entered. Alternatively, on the print order data input screen 94, the credit card number, the number of prints, the orderer's address, name, and telephone number may be input. In this case, the orderer receives the completed print by mail.
[0044]
After input, confirm the displayed contents and click the OK button if OK, or click the CANSEL button if you do not place an order.
[0045]
When the frame section to be combined is determined through step 108, the frame of the specified section is cut out from the moving image file and temporarily saved as a MOTION JPEG file (step 114).
[0046]
Thereafter, the MOTION JPEG file is converted into an MPEG file and stored (step 116). This is because when the image data cut out to the moving image / print server 30 is uploaded, the amount of image data is smaller in the MPEG format, and the communication traffic can be reduced.
[0047]
The inputted synthesis method information and order information are stored linked to the MPEG file.
[0048]
FIG. 6 is a flowchart showing a flow when an image file, an image composition method, and order information are uploaded to the moving image / print server 30.
[0049]
First, an MPEG image file to be uploaded is designated (step 118). Then, the designated MPEG file, image composition method information, and order information are uploaded to the moving image / print server 30 (step 120).
[0050]
If it is desired to synthesize continuous shot images, the images are synthesized by the method shown in FIGS. FIG. 7 is a diagram showing a list of continuously shot images displayed on the client personal computer 2, and FIG. 8 is a flowchart showing a flow from designation of the continuously shot images to uploading to the moving image / print server 30.
[0051]
Since the continuous shot image is not a moving image, the recording method is also performed in the client personal computer 2 by the JPEG method.
[0052]
As shown in FIG. 7, the continuous shot image to be combined is designated by the browser (step 122). Then, the designated continuous shot image is confirmed by the composite print button of the browser (step 124). At this point, the designated continuous shot image is saved in the JPEG format. Thereafter, JPEG continuous shot image data is uploaded to the moving image / print server 30 together with composition method information and order information separately input and stored (step 126).
[0053]
Next, the operation of the moving image / print server 30 when image data, composition method information, and order information are uploaded from the client personal computer 2 to the moving image / print server 30 will be described.
[0054]
FIG. 9 is a flowchart showing a flow from when the image data or the like is uploaded until the completed print is sent to the user or the photo shop.
[0055]
The moving image / print server 30 receives the image data in the MPEG format, the synthesis method information, and the order information uploaded via the Internet network (step 136). After the reception, the designated frame of the image data in the MPEG file format is decompressed by the moving image processing unit 38 and stored in the still image storage memory 44 as a still image. From the received compositing method information, the corresponding image compositing program of the disassembled photo, multiple exposure, and overlay compositing stored in the hard disk device 32 is activated, and the still image stored in the still image storage memory 44 is The image composition unit 46 performs composition processing (step 138). The synthesized still image is converted into data for transmission by the synthesized still image transmission unit 48 and transmitted to the client personal computer 2 from the communication interface 42 via the Internet network. If the transmitted composite still image is OK, the user transmits a message to that effect to the moving image / print server 30.
[0056]
When the moving image / print server 30 receives the “OK” information from the client personal computer 2, the print unit 40 prints out the composite still image for the number of prints included in the previously received order information (step 140). .
[0057]
It is determined whether or not the received order information includes the information of the receiving store (step 142). When the receiving store is designated, the printed output is delivered to the receiving store (step 144). When the store is not designated, the mail processing is performed (step 146), and the credit card number is inputted, so that the credit is charged (step 148).
[0058]
After step 144 or 148, the client personal computer 2 is notified by e-mail that the print has been sent to the orderer (step 150).
[0059]
A second embodiment will be described.
[0060]
FIG. 10 is a diagram showing an overall configuration of a moving image print ordering system 52 according to the second embodiment.
[0061]
The user wants to download the moving image stored in the image server 54 to his or her personal computer and print it in a desired state. Therefore, the instruction information of the image composition method is transmitted to the image server 54, the images are synthesized by the image server 54 by the image composition method, and the synthesized still image is transmitted to the lab server 56 and printed by the lab server 56. Is transported to the photo shop and the user goes to receive the print of the composite still image completed in the photo shop.
[0062]
More specifically, the client personal computer 2 (that is, the user personal computer), the image server 54, and the lab server 56 are connected through the Internet.
[0063]
An image composition method in which a user downloads moving image data stored in the image server 54 to the client personal computer 2, reproduces and designates a frame to be extracted from the moving image file of the moving image data, and an image composition method for composing the image of the designated frame. The instructing combination instruction information is generated, and the designated frame information, the combination instruction information, and the print order data are transmitted to the image server 54. The image server 54 cuts out the specified frame from the moving image file based on the received specified frame information, combines the specified frame cut out based on the combination instruction information, and sends the combined still image and the order information received from the client personal computer 2 to the lab server 56. Send.
[0064]
The lab server 56 prints the composite still image transmitted from the image server 54 in accordance with the order information.
[0065]
As an image composition method, as in the first embodiment, a form of separation photograph, multiple exposure, and overlay composition can be considered.
[0066]
The configuration and operation of the client personal computer 2 are the same as those in the first embodiment.
[0067]
The configuration of the image server 54 and the lab server 56 shown in FIG. 10 will be described.
[0068]
The image server 54 includes a moving image storage memory 58 in which moving image data is stored, a hard disk device 60, a moving image cutout unit 62 that cuts out a specified frame of a moving image file based on specified frame information, and a still image storage memory that stores the cut out still image 68, a still image combining unit 64 that combines the still images stored in the still image storage memory 68 based on the combination instruction information, and a combined still image that generates transmission data for transmitting the combined still image to the lab server 56. Sending unit 72, working memory 71, moving image compression unit 70 for compressing moving image data into MPEG1 format when downloading moving image data requested from client computer 2 to client computer 2, and for communicating with the Internet C for controlling each department in the communication interface 74 and the image server 54 Consists U66, each of these departments are connected via a bus 73.
[0069]
The hard disk device 60 stores various application software including an image composition program such as a decomposed photograph, multiple exposure, and overlay composition according to the present invention. The work memory 71 is a memory for temporarily storing data and performing various works. The still image synthesizing unit 64 performs image synthesis such as a decomposed photograph, multiple exposure, and overlay synthesis on the designated still image based on the synthesis instruction information by an image synthesis program.
[0070]
The lab server 56 communicates with a hard disk device 82, a composite still image storage memory 80 that stores the composite still image transmitted from the image server 54, a photo printer subsystem 84 that is a system for printing the composite still image, and the Internet network. And a CPU 78 for controlling each department in the lab server 56, which are connected via a bus 83.
[0071]
The operation of the second embodiment will be described.
[0072]
A description will be given from the case where the user actually operates the client personal computer 2.
[0073]
FIG. 11 shows a flow from when the user downloads the moving image data stored in the image server 54 to the client personal computer 2 and designates and transmits the frame, image combining method, and order information of the moving image file. It is a flowchart.
[0074]
First, the desired moving image data is downloaded from the image server 54 and reproduced (step 152). At this point, the moving image file is in the MPEG format. The reproduced moving image file of the moving image data is displayed on the monitor 20, and the user designates a moving image frame to be combined, a combining method, and order information while viewing the display on the monitor 20 (step 154). Since the method for specifying the moving image frame, the method for specifying the image composition method, and the method for specifying the order information are the same as those in the first embodiment, description thereof will be omitted. As in the first embodiment, the image composition method can be selected from three types of composition methods: decomposed photograph, multiple exposure, and subject addition.
[0075]
In the ordering method, it is also possible to input the credit card number, the number of prints, the orderer's address, name, and telephone number on the print order data input screen 94 and receive the completed print by mail. The point is the same as in the first embodiment.
[0076]
After inputting the moving picture frame designation information, the image composition method designation information, and the order information, confirm the displayed contents and click the OK button if OK, or click the CANSEL button if not ordering.
[0077]
The moving image frame designation information, the image composition method designation information, and the order information that have been input and pressed the OK button are transmitted to the image server 54 (step 156).
[0078]
Note that a continuous shot image can also be selected as an image synthesis target. In this case, the client personal computer 2 downloads the continuous shot image stored in the image server 54 and transmits the frame designation information, the image synthesis method designation information, and the order information to the image server 54 as described above.
[0079]
Next, the operation of the image server 54 will be described.
[0080]
FIG. 12 shows the flow of the operation of the image server 54 when downloading moving image data to the client personal computer 2, and moving image frame designation information, image composition method designation information, and order information are transmitted from the client personal computer 2 to the image server 54. 5 is a flowchart showing a flow of operation of the image server 54 from
[0081]
First, it is determined whether the user request from the client personal computer 2 is a moving image download or an image composition instruction (step 158).
[0082]
In the case of downloading a moving image, the moving image file designated by the user is compressed into the MPEG format (step 160). The compressed designated moving image file is transmitted to the client personal computer 2 (step 162).
[0083]
In the case of an image synthesis instruction, the frame designation information, the image synthesis method designation information, and the print order information transmitted from the client personal computer 2 are received (step 164). Based on the frame designation information, the moving image cutout unit 62 cuts out the designated frame of the moving image file (step 165).
[0084]
The image synthesis method designation information is discriminated for the cut out frame (step 166), and any one of the decomposed photo composition (step 168), multiple exposure composition (step 170), and subject overlay composition (step 172) is performed. . These image synthesizing is performed by the still image synthesizing unit 64, and an image synthesizing program is used to synthesize the specified still image frame based on the synthesizing instruction information, such as a decomposed photograph, multiple exposure, and superimposing synthesis.
[0085]
The received print order information is linked to the combined image data, and the combined image data and the print order information are transmitted to the lab server 56 (step 174).
[0086]
The lab server 56 receives the composite image data and print order information transmitted from the image server 54. The composite still image that is the content of the composite image data is temporarily stored in the composite still image storage memory 80. The composite still image is printed out by the photo printer subsystem 84 in accordance with the print order information.
[0087]
The print output by the lab server 56 is conveyed to a receiving photo store designated by the user. The user goes to the reception photo store to receive the printed composite still image. As in the first embodiment, it is also possible to deliver to the user what has been printed out by mail without designating a receiving store. In this case, credit is charged at the time of mailing.
[0088]
When preparations for sending the print output to the receiving photo shop or by mail are completed, the client PC 2 is notified by e-mail that the print has been sent to the orderer.
[0089]
Of the above-described image composition methods, an image composition method on a subject common to the first and second embodiments will be described in detail. Although there are some differences from the above-described flow of adding the subject, such as a frame designation method, the following method will be described as an example.
[0090]
FIG. 13 is a flowchart showing a flow of an image composition method for adding a subject.
[0091]
When the image composition processing program for adding a subject is started, the monitor device 20 of the client personal computer 2 displays a screen for selecting a still image file as a background image or a moving image file for image composition (not shown). A desired still image file and moving image file are selected on this screen (steps 176 and 178).
[0092]
Subsequently, n still images to be processed are selected from the moving image file selected in step 178, and the selected n still images are cut out from the moving image file and re-saved as continuous still images (step 180). 182). For n still images, an arbitrary numerical value n may be input, and n still images may be automatically selected from the movie file at equal intervals, or manually while viewing each frame image of the movie. You may make it select one by one. Alternatively, a certain period in the moving image may be designated and all the frame images within the period may be selected.
[0093]
Next, of the n still images, the still image number i to be processed is set to 1 (i = 1) (step 184). Note that n still images are numbered 1 to n in chronological order.
[0094]
A main subject is extracted from the i-th still image to be processed (step 186). The main subject is extracted based on the background image selected in step 176. That is, the color space distance between pixels at the same position of the i-th still image and the background image (the sum of the difference values of R, G, and B data of each pixel) is calculated, and pixels whose calculation results are equal to or greater than a specified value are calculated. Extracted as main subject candidate pixels.
[0095]
The main subject candidate pixel group extracted in this manner is subjected to image processing such as concatenation, and noise pixels are removed to obtain an image of only the main subject. The image of only the main subject is saved in the main memory 12 with the same pixel size as the original still image. That is, the area other than the main subject of the original still image is filled with pixels having no pixel value (for example, R, G, B = (-1, -1, -1)) and is the same as the original still image. Set to pixel size.
[0096]
Next, the image having the pixel value of only the main subject extracted in step 186 is added and mapped on the synthesized image memory area for one frame of the main memory 12 to perform the addition of the main subject (step 188). That is, after mapping so that the pixels of the same coordinates (x, y) overlap each other in the consecutive i-th image i and i + 1-th image (i + 1), the pixel values of the pixels of both images overlap. The added pixel value is determined so that the pixel value of (i + 1) is adopted. In the case of an image of only the main subject extracted from the first still image, it is mapped on the memory as it is.
[0097]
The above overlay synthesis algorithm is shown below.
[0098]
Each pixel of the image i and the image (i + 1) is scanned, and each pixel value (Rcomp, Gcomp, Bcomp) of the composite image comp after addition is determined. For convenience, pixel values other than the main subject pixel are defined as (R, G, B) = (− 1, −1, −1).
[0099]
When both the pixel of the image i and the pixel at the same position of the image (i + 1) have the pixel value of the main subject, the pixel value of the image (i + 1) is adopted. That is,
Pixels with (Ri, Gi, Bi) ≠ (-1, -1, -1) and (Ri + 1, Gi + 1, Bi + 1) ≠ (-1, -1, -1)
→ (Rcomp, Gcomp, Bcomp) = (Ri + 1, Gi + 1, Bi + 1)
When the pixel of the image i does not have the pixel value of the main subject and the pixel at the same position in the image (i + 1) has the pixel value of the main subject, the pixel value of the image (i + 1) is adopted. That is,
(Ri, Gi, Bi) = (-1, -1, -1) and (Ri + 1, Gi + 1, Bi + 1) ≠ (-1, -1, -1)
Pixels
→ (Rcomp, Gcomp, Bcomp) = (Ri + 1, Gi + 1, Bi + 1)
When the pixel of the image i has the pixel value of the main subject and the pixel at the same position in the image (i + 1) does not have the pixel value of the main subject, the pixel value of the image i is adopted. That is,
(Ri, Gi, Bi) ≠ (-1, -1, -1) and (Ri + 1, Gi + 1, Bi + 1) = (-1, -1, -1)
→ (Rcomp, Gcomp, Bcomp) = (Ri, Gi, Bi)
When both the pixel of the image i and the pixel at the same position of the image (i + 1) do not have the pixel value of the main subject, values (−1, −1, −1) having no pixel value after addition are set. That is,
(Ri, Gi, Bi) = (− 1, −1, −1) and (Ri + 1, Gi + 1, Bi + 1) = (− 1, −1, −1) pixels
→ (Rcomp, Gcomp, Bcomp) = (-1, -1, -1)
When the overlay synthesis for the main subject of the i-th image is completed as described above, it is determined whether or not the overlay synthesis for n images has been completed (step 190). Increments i by 1 (step 192) and returns to step 186. As a result, the main subject extraction process and the addition of the main subject are performed on the next still image in time series order.
[0100]
When the addition composition for n sheets is completed (i = n), the addition image is added to the background image and synthesized (step 194). In other words, the portion of the superimposed image that has no pixel value is filled with the background image pixels.
[0101]
The composite image created in this way is stored in the hard disk device 14 or displayed on the monitor device 20, uploaded to the moving image / print server 30 or the image server 54, and printed out.
[0102]
FIG. 14 is a creation image diagram of the composite image created as described above.
[0103]
Next, another embodiment of the overlay synthesis of the main subject image will be described.
[0104]
First, the processing from the extraction of main subjects from n still images is performed in the same manner as described above. A minimum rectangle including the image of the main subject extracted in this way is calculated, and an image (rectangular image) within the minimum rectangle is stored in the memory.
[0105]
Images shown in FIGS. 15A and 15B are an i-th rectangular image i and an i + 1-th rectangular image (i + 1), respectively.
[0106]
Next, the coordinates (x _i, y _i ) And the coordinates of the rectangular image (i + 1) (x _{i + 1,} y _{i + 1} ) Map the pixels so that they overlap.
[0107]
For example, when the image is aligned on the lower side of each rectangular image and superimposed so that the rectangular image (i + 1) overlaps the right half of the rectangular image i, the rectangular image (i + 1) is assumed to have a horizontal width of Max-xi. ) Coordinates of each pixel (x _{i + 1,} y _{i + 1} ) By adding Max-xi / 2 to the x-coordinate, _{i + 1} + Max-xi / 2 _, y _{i + 1} ). Thereafter, after mapping so that pixels with the same coordinates overlap, the pixel value after addition is determined so that the pixel value of the image (i + 1) is adopted for the portion where the pixel values of the pixels of both images overlap.
[0108]
In this way, by combining the rectangular image i shown in FIG. 15A with the rectangular image (i + 1) shown in FIG. 15B shifted to the right and combining them, the combined image shown in FIG. can get.
[0109]
A composite image as shown in FIG. 16 can be created by sequentially performing such overlay composition on n rectangular images (1 to n).
[0110]
In this embodiment, a still image (background image) that does not contain the main subject is shot separately from the movie. However, the present invention is not limited to this, and a movie that does not contain the main subject is shot. One frame of a moving image may be used as a background image. In addition, a plurality of still images are acquired from a moving image. However, the present invention is not limited to this, and a plurality of continuous images are captured in a continuous shooting mode in which images are continuously captured at a predetermined interval. It may be a picture.
[0111]
Further, as a method for extracting the main subject from the image, for example, a chroma key process is performed in which the main subject is photographed against a screen of a specific color such as a blue background and the main subject is extracted from the photographed image based on the specific color. May be adopted, or one of the pixels of the main subject in the image is pointed, the contour of the main subject is obtained from the continuity of the pointed pixel, and the main subject based on the contour You may employ | adopt the method of extracting. In the case of these main subject extraction methods, it is not necessary to separately shoot a background image that does not contain the main subject in order to extract the main subject.
[0112]
Next, still another embodiment of the overlay synthesis of the main subject image will be described.
[0113]
FIG. 17 is a diagram showing a schematic process until a still image obtained by enlarging a subject of interest from moving image data of a moving image file is synthesized.
[0114]
A moving image file composed of a plurality of continuous still images is read, and a moving image frame in the moving image file is disassembled to extract a designated frame. The designated frame is taken out in the same manner as described above. Then, it goes through three steps.
[0115]
The first step is a subject feature color extraction step 86 for extracting a subject feature color. In the subject feature color extraction step 86, a subject is specified by pointing several points in the subject, and a feature color list at each point is created.
[0116]
The second step is an image area extraction processing step 88 for performing image area extraction processing. In this step, a color approximate to the characteristic color created in the first step is extracted, the subject is specified by superimposing the extracted portions, and the subject area of all frames is overlaid to obtain the moving area of the image. Detect the movement range of the subject.
[0117]
The third step is a region cutout process 90. In this step, the region is cut out based on the movement range of the subject detected in the second step, and a still image is synthesized.
[0118]
Details of each step will be described.
[0119]
Details of the first step will be described. FIG. 18 is a diagram showing a feature color designation method in the first step, and FIG. 19 is a flowchart showing a flow of the feature color extraction step in the first step.
[0120]
In FIG. 18, first, the feature color of the subject that the user wants to specify is specified in a specific still image frame n in the moving image frame. If the subject that the user wants to specify is a girl, the user points with a mouse 24 a part of the subject, for example, point A as hair, point B as an arm, and point C as a torso. By pointing, the x and y coordinates of points A, B and C are determined. Then, the R, G, and B values at the A point are taken as the feature color ref (1), the R, G, and B values at the B point are taken as the feature color ref (2), and the R, G, and B values at the C point. The value is taken in and set as the characteristic color ref (3).
[0121]
Referring to FIG. 19, a specific still image frame n is displayed on the monitor 20 (step 196), and it is determined whether the characteristic color extraction process has been repeated as many times as necessary (step 198). Points to a portion of the subject (step 200). The characteristic colors at points A, B, and C are extracted by taking in the R, G, and B values (step 202). Thereafter, the process returns to the determination of whether the characteristic color extraction process has been repeated a required number of times (step 198).
[0122]
The second step will be described. FIG. 20 is a conceptual diagram showing an image area extraction processing step, and FIG. 21 is a flowchart of the image area extraction processing step.
[0123]
In FIG. 20, for the feature color ref (1), the feature color ref (2) of the point B, and the feature color ref (3) of the point C extracted in the first step, the colors approximated to each of them are shown. Extract.
[0124]
For the feature color ref (1) at point A, a portion within the color difference ΔC from the feature color ref (1) at point A is extracted. Since the still image frame n is composed of a bit plane, the difference between ref (i) and the pixel i is taken to check whether the portion is within the color difference ΔC, whether it is 0 or 1. When the value is 0, it is not applicable, and it is regarded as a portion where the colors do not match. When the value is 1, it corresponds and is regarded as a color-matched part. As a result, the characteristic color ref (1) at point A and the approximate color portion are extracted.
[0125]
For the feature color ref (2) at point B and the feature color ref (3) at point C, similar color portions are similarly extracted.
[0126]
These extracted characteristic color portions (position information) are superimposed on one still image frame n. The subject area of the still image frame n is defined (position specification). The process is performed for all the frames in the specified time range, and the defined subject area is overlapped for all the frames in the specified range. Then, a moving area of the subject image over a plurality of designated frames is obtained. The upper left x coordinate, upper left y coordinate (min_x, min_y), lower right x coordinate, lower right y coordinate (max_x, max_y) of the movement area are specified so that the movement area becomes a rectangle, and the movement area is finally The range of motion of the subject is detected.
[0127]
Referring to FIG. 21, first, the first still image frame to be processed is selected (step 204), it is determined whether or not the frame has been repeated until the end of the frame (step 206), and when the final frame is completed, the process proceeds to step 218. Otherwise, it is determined in step 208 whether or not the image area extraction for the feature color ref (i) is repeated, and if the extraction is completed, the process proceeds to step 214. If not completed, the color matching area within ref (i) and the color difference ΔC is extracted by the above-described method in step 210. Thereafter, the next characteristic color is set (step 212), and the process returns to step 208.
[0128]
In step 214, in the above-described example, the color matching areas for the characteristic colors of point A, point B, and point C are overlaid, and the subject area of the frame n is extracted. When this is completed, the next frame is targeted (step 216), and the process returns to step 206.
[0129]
In step 218, the subject areas of all designated frames are overlapped to obtain a moving area of the subject image. Then, the process proceeds to step 220, and the upper left x coordinate, upper left y coordinate (min_x, min_y), lower right x coordinate, lower right y coordinate (max_x, max_y) is specified and the moving area is finally defined and the process ends.
[0130]
The third step will be described. FIG. 22 is a flowchart showing the region cutout process. In this step, the image area of the movement range of the subject detected in the second step is cut out and a still image is synthesized.
[0131]
First, a target frame in which a moving area is defined is specified (step 222). The corresponding frame is read, and the defined moving area portion is cut out (step 224). The clipped image is output to the monitor (step 226). It is determined whether the target frame has ended (step 228). If not completed, the target frame in which another moving area is defined is specified (step 230), and the process returns to step 224. If yes in step 228, the process ends.
[0132]
After the image is cut out, for example, it is conceivable to combine the cut out still image as a divided photograph. In the synthesized image, the subject in each frame of the still image constituting the moving image is an enlarged image as compared with a normal divided photograph, so that the subject to be noticed can be seen larger and easier to see. Still images can be further synthesized from the divided photographs.
[0133]
Further, by superimposing the cut out still image images, it is possible to output a multi-exposure composite photograph or to combine it with an index image.
[0134]
As described above, according to this embodiment, it is possible to obtain an image centered on a subject to be noticed when, for example, a continuously photographed separated photograph image is printed.
[0135]
In addition, since the movement of the subject of interest is known during the attention frame period of the moving image, it is easy to cut out the frame area to be printed with the decomposed photograph.
[0136]
Furthermore, even when the extracted photograph extraction region cannot be automatically cut out automatically by operations such as rect_min_x, rect_min_y, rect_max_x, rect_max_y that define the coordinates of the upper left corner and the lower right corner of the rectangle, the reference image R [x, y ], G [x, y], and B [x, y], it is possible to easily specify an area to be manually cut out.
[0137]
According to the first embodiment described above, a necessary frame of a MOTION JPEG moving image is cut out, converted into an MPEG file, compressed, and the instruction information of the image composition method is created by the client personal computer 2 to create a moving image / print server. Therefore, a desired composition method can be set on the client side, and uploading can be performed with little transmission traffic when transmitting to the moving image / print server 30. Furthermore, by executing the image synthesis process with high added value on the moving image / print server 30, it is possible for the user to obtain a print that has undergone the image synthesis process with high added value.
[0138]
According to the second embodiment described above, the moving image data stored in the image server 54 is downloaded to the client personal computer 2, and the still image is displayed on the image server 54 with respect to the image frame designated from the client personal computer 2 and by the synthesis instruction information. Since they can be combined, it is possible to obtain a still image print with a desired quality for a desired frame even for a moving image obtained via a network.
[0139]
【The invention's effect】
According to the first aspect of the present invention, since the file converted into the upload file and the instruction information of the image composition method are created on the client side and uploaded to the server, a desired composition method can be set on the client side. When sending to, you can upload a file that considers the transmission speed.
[0140]
According to the second aspect of the present invention, a high-value-added image composition process is performed by the server-side image processing apparatus, so that the user can finally obtain a print that has been subjected to the image-value-added image composition process. be able to.
[0141]
According to the sixth aspect of the present invention, since the still image can be synthesized with the frame specified by the image processing apparatus on the client side and the synthesis instruction information from the moving image data stored in the server, the synthesized still image desired by the user is displayed. Obtained in high quality.
[0142]
According to the present invention of claim 7, since the moving image data is received from the server and reproduced, and the desired frame and composition method are designated from the image processing apparatus on the client side, the moving image data obtained via the network is also liked. It is possible to obtain a still image print that is synthesized by a desired synthesis method for a simple frame.
[Brief description of the drawings]
FIG. 1 is a diagram showing a schematic overall configuration of a moving image print ordering system according to a first embodiment.
FIGS. 2A to 2C show still image composite images when a predetermined range of a moving image is cut out and the cut out frames are combined in the form of a disassembled photo, multiple exposure, and overlay composition, respectively. Figure.
FIG. 3 is a block diagram showing a hardware configuration of a client personal computer.
FIG. 4 is a block diagram showing a hardware configuration of a moving image / print server.
FIG. 5 is a flowchart showing a flow in which a user designates information about a portion to be combined and an image combining method from a moving image file of moving image data stored in a hard disk device.
FIG. 6 is a flowchart showing a flow of uploading an image file, an image composition method, and order information to a moving image / print server.
FIG. 7 is a diagram showing a list of continuously shot images on a client personal computer.
FIG. 8 is a flowchart showing a flow from designation of a continuous shot image to uploading to a moving image / print server.
FIG. 9 is a flowchart showing a flow from when image data or the like is uploaded to when a completed print is sent to a user or a photo shop.
FIG. 10 is a diagram showing an overall configuration of a moving image print ordering system according to a second embodiment.
FIG. 11 is a flowchart showing a flow from when a user downloads moving image data stored in an image server to a client personal computer and designates and transmits a frame, an image combining method, and order information of a moving image file. .
FIG. 12 is a flowchart of the operation of the image server when downloading moving image data to the client personal computer, and the image server after the moving image frame designation information, the image composition method designation information, and the order information are transmitted from the client personal computer to the image server. The flowchart which showed the flow of operation | movement of.
FIG. 13 is a flowchart showing the flow of a subject overlay image composition method.
FIG. 14 is a creation image diagram of a composite image obtained by adding and combining images of a plurality of main subjects at the same position.
FIG. 15 is a diagram used for explaining a method of adding and synthesizing an image of a main subject by shifting.
FIG. 16 is a creation image diagram of a synthesized image obtained by shifting and overlaying images of a plurality of main subjects.
FIG. 17 is a diagram showing a schematic process until a still image in which a subject of interest is enlarged is synthesized from moving image data of a moving image file.
FIG. 18 is a view showing a feature color designation method in the first step.
FIG. 19 is a flowchart showing a flow of a feature color extraction process in the first process.
FIG. 20 is a conceptual diagram showing an image area extraction processing step.
FIG. 21 is a flowchart of an image area extraction processing step.
FIG. 22 is a flowchart showing an area cutout process.
[Explanation of symbols]
DESCRIPTION OF SYMBOLS 1 ... Movie print order system, 2 ... Client personal computer, 8 ... Communication interface, 10 ... CPU, 12 ... Main memory, 14 ... Hard disk drive, 18 ... Display memory, 20 ... Monitor device, 22 ... Keyboard, 24 ... Mouse, 26 ... Controller, 28 ... Bus, 30 ... Movie / print server, 32 ... Hard disk device, 34 ... CPU, 36 ... Working memory, 38 ... Movie processing unit, 40 ... Print unit, 42 ... Communication interface, 44 ... Still image storage memory , 46 ... still image composition unit, 48 ... composite still image sending unit, 50 ... bus, 52 ... movie print ordering system, 54 ... image server, 56 ... lab server, 58 ... movie storage memory, 60 ... hard disk device, 62 ... movie Cutout unit, 64 ... Still image composition unit, 66 ... CPU, 68 ... Still image storage memory, 7 ... Video compression unit, 71 ... Work memory, 72 ... Synthetic still image sending unit, 73 ... Bus, 74 ... Communication interface, 76 ... Communication interface, 78 ... CPU, 80 ... Synthetic still image storage memory, 82 ... Hard disk device, 83 ... Bus, 84 ... Photo printer subsystem, 86 ... Subject feature color extraction process, 88 ... Image area extraction process, 90 ... Area cutout process, 92 ... Frame designation screen, 94 ... Print order data input screen

Claims (11)

An image processing apparatus that processes video data and is connected to a server via a network,
Reading means to read video data,
A designation unit for designating information on a portion to be synthesized and an image synthesis method information from the movie file of the movie data read by the reading unit;
A conversion unit that cuts out the portion of the designated image to be combined from the video file and converts it into an upload file;
An image processing apparatus comprising: an upload unit that uploads a file converted into an upload file and instruction information on an image composition method to a server. An image processing apparatus having a server function connected to a client-side image processing apparatus via a network,
Receiving means for receiving an upload file transmitted from the image processing apparatus on the client side and image synthesis method instruction information for the file;
Generating means for generating a still image composite image from the received upload file and the instruction information of the image composition method;
An image processing apparatus comprising: transmission means for transmitting the generated still image composite image to a client-side image processing apparatus. The image processing apparatus according to claim 1, wherein the conversion unit uses a method of converting into an image recording format file based on a frame difference as a method of converting into an upload file. The image processing apparatus according to claim 1, wherein the image composition method includes a method of combining and synthesizing a subject of interest. 5. The image processing apparatus according to claim 1, wherein the image composition method includes a method of securing position information of a subject of interest at the time of composition and specifying a position of the subject of interest based on the position information. An image processing apparatus having a server function connected to a client-side image processing apparatus via a network,
Storage means for storing video data;
Clipping means for cutting out a designated frame designated by a client-side image processing device from a moving image file of moving image data stored in the storage means;
Receiving means for receiving information on a designated frame designated by the image processing apparatus on the client side and instruction information on an image synthesis method for synthesizing the image of the designated frame;
Synthesizing means for synthesizing a still image from the designated frame clipped by the clipping means and the instruction information of the image synthesizing method;
An image processing apparatus comprising: transmission / reception means for transmitting / receiving data of moving image data, still image data, and print order data of a synthesized still image. An image processing apparatus that processes video data and is connected to a server via a network,
Receiving means for receiving video data from the server;
Playback means for playing back the moving image data received by the receiving means;
Generating means for generating designation frame information for designating a frame to be cut out from the movie file of the movie data and instruction information for instructing an image synthesis method for synthesizing an image of the designated frame;
An image processing apparatus comprising: transmission means for transmitting the designated frame information and instruction information and print order data of the synthesized image to the server. The image processing apparatus includes a printing unit, and the printing unit includes:
A data receiving unit for receiving still images and print order data combined by the combining unit;
The image processing apparatus according to claim 6, further comprising: a printing unit that prints a still image received by the data receiving unit. The image processing apparatus according to claim 6, wherein the synthesizing unit uses an image recording format synthesis method based on a frame difference. The image processing apparatus according to claim 6, wherein the image composition method includes a method of combining and synthesizing a subject of interest. The image processing apparatus according to claim 6, wherein the image composition method includes a method of securing position information of a subject of interest at the time of composition and specifying a position of the subject of interest based on the position information.

Images