JP2000287127A - Image synthesizing device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To apply the most proper image processing to a background by synthesizing the image of a subject photographed with a camera with a background image in a selected optional position and size and selecting either the display of guidance in the optimum synthesis position or execution of optional movement editing processing by user. SOLUTION: A pattern memory 255 has a storing area previously devided to separately store two types of data and stores background image data to be superimposed on the image of the subject, lightness and saturation data for correcting the lightness and saturation of the subject and guide data such as an operation menu to be displayed on a monitor 241. An image processing part 256 executes a processing for extracting the contour of the subject image from subject image data which is fetched from an image input part 254 by the control of a CPU 251. The user superimposes the subject image on the designated position of the background image selected by a touch operation by the designated angle and size and executes synthesis.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image synthesizing apparatus for synthesizing a digital image of a subject with another digital image and outputting the synthesized image.

[0002]

2. Description of the Related Art Conventionally, in an image printing apparatus installed in a street, a storefront, a game center, or the like, a frame selected by a user from a plurality of frame images (heart mark, character, landscape image, etc.) prepared in advance. The image is combined with the subject image captured by the camera, and printed out and provided to the user. When using this image printing apparatus, the user first selects a desired frame image from a plurality of prepared frame images. When the frame image is selected, the selected frame image and the user who is the subject photographed by the camera are simultaneously displayed on the monitor screen. The user moves while looking at the monitor screen,
Determine your position in the frame image. After positioning,
The subject is photographed, and the subject image and the frame image are combined and printed out. JP-A-10-186473
The publication describes a plurality of different foreground frames and background frames prepared by a user in order to create a deep and interesting composite image. The image is synthesized by sandwiching the subject image in between.

[0003]

In a conventional image printing apparatus, a background image used for synthesizing a subject is mainly composed of a two-dimensionally drawn picture or frame. In these pictures and frames, a space for taking in an image of a subject at the time of composition is prepared in advance. Therefore, in the case of composition, since the image of the photographed subject is simply pasted in the determined space of the selected background image, the user can paste his own image at an arbitrary position with respect to the background. could not. Further, the user could not confirm where he or she was in the background image until the composite result was printed.

In order to deal with such a problem, the background data and the photographed subject data can be respectively held as digital information, so that the user can freely edit the position of the subject in the synthesizing stage. 2. Description of the Related Art There is known an image printing apparatus that combines a captured image of a subject with a possible background image by a chroma key combining process in a natural state as if the subject is present in the background. However, after the user actually takes his own image, the user moves the image size, angle, position, etc.,
Since the processing must be specified, the user must be familiar with the operations and procedures. Further, it takes a very long time for one shooting, combining, and printing, and the efficiency is poor. .

[0005] An object of the present invention is to determine at which position on the screen an image of a subject is synthesized in an image printing apparatus.
Before printing, show the positional relationship between the subject and the background data as a preview, show the position of the optimal background data of the desired subject, and edit the image to an appropriate size for the background and perform image composition. It is in.

[0006]

SUMMARY OF THE INVENTION An object of the present invention is to provide an image synthesizing apparatus for synthesizing and printing a background image arbitrarily selected by a user and an image of the user himself, and a camera for photographing a subject, Background image storage means for storing a plurality of prepared background images, and background image selection means operated when a user selects an arbitrary background image from the background images stored in the background image storage means When,
A subject image photographed by the camera is placed at an arbitrary position in the background image selected by the background image selecting means.
When combining images by size, this can be achieved by providing a process for displaying guidance at an optimum combination position and an editing process selecting means for selecting any one of arbitrary moving editing processes by the user.

[0007]

DESCRIPTION OF THE PREFERRED EMBODIMENTS One embodiment of the present invention will be described below in detail with reference to the drawings. FIG. 1 is a functional block diagram showing an embodiment of the image synthesizing apparatus according to the present invention. As shown in FIG. 1, a camera (digital camera) 2 for photographing a subject 1 and a background image A region-based brightness / saturation storage unit 4 stored; a background image selection unit 5 operated when a user selects an arbitrary background image from background images stored in the background image storage unit 3; In order to combine the subject image photographed by the camera 2 at an arbitrary position, angle, and size in the background image selected by the background image selecting means 5, the user may select the subject image combining position, angle, The subject image position, angle,
A size designating means 6 and a subject image photographed by the camera 2 are placed in the background image selected by the background image selecting means 5 by the subject image position / angle / size designating means (hereinafter abbreviated as designating means). ) Position, angle indicated by 6;
An image processing control unit 8 is provided which corrects and combines the brightness and saturation data stored in the brightness / saturation storage unit 4 according to size and outputs the corrected data from the output unit 7. The background image of each set in the background image storage means 3 is composed of a plurality of background image groups obtained by separating background components present in each set of background images in accordance with the position in the depth direction. It is possible to select not only the position of the background image on the two-dimensional plane but also the position of the subject image synthesis in the depth direction. Further, the background image group is further divided into a plurality of regions, and when the subject image is arranged for each of the divided regions, the brightness / saturation data such that the subject image fits into the background image without a sense of incongruity is obtained.
The brightness and saturation data stored in the saturation storage means 4 in advance and the brightness / saturation data stored in the area-specific brightness / saturation storage means 4 according to the arrangement position of the subject image designated by the user. The saturation is corrected and the background image is synthesized. Here, the background image stored in the background image storage unit 3 and the brightness / saturation data stored in the area-specific brightness / saturation storage unit 4 are prepared in a storage medium that stores these as one set. In addition, it is also possible to acquire from this storage medium, and it is also possible to directly input a background image and lightness / saturation data from an input device attached to the present apparatus. By using a recording medium,
The background image according to the installation place or use environment of the present apparatus can be used very easily, and updating can be easily performed. It should be noted that a configuration is possible in which the information is acquired from the server computer using a communication line such as the Internet instead of the recording medium.

FIG. 2 is a side view showing an embodiment of the image synthesizing apparatus. A space for accommodating a plurality of subjects 22 is provided inside a housing 21. On the front side of the subject 22, a camera 23 for photographing the subject 22, a monitor 24 provided with a touch panel corresponding to the background image selecting means 5 in FIG.
Is arranged. Further, a control device 25 including an image processing control unit 8, a background image storage unit 3, and an area-specific brightness / saturation storage unit 4 is arranged. The camera 23 is, for example, CC
A D camera is used, and its field of view is set so that an image of the whole body from the head to the feet of one or more subjects 22 who have entered the inside of the housing can be taken.

FIG. 3 is a block diagram showing a configuration of a functional portion of the image synthesizing apparatus shown in FIG.
U251, memory 252, I / O control unit 253, image input unit 254, pattern memory 255, image processing unit 25
6, an image output unit 257 is provided, and an I / O control unit 253 is provided.
Is connected to a touch panel 242. The video output unit 257 is connected to a video printer 26 and a monitor (display) 241. Touch panel 24
Numeral 2 is arranged on the screen surface of the monitor 241 (corresponding to the monitor 24 in FIG. 2), and outputs a position signal indicating the position when touched or pressed at an arbitrary position on the display screen. In the form, selection of various operation menus,
It is used to select a background image and to specify the composition position, angle, and size of the subject image. As the camera 23, for example, a CCD camera is used.

An image input unit 254 converts image data photographed by the camera 23 from an analog signal to a digital signal by an A / D converter (not shown). CPU 251
Extracts the contour of the subject image from the image captured by the camera 23, combines the contour with the background image stored in the pattern memory 255, and performs print output processing to the video printer 26. The memory 252 stores the subject image data fetched from the image input unit 254, and also stores, as a program, a command issued to the CPU 251 and a command group serving as control means of the composite image creating apparatus. Also, CPU2
The processing data of 51 is also temporarily stored as a work memory.

The pattern memory 255 has a storage area of 255A in advance so as to store two types of data separately.
And 255B, and stores background image data to be superimposed on the subject image, brightness / saturation data by region for correcting the brightness and saturation of the subject, and guide data such as an operation menu displayed on the monitor 241. Have been. The pattern memory 255A stores background image data to be superimposed on the subject image, brightness / saturation data for each region for correcting the brightness / saturation of the subject, and an operation menu guide displayed on the monitor 241. Pattern memory 255
B stores an image captured by the user. The background image data and the brightness / saturation data for each area will be described later.

The image processing section 256 executes contour extraction processing of a subject image from subject image data taken from the image input section 254 under the control of the CPU 251. Then, the user (subject) superimposes and synthesizes the image at a specified position and a specified size in the background image selected by the touch panel operation. The image output unit 257 includes the image processing unit 2
The combined image is displayed on the monitor screen and printed out from the video printer 26.

FIGS. 4, 5 and 6 show background image data and brightness / saturation data by area stored in the pattern memory 255. FIG. FIG. 4 is a diagram showing the configuration of one set of background image data stored in the pattern memory 255. FIGS. 5 and 6 show the configuration of area-specific brightness / saturation data stored for one set of background images. FIG.
55 stores in advance the plurality of sets of background image data and the brightness / saturation data for each area corresponding to the sets. In a case where there is one background image 401 as shown in FIG. 4, in the present invention, as shown in FIG. 4, a background component in one background image is changed according to the distance in the depth direction from the front. Is divided (separated) into a plurality of images while determining.

That is, in the example of FIG. 4, the background components 402 and 403 on the first front side are divided into the first separated background image 41.
1, the background component 404 located immediately behind is separated as the second separated background image 412, and thereafter, the background component 405 is similarly separated into the third separated background image 413 and the background. The component 406 is separated as a fourth separated background image 414, and the background component 407 is separated as a fifth separated background image 415. In this case, the number of separation is arbitrary. The pattern memory 255 stores the data of one background image 401 as a background image group including the separated background images 411 to 415.

When synthesizing the subject image 410 with the background image, any two consecutive two or more of the plurality of separated background images are operated by a user's touch panel instruction operation as the subject 22.
By sandwiching the subject image between the separated background images, the subject image is synthesized at an arbitrary position in the depth direction.

As described above, when synthesizing a subject image at an arbitrary position in the depth direction of the background image, when creating a natural synthesized image that is familiar with the background at the synthesis position, and when synthesizing the subject image so as to match the background at the synthesis position. It is necessary to correct the brightness and saturation of the image. The brightness / saturation data for each area is necessary for that purpose. As shown in FIGS. 5 and 6,
Each of the separated background images shown in FIG. 4 is further divided into a plurality of regions (9 regions in the example of FIG. 6) by a human. Hereinafter, each of the divided areas is referred to as a divided area. The number of divisions is arbitrary, and the sizes of the regions to be divided may not be uniform.

In FIG. 6, each of the separated background images is equally divided into nine.
An example in which the image is divided into two rectangular areas is shown. For each of the divided areas, a human determines in advance lightness / saturation data that is optimal when the subject image is placed in the divided area, and indicates the number of separated background images as shown in FIG. Are stored as the lightness 601 and the saturation 602 of each divided area (a to i) for each position. The brightness 601 and the saturation 602 of each divided area to be stored are determined as follows. A model image is placed in a divided area of the separated background image, and the brightness and saturation of the model image are synthesized by the human while adjusting the brightness and saturation of the model image. Is the brightness / saturation of the divided area. By performing this for all the divided areas of the all-isolated background image, the brightness 601 and the saturation 6
Decide 02. For all of the plurality of background images prepared in advance, the separated background image of the background image is divided into divided regions, and the brightness and saturation are determined for each divided region and stored in the pattern memory 255 as described above. When the position of the subject image is designated by the user, the data of the brightness 601 and the saturation 602 at the placement position are referred to and applied as the brightness and the saturation of the subject image.

FIG. 7 is a flowchart showing the editing process of the position of the user image. First, the subject (user) 22
Selects a desired background image from a plurality of background images displayed on the monitor / touch panel 24 (step 70).
1). Next, video shooting of the subject 22 is performed by the camera 23 (step 702). Before combining, the user is required to edit the position of the shot image with respect to the background,
Automatically edit to the optimal position and size automatically,
Alternatively, the user is allowed to select whether or not to edit (step 703). When the automatic editing is selected, a rectangle having the size of the captured image is displayed at the position to be combined with the background selected as shown in FIG. 8 (step 7).
04). If the combining position shown here is acceptable, the combining is performed as it is and printing is performed. If the user does not like the displayed position, the process returns to step 703 where the user himself edits the image. The image is moved to an arbitrary position using the touch panel and the image is synthesized (the process proceeds to FIG. 11).

FIG. 9 is a flowchart showing details of the flow of the automatic processing for displaying the guidance rectangle at the optimum image position in step 704. When the automatic processing is selected in step 703, the number of elements (how many elements) the background image selected by the user is extracted based on the data of FIGS. ). At least two or more large elements (such as buildings and trees) are extracted from the elements (step 902). The depth data between the plurality of extracted elements and the size of each element are calculated (step 903). It is checked whether the image of the user to be synthesized is a whole body, a half body, or a plurality of images by referring to the pattern memory 255 (step 904). If a plurality of whole-body images are shown, the size of the user image is compared with the element calculated in step 903. First,
It is determined whether or not the component to be compared is at least half before the entire background image (step 905). If the target background image element is located before half of the total number of background images (shallow depth), the components of the background image are further compared with the size of the user, and as a result of the comparison, the size of the user is reduced. If the calculated unit ratio of the background element is equal to or smaller than 1/2 (step 906), a search is made for a space portion of the image in which the compared background element is shown, and a rectangle is displayed at that location (step 906). Step 907). If the size of the user is １ ／ or more, the user moves to another background element (element having a large depth) and further reduces and displays the image (step 908).

If it is determined in step 904 that the image of the user is half body, it is determined whether or not the component is more than half of the whole background image and is behind (deep) (step 9).
09) Further, a portion where the elements of the background image overlap is detected and displayed there (step 910). Step 90
In step 9, if the target background image element is more than half forward (shallow depth), the process moves to the user's moving and editing process to an arbitrary position (step 911).

FIG. 10 is a flowchart showing image composition and image position editing. This will be described with reference to FIG. FIG.
The background image selected in step 701 and step 70
The monitor image of the subject captured in step 2 is superimposed and displayed on the monitor / touch panel 24 on the screen. Next, the subject (user) 22 determines a pose, and shoots the subject 22 with the camera 23 at the shutter timing directly specified by the subject (user) 22 or the shutter timing determined by the apparatus (step 1001). The contour of the subject 22 is extracted from the photographed image (step 1002), and the extracted subject image is used as the subject (user) 22 in step 1001.
(Step 1003). The corrected subject image and the background image are combined (step 100).
4) Print and output by the video printer 26 (step 1005).

FIG. 11 shows the flow of operations from the position designation on the plane in the background image to the insertion / position designation in the depth direction of the subject image, and the display contents of the monitor and touch panel 24. Hereinafter, the details of the method of designating the position, angle, and size will be described with reference to FIGS. 11 and 12 in accordance with the step sequence of FIG. Here, the selected background image and the subject monitor image captured by the camera 23 are displayed on the monitor / touch panel 24 according to the designation of the subject (user) 22.
An example of moving, rotating, enlarging (or reducing), combining and displaying will be described.

First, the control device 25 displays a subject monitor image captured by the camera 23 in the foreground on the background image selected by the subject (user) 22 in a predetermined rectangular area (step 1101). Next, the control device 25
Displays guidance on the monitor screen of the monitor / touch panel 24 to specify the position of the subject image on the two-dimensional plane, and prompts the subject (user) 22 to move the position of the subject image (step 1102). ). Thus, the subject (user) 22 moves on the screen by touching the subject monitor image with a finger by operating the touch panel, and specifies a position on a two-dimensional plane (step 1103). That is, the subject monitor image 11031 indicated by the broken line is moved in the direction of the arrow on the screen and stopped at the desired combination position. 1
Reference numeral 1032 denotes a subject monitor image stopped at a desired combination position. Next, when the position on the two-dimensional plane has been specified, the subject (user) 22
Then, the user presses the "position determination button" displayed above (step 1104).

Next, the control device 25 displays guidance for designating the rotation angle of the subject monitor image (step 1105). The subject (user) 22 rotates his / her finger while touching the subject monitor image 11032 on the monitor / touch panel 24 so that the subject can be monitored in accordance with the amount of movement of the finger around a predetermined fulcrum such as the center point of the subject monitor image. By rotating the image 11032, the rotation angle of the subject monitor image 11032 is specified (step 1106). The control device 25 controls the subject monitor image 1103 according to the designation from the subject (user) 22.
Rotate 2. 11033 is a subject monitor image in which the rotation angle is specified. When the rotation angle of the subject image has been specified, the subject (user) 22 presses an “angle determination button” displayed on the monitor / touch panel 24 (step 1107).

Next, the control device 25 displays guidance to specify the size of the subject image (step 11).
08). The subject (user) 22 is the subject monitor image 1
The user touches 1033 and moves the finger in a direction away from or near the center line of the subject monitor image area displayed as a rectangular area. When the finger moves away from the center line, it is determined that the image is enlarged, and when the finger moves toward the center line, it is determined that the image is reduced. The subject (user) 22 specifies the size of the subject monitor image by moving the finger (step 1109). The control device 25 controls the subject monitor image 1 according to the designation from the subject (user) 22.
The size of 1033 is changed. 11034 represents an enlarged monitor image. After specifying the size of the subject image, the subject (user) 22 displays the “size determination button” displayed on the monitor / touch panel 24.
Is pressed (step 1110).

Next, the control device 25 displays guidance for designating the insertion position of the subject monitor image in the depth direction (step 1111). The subject (user) 22 specifies an insertion position in the depth direction on the monitor / touch panel 24 (step 1112). The control device 25 controls the subject monitor image 1 according to the designation from the subject (user) 22.
The insertion position of the 1034 in the depth direction is changed. Details will be described later with reference to FIG. 11035 is a subject monitor image in which the insertion position in the depth direction has been determined. When the insertion position in the depth direction has been specified, the user presses a “depth insertion position determination button” displayed on the monitor / touch panel 24 (step 1113).

FIG. 13 shows a method of designating the position of inserting the subject image in the depth direction and the display on the monitor / touch panel 24. First, the position of the subject (user) 22 on the plane designated in steps 1003 to 1110 is placed in front of the first separated background image 411 which is the foreground on the background image selected by the subject (user) 22. A subject monitor image 11034 deformed to an angle and size is displayed. A button for designating the insertion position in the depth direction is displayed on the monitor together with the subject monitor image 10034. When the subject (user) 22 presses a button 1301 indicating a backward direction (a direction toward the back), the subject monitor image 11034 is moved to the back of one of the separated background images 411 to 415 previously separated for each element in the depth direction. I do. Before the backward direction button 1301 is pressed, the subject monitor image
Back button 1 if it is on the front of the sheet separated background image
By pressing 301, the subject image is positioned between the n-th separated background image and the (n + 1) -th separated background image. That is, the subject image in front of the n-th separated background image exists in front of the (n + 1) -th separated background image.

In this state, the control device 25 combines the entire separated background image and the subject image, that is, inserts and combines them, and displays them on the monitor screen. Further, when the rear direction button 1301 is pressed, the subject monitor image 1103 is further moved backward by one image.
4 is moved and synthesized and displayed on the monitor.
When 02 is pressed, the subject monitor image 1 is moved forward by one image.
1034 is moved and synthesized, and displayed on a monitor. In this way, the subject monitor image is synthesized by shifting the insertion position of the subject monitor image sequentially and displayed on the monitor, so that the subject (user) 2
2 can select the insertion position of the subject monitor image 11034 with respect to the separated background images 411 to 415. Furthermore, the brightness and saturation are determined in advance according to the position including the depth on the background image, and the brightness and saturation of the subject image are corrected and combined according to the arrangement position of the subject image at the time of the above-described image combining, thereby combining the images. In addition, the subject image can be blended with the background image.

FIG. 14 shows a procedure for correcting a subject image. First, the position of the subject image designated by the subject (user) 22 on the background image plane, the rotation angle, the enlargement /
Four pieces of information of the reduction magnification and the insertion position in the depth direction are extracted (step 1401). Next, the subject image is
, And move based on the information of (step 140)
2). Next, the number of the separated background image in which the deformed or moved subject image is located and which divided region in the region of the separated background image occupies is extracted. Here, when the subject image extends over a plurality of divided regions, the divided region occupying the largest area is adopted (step 1403). Next, the brightness / saturation data for each divided region corresponding to the above-described insertion position in the depth direction and the divided region occupied by the taken out subject image is extracted from the pattern memory 255 in FIG. 3, and the extracted brightness / saturation data for each divided region is extracted. The subject image is corrected as the brightness / saturation of the subject image (step 1404). For example,
If there is a subject image between the (n-1) -th separated background image and the n-th separated background image and the divided area occupied by the subject image in the n-th separated background image is i, The brightness / saturation of the subject image is corrected by the brightness / saturation data of the n-th divided area i in the “depth direction insertion position” in the saturation data. As described above, it is possible to create a natural composite image that is interesting and blends with the background image.

[0030]

According to the present invention, it is possible to predict in advance where the photographed image is to be pasted on the background at the time of combining the photographed image with the background. Further, since the image position can be moved to an arbitrary position by himself before the combination, the combination result intended for the photographer can be predicted before the combination.

[Brief description of the drawings]

FIG. 1 is a functional configuration diagram showing an embodiment of an image synthesizing apparatus according to the present invention.

FIG. 2 is a side view showing an embodiment of the image synthesizing apparatus of the present invention.

FIG. 3 is a block diagram showing an embodiment of a functional part of the image synthesizing apparatus of FIG. 2;

FIG. 4 is an explanatory diagram showing a configuration and a storage method of a background image.

FIG. 5 is an explanatory diagram showing a configuration and a storage method of data for brightness and saturation correction.

FIG. 6 is an explanatory view showing a continuation of FIG. 5;

FIG. 7 is a flowchart illustrating editing processing of a position of a user image.

FIG. 8 is a diagram showing an editing state of a user image.

FIG. 9 is a flowchart illustrating details of an automatic processing flow for displaying a guidance rectangle at an optimal image position in step 704.

FIG. 10 is a flowchart illustrating an image synthesis processing procedure.

FIG. 11 is an explanatory diagram showing a procedure for specifying a position, an angle, and a size of a subject image and a display example of a monitor screen.

FIG. 12 is an explanatory view showing a continuation of FIG. 11;

FIG. 13 is a flowchart illustrating a designation procedure of a depth direction insertion position of a subject image and a display example of a monitor image.

FIG. 14 is a flowchart illustrating a procedure for correcting a subject image.

[Explanation of symbols]

Reference numeral 22: subject, 23: camera, 24: monitor and touch panel, 25: control device, 26: video printer, 251
... CPU, 252 ... memory, 253 ... I / O control unit, 2
54 ... image input unit, 255 ... pattern memory, 256 ...
Image processing unit, 257: Image output unit, 242: Touch panel.

Continued on front page F-term (reference) 5B050 BA11 BA12 CA07 EA12 EA19 FA02 FA03 FA05 FA08 FA13 5C022 AA13 AB68 AC01 AC13 AC69 5C023 AA02 AA03 AA04 AA16 AA31 AA37 AA38 BA11 CA03 DA04 DA08 5C054 AA01 EA01 EA07 FC12 AA12 AA19 AA21 AA22 AA24 BA03 BA04

Claims (1)

[Claims] An image synthesizing apparatus for synthesizing and printing a background image arbitrarily selected by a user and an image of the user himself, and stores a camera for photographing a subject and a plurality of background images prepared in advance. Background image storage means, a background image selection means operated when a user selects an arbitrary background image from background images stored in the background image storage means, and a subject photographed by the camera The image is placed at an arbitrary position in the background image selected by the background image selecting means.
An image synthesizing apparatus comprising: a process of displaying guidance at an optimum synthesis position when an image is synthesized in a size; and an editing process selection unit that selects any one of arbitrary moving editing processes by a user.