JP2008079185A - Automatic photograph creating apparatus, and automatic photograph creating method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a photograph creating apparatus which can perform chromakey processing even when a key color added to a part within a photographing range except for the background is different from the key color of the background. <P>SOLUTION: A main control part 80 of the photograph creating apparatus segments a first area corresponding to a back surface in a casing inner surface from a photographed image (S401), corrects the image (S402) to create a chromakey mask (S403), segments a second area corresponding to a floor surface from the photographed image (S404), corrects the image according to the variation of its color (S405), and thereafter creates a corresponding chromakey mask (S406). Then, a user selects a background image (S407), and the background image is chromakey-composed to the photographed image (S408). By correcting the images by each area like this to equalize different colors, a proper chromakey mask is created to perform chromakey composition. <P>COPYRIGHT: (C)2008,JPO&INPIT

Description

  The present invention relates to an automatic photo creation apparatus that captures a user with a camera and outputs a composite image generated based on the captured image as a photograph. More specifically, the present invention relates to a composite image generated based on a chroma key composition method. The present invention relates to an automatic photo creation apparatus that outputs a photograph.

  2. Description of the Related Art Conventionally, there is known a photo creation device for a game where a user is photographed with a camera and the photographed image is output as a photograph. Such a play photo creating apparatus has high playability or entertainment, and therefore, various images prepared in advance according to the user's preference are prepared for images to be synthesized with the photographed images (hereinafter referred to as “additional images”). In many cases, a user can select from various background images and foreground images, or can freely draw by a user using a pen-type input unit such as a touch pen or a light pen. Since such an action by the user is performed on the photographed image, it is called “graffiti”.

In recent years, a so-called chroma key composition technique is used to perform graffiti for compositing an additional image appropriately selected by a user from a plurality of pre-stored images so as to be a background of the photographed user. is there. In this chroma key composition method, a background of a single color (for example, blue or green) is placed behind the user to be photographed, and the background color included in the photographed image obtained by photographing this background and the user is the key color. Mask (create a chroma key mask) an area (background area) in the photographed image containing the color as (chroma). By synthesizing a part of the image appropriately selected by the user with the masked background area, a synthesized image is obtained as if the image selected by the user was photographed as an actual background. 2. Description of the Related Art Conventionally, there is an automatic photo creation apparatus that uses such a chroma key composition method (see, for example, Patent Document 1).
Japanese Patent Laid-Open No. 5-161064

  However, the background with the key color in the conventional photo creating apparatus is usually provided only on the inner surface of the casing (hereinafter referred to as “rear surface”) behind the user to be photographed. . Because the background color for chroma key composition must be the same, for example, even if the same background color is given to the user's top, bottom, left, and right surfaces (that is, the left and right side surfaces, top surface, and floor surface inside the housing) This is because the back does not have the same color.

  Even if the background with the key color is provided only on the back of the housing in this way, there is no particular problem if the shooting range is the upper body of the user. It becomes a problem when it is the whole body. Hereinafter, a description will be given with reference to FIG.

  FIG. 10 is a diagram illustrating an imaging range in a case where almost the entire body of the user is imaged. As shown in FIG. 10, the background with the key color is only the back of the housing, and if the vicinity of the user's feet is photographed, the floor surface without the key color is reflected. Therefore, the floor surface that should be the background cannot be masked. As described above, even if a key color is given to the floor surface, it does not become the same color as the back surface. Therefore, as shown by the dotted line in FIG. 10, the shooting range 901 is set so that the floor near the user's feet is not shown but only the user and the back are shown (specifically, the angle of view of the camera and the shooting angle). Direction etc. are set appropriately).

  However, in recent automatic photo creation devices for games, the whole body including the user's feet is often photographed, and it is not preferable that the photographing range is limited. Further, the camera installation position may be not only in front of the user but also upward (or obliquely upward). In this case, if the shooting range is set so that the floor surface is not captured, the range is significantly limited.

  Therefore, the object of the present invention is to cope with each case even when the key color attached to the portion within the shooting range other than the background provided behind the user is different from the key color attached to the background. It is an object of the present invention to provide an automatic photo creation apparatus capable of creating a chroma key mask and performing chroma key processing.

A first invention is an automatic photograph provided with photographing means for acquiring an image including a user who is a subject in a predetermined photographing region, and output means for outputting a composite image including a photographed image obtained by the photographing means as a photograph. A creation device,
A background presenting means that is a subject to be photographed by the photographing means and has a first photographing surface with a first color and a second photographing surface with a second color;
Image correction according to the color of the area occupied by the image of the first image-capturing surface is performed on the first area including the image of the first image-captured surface of the captured image, or the image capturing By performing image correction according to the color of the area occupied by the image of the second image-captured surface on the second area including the image of the second image-captured surface of the image, the first area Mask creating means for creating a predetermined chroma key mask corresponding to the color of the area occupied by the image of the imaged surface and the color of the area occupied by the image of the second imaged surface;
Using a chroma key mask created by the mask creating means, and performing a chroma key process for synthesizing a predetermined additional image to the captured image, thereby providing a chroma key processing means for generating the synthesized image. Features.

According to a second invention, in the first invention,
The first and second colors are substantially the same;
The color of the area occupied by the images of the first and second imaging surfaces is different.

According to a third invention, in the first invention,
The mask creating means creates the predetermined chroma key mask corresponding to the region of the same color by performing the image correction so that the regions occupied by the images of the first and second imaging surfaces are the same color. It is characterized by.

According to a fourth invention, in the first invention,
The first surface to be photographed has a surface different from a plane substantially perpendicular to the photographing direction of the photographing means,
The mask creating means performs predetermined image correction on the first area so that the color change of the area occupied by the image of the first surface to be photographed is eliminated.

A fifth invention is the fourth invention,
The first surface to be imaged is inclined from a surface along a direction substantially perpendicular to the imaging direction of the imaging means;
The mask creating means applies the first region to the first region so that the change in color of the region occupied by the image of the first surface to be captured that changes according to the inclination of the first surface to be captured is eliminated. A predetermined image correction is performed.

According to a sixth invention, in the fifth invention,
The first imaged surface is a floor surface provided below the user, a top surface provided above the user, and a left side of the user among the inner surfaces of the housing that the user should enter. Any one or more of a left side provided and a right side provided on the right side of the user;
The second surface to be photographed is a back surface provided in the rear direction of the user starting from an installation position of the photographing unit among inner surfaces of the housing.

According to a seventh invention, in the first invention,
An operation input means for receiving an operation input by a user for selecting the predetermined additional image from a plurality of predetermined images;
The chroma key processing means performs a chroma key process for synthesizing the additional image received by the operation input means with respect to the photographed image.

In an eighth aspect based on the first aspect,
An image serving as a reference for image correction performed by the mask creating unit, the background image including the first and second regions and not included in the subject by the user at a predetermined time point. It further comprises sampling means for obtaining by the photographing means,
The mask creating means is based on the color of the area occupied by the image of the first imaged surface in the background imaged image acquired by the sampling means, or further, the second imaged image in the background imaged image. The image correction is performed based on the color of the area occupied by the image of the surface.

In a ninth aspect based on the eighth aspect,
An error detecting means for detecting an abnormality including a color to be included in the image of the user in the background photographed image acquired by the sampling means;
The mask means does not use a background photographed image in which an abnormality is detected for the image correction when an abnormality is detected by the error detection means.

A tenth aspect of the invention is an automatic photograph comprising a photographing step for acquiring an image including a user who is a subject in a predetermined photographing region, and an output step for outputting a composite image including the photographed image obtained in the photographing step as a photograph. A creation method,
The first object to be photographed in the photographing step, the first object in the background presentation means having a first surface to be imaged with a first color and a second surface to be imaged with a second color. The first area including the image of the imaging surface is subjected to image correction corresponding to the color of the area occupied by the image of the first imaging surface, or further including the image of the second imaging surface. By performing image correction according to the color of the area occupied by the image of the second imaging surface on the area of 2, the color of the area occupied by the image of the first imaging surface and the second A mask creation step for creating a predetermined chroma key mask corresponding to the color of the area occupied by the image of the imaged surface;
Using a chroma key mask created in the mask creating step, and performing a chroma key process for synthesizing a predetermined additional image with the captured image, thereby generating a synthesized key image. Features.

  An eleventh invention is a program that causes a computer to execute the automatic photo creating method according to the tenth invention.

  According to the first aspect of the invention, the image correction corresponding to the color of the area occupied by the image of the first imaging surface is performed on the first area, or the second area is further corrected with respect to the second area. By performing image correction according to the color of the area occupied by the image of the surface to be imaged, a predetermined chroma key mask corresponding to the color of the area is created, so the first and the second to be the background within the imaging range Even when the colors of the two surfaces to be photographed are different, chroma key masks corresponding to each of them can be created and chroma key processing can be performed.

  According to the second aspect of the invention, the attached colors are substantially the same, but as a result (for example, due to differences in material, shooting direction, illumination direction, etc.), the images of the first and second shooting surfaces occupy. Even when the colors of the regions are different, chroma key masks corresponding to the respective colors can be created and chroma key processing can be performed.

  According to the third aspect of the invention, by performing image correction, the areas occupied by the images of the first and second imaging surfaces are made the same color, so that only one chroma key mask corresponding to the same color can be created. The configuration can be simplified.

  According to the fourth aspect of the invention, since the first surface to be imaged has a surface different from a plane substantially perpendicular to the imaging direction of the imaging means, the hue is not uniform, Since the predetermined image correction is performed on the first region so that the color change is eliminated by the mask creating means, this configuration is based on an image having ideal uniformity and hue for chroma key processing. Chroma key processing can be performed.

  According to the fifth aspect of the invention, since the first surface to be imaged is inclined from a plane substantially perpendicular to the imaging direction of the imaging unit, the hue changes according to the inclination. As a result, a predetermined image correction is performed on the first area so that the color change corresponding to the inclination is eliminated. Therefore, with this configuration, an image having ideal uniformity and hue for chroma key processing can be obtained. Based on this, chroma key processing can be performed.

  According to the sixth aspect, since the first and second imaging surfaces correspond to any of the inner surfaces of the housing, the first and second objects to be taken when the inner surface of the housing is used as the background. Even when the color of the photographing surface is different, it is possible to create a chroma key mask corresponding to each and perform chroma key processing.

  According to the seventh aspect of the invention, since the operation input unit receives an operation input for selecting an additional image from the user and the chroma key processing unit synthesizes the additional image, the user selects the additional image selected by the user ( A composite image as a (virtual) background image can be obtained, and the playability of the apparatus can be improved.

  According to the eighth aspect, a background photographed image that is not included in the subject by the sampling means is acquired by the sampling means at a predetermined time, and image correction is performed based on the background photographed image. Differences in the angle between the first and second surfaces to be photographed, the difference in material, or the difference in dirt (for example, the floor surface becomes dirty due to being stepped on by the user, the color becomes dull or worn and the color becomes dull. The chroma key processing can be performed based on an image having ideal uniformity and hue for chroma key processing.

  According to the ninth aspect of the invention, since the error detection means detects an abnormality in which the background photographed image includes a color that should be included in the user image, the background photographed image including the user image is subjected to chroma key processing. By doing so, it is possible to prevent an abnormal composite image from being created.

  According to the tenth aspect, the same effect as that of the first aspect can be achieved in the automatic photo creation method.

  According to the eleventh aspect, the same effect as that of the first aspect can be achieved in the automatic photo creation program.

Hereinafter, an embodiment of the present invention will be described with reference to the accompanying drawings.
<1. Overall configuration>
FIG. 1 is a side view of an automatic photo creation device (play photo creation device) according to an embodiment of the present invention, and FIG. 2 is a front view of a main body 4 of this photo creation device (as a subject). FIG. Hereinafter, the overall configuration of the automatic photo creation apparatus according to the present embodiment will be described with reference to these drawings.

  The automatic photo creation apparatus includes a photographing room 2 in which a user U enters and a main body 4 that generates a composite image including a photographed image of the user U and outputs it as a photo sticker (or a photo card). The number of users U may be one or more. However, for convenience of explanation, it is assumed that there are two users U.

  The shooting room 2 has a substantially rectangular parallelepiped shape, and is a top surface as viewed from the user U entering inside, a floor surface as a lower surface, left and right side surfaces as left and right surfaces, It has a back surface that is a surface and a front surface that is a front surface. The main body 4 is disposed along the front surface. In addition, the opening part for the user C to go in and out and the light-shielding curtain which covers a part or all of the said opening part are provided in a part of right and left side surface.

  As will be described later, a single color (in this case, blue or green) for chroma key composition processing is assigned to the back surface and the floor surface of the photographing room 2. However, since the plate material that forms the back surface and the plate material that forms the floor surface are different, even if they are given the same color (or even if they are devised to be the same color), the color is completely It cannot be the same. In addition, the back surface is arranged almost perpendicular to the shooting center direction of the camera, and the illumination is applied almost uniformly as will be described later, so the color is almost uniform, but the floor is in the shooting direction and lighting direction of the camera. On the other hand, since it has a predetermined angle, the color is not uniform. Therefore, a general chroma key composition process cannot be performed, but in this embodiment, the chroma key composition process is performed by a characteristic configuration described later. In the present specification, the term “hue” refers to a matter related to color including elements that are distinguished from each other in terms of visual sense such as hue and saturation.

  The main body 4 is disposed below the camera 10 as a camera 10 as a photographing means for photographing the user U, a flash 11 as an illuminating means disposed at an appropriate position above the camera 10 to emit a flash. A preview monitor 22 which is a display device including a CRT (Cathode Ray Tube) or a liquid crystal display panel, a liquid crystal display which provides an operation screen for photographing to the user U, and an operation position detection which covers the operation screen And a liquid crystal touch panel 20 made of a transparent sheet.

  The camera 10 is typically a digital camera that generates a digital image signal using a CCD (Charge Coupled Device). The camera 10 captures an image of the user U in the imaging room 2 and represents the captured image. Is output.

  The preview monitor 22 is configured to display an image represented by the image signal in real time, and is used to confirm a captured image obtained by the camera 10.

  The liquid crystal touch panel 20 functions as a display unit that provides the above-described operation screen and the like, and also functions as an operation unit that receives various operations by the user U, such as an operation unit for photographing. The liquid crystal touch panel 20 is operated by a touch pen (not shown), but may be operated by a finger of the user U, for example, a liquid crystal touch panel having an electrostatic induction type operation position detection function that does not include the transparent sheet. It may be.

  Further, the main body 4 is composed mainly of a computer and outputs a composite image including a photographic image obtained by the main controller, the I / O controller, and the camera 10 that controls each part as a photographic sticker (or photographic card). It has a built-in printer. Further, the main body 4 is provided with a coin insertion slot 26 below the front surface and a photographic sticker outlet 28 on the side surface. Furthermore, the main body 4 includes illumination devices 13L and 13R that are always lit during the operation of the automatic photo creation device, on the left and right sides of the camera 10, respectively.

<2. Functional configuration>
FIG. 3 is a block diagram illustrating a configuration of the main part of the automatic photo creation apparatus according to the present embodiment as viewed from the functional aspect. Functionally, the automatic photo creation device is functionally configured to include a photographing unit 52, a confirmation display unit 54, a main display / operation unit 62, an output unit 66, an illumination unit 70, an I / O control unit 72, a main unit. And a control unit 80.

  In this configuration, the photographing unit 52 corresponds to the camera 10 configured using an image sensor such as a CCD, and captures an image signal in real time and represents the image (this image is referred to as a “through image”). Output. This image signal is input to the main controller 80 and temporarily stored as through image data in its internal memory. The through image data is supplied from the main control unit 80 to the confirmation display unit 54 as a through image signal. The confirmation display unit 54 corresponds to the preview monitor 22 and displays a through image in real time based on the monitor image signal.

  The main display / operation unit 62 corresponds to the liquid crystal touch panel 20 and accepts an operation of a user U for determining a layout of a photograph to be output, a shutter operation, and the like. To the main control unit 80. Now, assuming that a shutter button (not shown) is pressed for shooting by the user U, the flashing is performed in the shooting direction of the camera 10 after elapse of a predetermined time of about several seconds based on an instruction from the main control unit 80. A flash is emitted from 11. At that time, an image signal output from the unit 52 as a signal representing a captured image of the user U is input to the main control unit 80, and the captured image is stored in a memory in the main control unit 80 or a hard disk device as an auxiliary storage device. Stored as data.

  The output unit 66 corresponds to a printer built in the main body unit 4, and is an image represented by the captured image data that has been subjected to image processing in accordance with the graffiti of the user U on the liquid crystal touch panel 20 based on an instruction from the main control unit 80. As a photo sticker (or photo card).

  The illumination unit 70 corresponds to the illumination devices 13L and 13R and the flash 11 in the main body unit 4, and is turned on / off and dimmed by the I / O control unit 72 based on an instruction from the main control unit 80.

  The I / O control unit 72 corresponds to an I / O control device built in the main body unit 4, and controls the illumination unit 70 based on an instruction from the main control unit 80. Further, an input signal such as a detection signal from a coin detection unit (not shown) described later is transferred to the main control unit 80.

  The main control unit 80 is built in the main body unit 4 and corresponds to a main control unit mainly composed of a computer including a CPU, a memory, a frame buffer, a timer, an auxiliary storage device, and the like. When the CPU executes the program, an instruction is given to each unit as described above in order to control each unit based on the operation signal or the like input as described above. Further, as will be described later, by synthesizing the captured image data and the image data to be used as a background, a predetermined background image becomes a background with respect to the image of the user U included in the captured image. A composite image that is a composite image is generated. The generated image is displayed on the main display / operation unit 62 by being written in the frame buffer. Further, a part of the image data is rewritten based on an operation signal for performing graffiti processing on the composite image, thereby generating a composite image that is an image obtained by drawing a predetermined image on the composite image. The generated composite image is displayed on the main display / operation unit 62. When the image composition as described above is completed, the composite image is then output from the photo output unit 66 as a photo sticker or the like. The outputted photographic sticker or the like is taken out from an outlet 28 provided on the side surface of the main body 4.

  In addition to the above components, a coin detection unit (not shown) for detecting coins inserted into the coin insertion slot 26 in the main body unit 4 is further provided in the main body unit 4, and the main control unit 80 Based on the detection result of the detection unit, each unit is controlled so as to allow the user U to play with the automatic photo creation apparatus, such as generating a composite image by photographing or drawing a captured image for a predetermined time. Since the control operation by the coin control unit and the main control unit 80 based on the detection result is the same as that of the conventional automatic photo creating apparatus and is well known, detailed description thereof will be omitted.

  The predetermined program executed in the main control device is provided by, for example, a CD-ROM that is a recording medium on which the program is recorded. That is, a CD-ROM serving as a recording medium for the predetermined program is mounted on a CD-ROM driving device built in the main control device as an auxiliary storage device, and the predetermined program is read from the CD-ROM to be an auxiliary storage device. Installed on the hard disk device. The predetermined program may be provided via a recording medium other than a CD-ROM or a communication line. When a predetermined operation for starting the automatic photo creating apparatus is performed, the predetermined program installed in the hard disk device is transferred to a memory in the main control device and temporarily stored therein, and the main control device It is executed by the internal CPU. Thereby, the control process of each part by the main controller is realized.

<3. Operation>
<3.1 Overall operation>
As described above, the operation of the main control unit 80 in the present embodiment is realized by the internal CPU executing the predetermined program. FIG. 4 is a flowchart showing the operation procedure of the main control unit 80 realized in this way. In the present embodiment, when a predetermined coin is inserted into the coin insertion slot 26, the automatic photo creation apparatus operates as follows.

  First, the main control unit 80 performs initialization processing of the apparatus, that is, initialization of each hardware and initialization processing of data stored in the main control unit 80 as a set value, and further for predetermined shooting. A timer is set so that a time-out occurs after the elapse of time (hereinafter referred to as “shooting time”), and the timer is started (step S10).

  Subsequently, the main control unit 80 performs a photographing process for photographing the user U (step S20). In this photographing process, the main control unit 80 determines whether or not a shutter button (not shown) or a shutter button displayed on the screen of the main display / operation unit 62 has been pressed based on the operation signal, and until the shutter button is pressed. stand by. It should be noted that when the shooting time ends without the shutter button being pressed, the processing when the shutter button is pressed is automatically performed.

  When the shutter button is pressed, the main control unit 80 notifies the user of the start of shooting on the screen of the main display / operation unit 62, and the shooting unit (camera) 52 takes a picture after a predetermined time of about several seconds. Send a signal (shutter signal). Accordingly, the illumination unit (flash) 70 emits flash in synchronization with the shutter operation of the photographing unit 52, and the main control unit 80 starts the capture process. That is, the main control unit 80 receives the image signal output from the imaging unit 52 as a signal representing the captured image of the user U illuminated by the flash from the illumination unit (flash) 70, and stores it in the memory or the auxiliary storage device. Store as a captured image file. Note that the capture process is executed each time the shutter button is pressed during a predetermined shooting time, and the shot image is temporarily saved as an image file.

  Next, the main control unit 80 determines whether the shooting has been completed by performing a predetermined number of times of shooting, the shooting indicating that the timer indicating the shooting time has timed out, or before the shooting time has elapsed. Even so, it is determined whether or not the photographing is finished by performing an operation for instructing the main display / operation unit 62 to finish the photographing process (step S30). When the shooting is finished, the process proceeds to the next step S40. If shooting is not completed, the process returns to step S20, and the above process is repeated until shooting is completed (S30 → S20 → S30).

  When shooting is finished, the main control unit 80 performs chroma key processing on the background image that is the additional image selected by the user's operation input to the main display / operation unit 62 (step S40). Details of the chroma key processing will be described later. Before (or after) this process, a layout process may be performed in which a plurality of photographed images (or a composite image including this) are appropriately arranged on the photo sticker to be output. This layout process is an image in which a photographed image or a composite image is arranged at a predetermined position by accepting an operation for determining how to place a photographed image within a predetermined time limit. A composite image for printing a photo sticker or the like is generated.

  Subsequently, graffiti processing is performed by the main control unit 80 and the main display / operation unit 62 (step S50). In this graffiti process, a predetermined time for graffiti processing (hereinafter referred to as “graffiti time”) is set by the main control unit 80, and the user U uses the liquid crystal touch panel 20 to perform the graffiti within this graffiti time. I do. Specifically, it is possible to select a pen tip icon or a stamp image that is displayed on the LCD touch panel 20 so as to select a photographed image and various icons for graffiti, for example, the thickness and color of a virtual graffiti pen. For example, a stamp icon or the like is displayed, and an operation by the user U is accepted. That is, the user U selects one of these various icons on the liquid crystal touch panel 20 (that is, moves the cursor over the displayed icon to confirm), or specifies a desired position on the display screen. Perform virtual graffiti by performing operations.

  Subsequently, in the printing process in step S60, the composite image generated by the graffiti process in step S50 (more specifically, the photographed image is laid out at an appropriate position and size by the layout process described above, and various graffiti is performed. Further, as will be described later, a composite image in which the background image is chroma key-combined is output. That is, the main control unit 80 instructs the output unit (printer) 66 to output a combined image (chroma key combined) with graffiti or the like. Thereby, the printer as the output unit 66 outputs the composite image as a photographic sticker.

  When the printing process in step S60 is finished, the series of processes shown in FIG. 4 is finished, and the process shown in FIG. 4 is executed until a coin corresponding to a predetermined amount is newly inserted into the coin slot 26. It will be in a waiting state. In this execution waiting state, the processing shown in FIG. 4 is in an execution waiting state until a coin corresponding to a predetermined amount is newly inserted into the coin insertion slot 26. When a coin corresponding to a predetermined amount is newly inserted in this execution waiting state, the same processing (steps S10 to S60) as described above is executed.

<3.2 Operation in chroma key processing>
Next, detailed processing contents of the above-described chroma key processing (step S40) will be described. FIG. 5 is a flowchart showing a detailed subroutine processing procedure of the report processing. In step S401 shown in FIG. 5, the main control unit 80 cuts out a predetermined first region from the captured image acquired in step S20 described above. That is, a new image in which only the portion is duplicated is generated. Hereinafter, a specific description will be given with reference to FIG.

  FIG. 6 is a diagram showing a shooting range when shooting a user in the shooting room with a camera. In FIG. 6, among the top surface, floor surface, back surface, and left and right side surfaces constituting the inner surface of the photographing room 2, the back surface 110 and the floor surface 120 are assigned the same single color (here, blue or green). . However, since the floor 120 has a predetermined angle that is far from being perpendicular to the shooting direction of the camera 10 and the projection direction of various illuminations, the floor 120 does not have a uniform color, and specifically, the closer to the back 110. The color is dark.

  The camera 10 has a predetermined angle of view determined by the optical structure, and the shooting range 101 is determined in advance by the angle of view and the shooting direction of the camera 10. In FIG. 6, the imaging range 101 includes the whole body of two users U and a part of the back surface 110 and the floor surface 120 that are the background of the user U. Therefore, the captured image acquired in step S20 is an image of the first area 101a that is a part of the captured image including the back surface 110 as the background of the user U (hereinafter simply referred to as “first area”). And a second region 101b image (hereinafter also simply referred to as “second region”) that is a part of a captured image including the floor 120 as a capturing range as the background of the user U. Yes.

  In step S401 described above, based on the fact that the position of the first area 101a included in the shooting range 101 is determined in advance by the angle of view, the shooting direction, and the like of the camera 10, only the image of the first area 101a from the shot image is displayed. Cut out (duplicated). Note that, when this image is cut out from the photographed image, there is no change in the photographed image itself, such as deleting a corresponding part of the photographed image.

  Next, in step S402, the main control unit 80 performs predetermined first image correction on the cut image of the first area 101a. This first image correction is a well-known image correction as preprocessing (for optimization) in chroma key composition, and is a cut-out image (here, the first region, more specifically, the first image correction). In accordance with the hue and brightness of the back surface 110 image included in the area, the hue and brightness of the first area are corrected to become ideal values for chroma key processing. Note that if the actually acquired captured image has a hue or brightness that is almost optimal for chroma key processing corresponding to the key color, the processing in step S402 can be omitted.

  Subsequently, in step S403, the main control unit 80 creates a first chroma key mask for the first region whose image has been corrected in step S402. This first chroma key mask is a well-known composite mask (alpha channel), and is a region having a key color (here, blue or green) in the target image (here, the first region), that is, the first chroma mask. This is data for making the area occupied by the image of the back surface 110 included in one area transparent and making the other areas non-transparent. In general, the composite mask is set to a value α indicating transparency from 0 (completely transparent) to 255 (completely nontransparent) for each pixel of the target image. Note that only 0 and 255 are used as α here.

  By using such a chroma key mask, as will be described in detail later in steps S407 and S408, the region of the background image corresponding to the non-transparent region (α = 255) of the photographed image (selected by the user as appropriate) is The area of the background image that is hidden (masked) and corresponds to the transparent area (α = 0) is synthesized so that it can be seen. Therefore, a composite image can be obtained as if a non-transparent area portion of the target captured image is cut out and the cut-out portion is pasted on the background image.

  In step S404, as in step S401 described above, the main control unit 80 cuts out (duplicates) a predetermined second region from the captured image acquired in step S20 described above. As described above, this does not change the captured image itself.

  Subsequently, in step S405, the main control unit 80 performs predetermined second image correction on the cut image of the second region 101b. This second image correction is an image correction different from the first image correction, and cannot be omitted unlike the first image correction. This will be described in detail below.

  This second image correction includes known image correction as preprocessing (for optimization) in chroma key composition, and the floor surface 120 corresponding to the second area 101b is predetermined with respect to the shooting direction of the camera 10. And image correction for correcting color change in the region caused by having an angle (that is, tilting from a substantially vertical direction).

  First, a well-known image correction as a pre-processing (for optimization) in chroma key composition included in the second image correction is performed by extracting a cut-out image (here, the second region, more specifically the second image). According to the hue and brightness of the image of the floor surface 120 included in the area (1), it is performed by the same method as the first image correction, but the image correction is not the same content as this. This is because the colors of the back surface 110 and the floor surface 120 corresponding to the first and second regions 101 a and 101 b are not the same due to the difference in angle with respect to the shooting direction of the camera 10. Further, even if the difference in material between the back surface 110 and the floor surface 120 is taken into consideration, the colors of the back surface 110 and the floor surface 120 are not the same.

  Further, the image correction for correcting the color change in the area included in the second image correction is not necessary in the first image correction. This is because the floor surface 120 corresponding to the second region 101b is not substantially perpendicular to the shooting direction of the camera 10, and the floor surface 120 is substantially perpendicular to the illumination projection direction. This is because it is considerably inclined (only has a shallow angle), so that a portion near the illumination is bright and a portion far from the illumination (that is, a portion near the back surface 110) is dark. This change in hue is typically a continuous change (gradation) according to the position. Therefore, by using a correction table, a correction pixel map, or a mathematical formula determined so as to compensate (resolve) this change, an image having ideal uniformity and hue for chroma key processing is obtained. Thus, the second region can be corrected. Note that the table and mathematical formulas for compensating for the above change are relatively easy by well-known methods such as three-dimensional graphics (for example, ray tracing) and various simulations based on the shape and material of the housing, the position of the camera and lighting, etc. However, it may be calculated based on actual measurement values. Note that a technique for appropriately obtaining an image for compensation will be described later in a modification.

  Subsequently, in step S406, the main control unit 80 creates a second chroma key mask for the second region whose image has been corrected in step S405. This second chroma key mask is also a well-known composite mask (alpha channel) in the same manner as the first chroma key mask, and is an area having the key color in the target image (here, the second area), that is, This is data for making the area occupied by the image of the floor surface 120 included in the second area transparent and making the other areas non-transparent. The key color for creating the second chroma key mask is blue or green, but is not the same as the key color for creating the first chroma key mask. As described above, since the floor 120 has a predetermined angle with respect to the shooting direction of the camera 10 and the projection direction of various illuminations, the back surface 110 that is substantially perpendicular to these does not have the same hue. is there. Of course, from the image correction in steps S402 and S405, when these hues are the same color, the same key color is obtained, but a detailed modification will be described later.

  Next, in step S <b> 407, the main control unit 80 displays some or all of the plurality of background images created in advance on the main display / operation unit 62, and accepts selection by the user from the main display / operation unit 62. . Specifically, various background images are displayed on the liquid crystal touch panel 20, and the user U selects one of these background images on the liquid crystal touch panel 20 (that is, the screen cursor is moved or the panel is moved with a finger). The operation is performed by pressing the button. As a result, the user can obtain a composite image in which the additional image selected by the user is a (virtual) background image, so that the playability of the apparatus can be improved.

  Subsequently, in step S408, the main control unit 80 applies the first and second chroma key masks created in steps S403 and S406, thereby providing a background image as an additional image selected by the user in step S407. Is synthesized with the chroma key for the captured image acquired in step S20 to create a composite image to be output. Thereafter, the chroma key process ends, and the process returns to the process shown in FIG.

<4. Effects>
As described above, according to the present embodiment, the back surface 110 provided behind the user U and the floor surface 120 which is a portion within the shooting range different from the back surface 110 are assigned key colors, Even in the case of different shades, chroma key masks corresponding to each can be created and chroma key processing can be performed.

<5. Modification>
<5.1 First Main Modification>
In the above embodiment, the image correction for correcting the color change in the region included in the second image correction is performed in step S405, and the correction table and the correction pixel determined to compensate (resolve) this change. By using a map or a mathematical expression, the second area is corrected so as to obtain an image having ideal uniformity and hue for chroma key processing, but instead of this configuration, In the first main modification example, an image used as a reference (reference) is appropriately acquired. Hereinafter, a description will be given with reference to FIG.

  FIG. 7 is a flowchart showing an operation procedure of the main control unit 80 for acquiring a background photographed image used as a reference used for compensation. In this modification, the operation of the main control unit 80 shown in FIG. 7 is performed in parallel with the operation shown in FIG. 4 described above. However, as will be described below, only the non-use time detection process and its determination process actually operate in parallel.

  In step S <b> 71 shown in FIG. 7, the main control unit 80 detects an elapsed time when the user U does not use the device. As a specific example, after a predetermined coin is inserted into the coin insertion slot 26 and the series of processes shown in FIG. 4 is completed, the apparatus is not used until the next coin is inserted and the series of processes is started. Elapsed time is measured.

  Next, in step S72, the main control unit 80 determines whether or not the elapsed time measured in step S71 exceeds a predetermined time (it will not be used by the user U for a while) and a sufficient time has passed. If sufficient time has elapsed, the process proceeds to the subsequent step S73. If sufficient time has not elapsed, the process returns to the detection process of step S71, and the process is repeated until sufficient time has elapsed. (S72 → S71 → S72).

  Subsequently, in step S73, the main control unit 80 acquires a background photographed image by automatically photographing the back surface 110 and the floor surface 120 to be the background. At this time, since the non-use time has passed sufficiently, it is assumed that the user U is almost certainly not inside the case, but it is also possible that the user U enters the case immediately before the main photographing process. It is rarely considered that the user U is included in the background photographed image obtained in the photographing process. If an image including the user U is used as a subject for chroma key processing as a background photographed image, the background becomes abnormal and the resultant composite image becomes abnormal.

  Therefore, in step S74, the main control unit 80 includes the abnormal color that is significantly different from the blue or green color of the back surface 110 and the floor surface 120 to be the background, typically the skin color of the user U, in the background photographed image. Detect whether or not.

  Subsequently, in step S75, the main control unit 80 determines whether or not an abnormal color is detected in step S74. If an abnormal color is detected (the user U is detected, the non-use time is detected). Is reset to zero), the process returns to step S71, and the above process is repeated until the background image captured again after the non-use time has elapsed (S75 → S71 → ... → S75). If no abnormal color is detected as a result of the determination, the background photographed image is stored, the non-use time is reset to zero, and the process returns to step S71.

  The background captured image thus stored is used for image correction for correcting a color change in the area included in the second image correction in step S405. For example, a correction pixel map including correction data for correcting the hue for each pixel so as to cancel the color change of the background photographed image is created based on the background photographed image, and ideal for chroma key processing based on this map. The second area is corrected so as to obtain an image having uniform uniformity and hue. This background photographed image may be used for image correction as preprocessing (for optimization) in chroma key composition in step S405 or step S402. Further, instead of the above configuration, a composite image may be created by utilizing the fact that the difference image obtained by subtracting the background captured image from the captured image corresponds to a person image. In this case, since the mask corresponding to the difference image is not the chroma key mask, the chroma key process is not necessary, but such a composite image creation process can be almost equated with the chroma key process except that the key color is not used.

  With this configuration, the back surface 110 and the floor surface 120 corresponding to the first and second regions 101a and 101b are different in angle with respect to the shooting direction of the camera 10, the difference in material, the difference in dirt, etc. (that is, the floor surface 120 is An image having an ideal uniformity and hue for chroma key processing, which is almost completely compensated for (difference that the user U steps on it becomes dirty and the color becomes dull or wears easily and the color is faint) The second region can be corrected so that

<5.2 Second Main Modification>
In the above embodiment, for example, as shown in FIG. 6, the imaging range 101 includes only the back surface 110 and the floor surface 120 in addition to the user U, but depending on the angle of view, installation position, and the like of the camera 10. Other surfaces of the inner surface of the housing may be photographed. Hereinafter, a description will be given with reference to FIGS.

  FIG. 8 is a side view of the automatic photo creating apparatus according to the second main modification. As can be seen from the comparison with the automatic photo creation apparatus in the above embodiment shown in FIG. 1, in the apparatus of this modification, the camera 10 is installed in the vicinity of the top surface and is photographed so that the user U is looked down from above. can do.

  FIG. 9 is a diagram showing a shooting range when shooting a user in the shooting room with a camera. As can be seen from comparison with the imaging range 101 shown in FIG. 6, the imaging range 201 shown in FIG. 9 includes the right side 130, the left side 140, and the top surface 150 in addition to the back surface 110 and the floor surface 120. It is.

  As described above, in the present modification, the imaging range 201 includes more surfaces than the configuration of the above embodiment in which the back surface 110 is included as the first region and the floor surface 120 is included as the second region 102. However, even in such a case, changes in the hue of each surface can be easily compensated by applying the configuration of the above embodiment. That is, the processes from step S404 to S406 shown in FIG. 5 are performed not only in the second region corresponding to the floor 120 but also in the third to fifth corresponding to the right side 130, the left side 140, and the top surface 150, respectively. By processing in the same manner for the area, chroma key processing corresponding to each surface can be performed in step S408.

<5.3 Other Modifications>
In the above-described embodiment, the same color is applied to the back surface 110 and the floor surface 120, and as a result, the hue is different, but the colors applied to these surfaces are not necessarily the same, For example, the back surface 110 may be blue and the floor surface 120 may be green. Even if the hues are the same as a result, the chroma key process can be performed in the same manner by the configuration of the above embodiment, but in this case, the same chroma key process can be performed by a known configuration. .

  In the above embodiment, the key color used in step S403 is different from the key color used in step S406, but the color of the area occupied by the back surface 110 in the first area in step S402 is the floor color in the second area. The image is corrected so as to be the color of the area occupied by the surface 120, or the color of the area occupied by the floor surface 120 in the second area is changed to the color of the area occupied by the back surface 110 in the first area in step S405. If the image is corrected, the key color used can be made the same. In addition, the key color can be made the same by correcting the color of the area occupied by the back surface 110 and the floor surface 120 so that the predetermined ideal key color is obtained in steps S402 and S405. By making the key colors the same in this way, it is not necessary to create a chroma key mask for each region in steps S403 and S406. For example, one chroma key mask common to the first and second regions is created in step S406. Since it only has to be created, the processing can be simplified.

  In the above embodiment, the key color is given to the inner plane of the casing, but the key is applied to a plane or curved surface (such as a chair or a decorative board) other than the inner surface of the casing, a curtain provided in the photographing range, or the like. The structure to which a color is attached | subjected may be sufficient.

  In the above embodiment, the non-use elapsed time is detected in step S71 shown in FIG. 7, and the progress is determined in step S72. Of course, this is an example. The shooting process in step S74 may be performed at the time or at the end, or the time after several days to several months have passed since the previous shooting (preferably the time when it is estimated that the camera is not used, for example, the process of non-use After the elapse of time, the photographing process may be performed when the apparatus is activated or when the apparatus is terminated.

1 is a side view of a photo creation device according to an embodiment of the present invention. It is a front view which shows the main-body part of the photo production apparatus which concerns on the said embodiment. It is a block diagram which shows the structure which looked at the principal part of the photo production apparatus which concerns on the said embodiment from the functional surface. It is a flowchart for demonstrating operation | movement of the main control part in the photo production apparatus which concerns on the said embodiment. It is a flowchart which shows the procedure of the chroma key process in the said embodiment. It is a figure which shows the imaging | photography range when image | photographing the user in the imaging | photography room in the said embodiment with a camera. 7 is a flowchart illustrating a procedure of an operation of a main control unit for acquiring a background captured image in the first main modification of the embodiment. It is a side view of the photograph production apparatus in the 2nd main modification of the said embodiment. It is a figure which shows the imaging | photography range when image | photographing the user in the imaging | photography room in the said modification with a camera. It is a figure which shows the imaging | photography range when image | photographing the user in the imaging | photography room in a conventional example with a camera.

Explanation of symbols

2 ... Photographing room 4 ... Main body 10 ... Camera 11 ... Flash 20 ... LCD touch panel 52 ... Photographing unit (camera)
54 ... Confirmation display (preview monitor)
62 ... Main display / operation unit (LCD touch panel)
66 ... Output unit (printer)
70: Illumination unit 72: I / O control unit (I / O control device)
80: Main control unit (main control device)
101, 201 ... shooting range 110 ... back surface 120 ... floor surface 130 ... right side surface 140 ... left side surface 150 ... top surface U ... user

Claims (11)

An automatic photo creation apparatus comprising: a photographing unit that acquires an image including a user who is a subject in a predetermined photographing region; and an output unit that outputs a composite image including a photographed image obtained by the photographing unit as a photograph,
A background presenting means that is a subject to be photographed by the photographing means and has a first photographing surface with a first color and a second photographing surface with a second color;
Image correction according to the color of the area occupied by the image of the first image-capturing surface is performed on the first area including the image of the first image-captured surface of the captured image, or the image capturing By performing image correction according to the color of the area occupied by the image of the second image-captured surface on the second area including the image of the second image-captured surface of the image, the first area Mask creating means for creating a predetermined chroma key mask corresponding to the color of the area occupied by the image of the imaged surface and the color of the area occupied by the image of the second imaged surface;
Using a chroma key mask created by the mask creating means, and performing a chroma key process for synthesizing a predetermined additional image to the captured image, thereby providing a chroma key processing means for generating the synthesized image. An automatic photo creation device. The first and second colors are substantially the same;
2. The automatic photo creation apparatus according to claim 1, wherein colors of areas occupied by the images of the first and second imaging surfaces are different.   The mask creating means creates the predetermined chroma key mask corresponding to the region of the same color by performing the image correction so that the regions occupied by the images of the first and second imaging surfaces are the same color. The automatic photo creating apparatus according to claim 1, wherein: The first surface to be photographed has a surface different from a plane substantially perpendicular to the photographing direction of the photographing means,
2. The mask creating means performs predetermined image correction on the first area so that a change in color of the area occupied by the image of the first surface to be photographed is eliminated. Automatic photo production device described in 1. The first surface to be imaged is inclined from a surface along a direction substantially perpendicular to the imaging direction of the imaging means;
The mask creating means applies the first region to the first region so that the change in color of the region occupied by the image of the first surface to be captured that changes according to the inclination of the first surface to be captured is eliminated. The automatic photo creation apparatus according to claim 4, wherein predetermined image correction is performed. The first imaged surface is a floor surface provided below the user, a top surface provided above the user, and a left side of the user among the inner surfaces of the housing that the user should enter. Any one or more of a left side provided and a right side provided on the right side of the user;
The said 2nd to-be-photographed surface is a back surface provided in the back direction of the said user from the installation position of the said imaging | photography means among the inner surfaces of the said housing | casing, The characterized by the above-mentioned. Automatic photo creation device. An operation input means for receiving an operation input by a user for selecting the predetermined additional image from a plurality of predetermined images;
2. The automatic photo creation apparatus according to claim 1, wherein the chroma key processing unit performs a chroma key process for synthesizing an additional image received by the operation input unit with respect to the photographed image. An image serving as a reference for image correction performed by the mask creating unit, the background image including the first and second regions and not included in the subject by the user at a predetermined time point. It further comprises sampling means for obtaining by the photographing means,
The mask creating means is based on the color of the area occupied by the image of the first imaged surface in the background imaged image acquired by the sampling means, or further, the second imaged image in the background imaged image. The automatic photo creation apparatus according to claim 1, wherein the image correction is performed based on a color of an area occupied by a surface image. An error detecting means for detecting an abnormality including a color to be included in the image of the user in the background photographed image acquired by the sampling means;
9. The automatic photo creating apparatus according to claim 8, wherein when an abnormality is detected by the error detection unit, the mask unit does not use a background photographed image in which the abnormality is detected for the image correction. An automatic photo creation method comprising: a photographing step for obtaining an image including a user who is a subject in a predetermined photographing region; and an output step for outputting a composite image including a photographed image obtained in the photographing step as a photograph,
The first object to be photographed in the photographing step, the first object in the background presentation means having a first surface to be imaged with a first color and a second surface to be imaged with a second color. The first area including the image of the imaging surface is subjected to image correction corresponding to the color of the area occupied by the image of the first imaging surface, or further including the image of the second imaging surface. By performing image correction according to the color of the area occupied by the image of the second imaging surface on the area of 2, the color of the area occupied by the image of the first imaging surface and the second A mask creation step for creating a predetermined chroma key mask corresponding to the color of the area occupied by the image of the imaged surface;
Using a chroma key mask created in the mask creating step, and performing a chroma key process for synthesizing a predetermined additional image with the captured image, thereby generating a synthesized key image. An automatic photo creation method.   A program for causing a computer to execute the automatic photo creating method according to claim 10.

Images