JP5349168B2 - Imaging apparatus, program, information storage medium, and imaging method - Google Patents

Description

  The present invention relates to a photographing apparatus, a program, an information storage medium, and a photographing method.

  2. Description of the Related Art Conventionally, a photo sticker manufacturing machine (an example of a photo printing apparatus) that edits and prints an image of a subject photographed by a photographing unit such as a CCD camera is known.

  In the photo sticker manufacturing machine, it is possible to edit a subject image and generate and enjoy a photo sticker having a high degree of freedom different from that of a normal photo.

  Recently, a service for transmitting a photo generated by a photo sticker manufacturing machine to a mobile phone or the like is also provided, and an image taken and edited can be used for a standby image of a mobile phone or the like.

JP 2006-162719 A

  In the above document 1, a “recommended pose” image corresponding to the selected shooting mode and shooting range is read from the shooting preparation image storage unit, and the read “recommended pose” image is provided in the vicinity of the camera, There is described a photo sticker vending machine that takes a pose by taking a pose with reference to a “recommended pose” image displayed by a user by displaying it on a captured image display unit that displays an image. However, variations in the resulting image were limited only by issuing a “recommended pose”.

  Recently, there is also a photo sticker vending machine that generates a single composite image (composite photo) by combining a plurality of photographed images to form a photo sticker. However, simply combining a plurality of photographed images makes the user bored, so it is preferable to provide a composite image rich in variations.

  For example, a subject such as a different pose is photographed a plurality of times, and a plurality of photographed images and a background image are combined to generate a combined image. In such a case, if a confirmation moving image obtained by synthesizing the background image and the currently captured image is displayed (live view display), it becomes easier to image the completed image.

  However, if the background image is simply displayed as the confirmation moving image, the position where the captured image reflection area of the background image (for example, the area where the captured image surrounded by a frame or the like is reflected) is displayed varies. According to the position, the user has to stand or squat.

  In addition, when trying to confirm the state where the photographed image is rotated and pasted on the background image with the confirmation moving image (live view image), the user has to shoot with a tilted posture.

  The present invention has been made to solve the above-described problems. According to some aspects of the present invention, a photographing device, a program, an information storage medium, and a photographing method that are excellent in a user interface at the time of photographing can be provided in a photographing device that can confirm a completed image with a confirmation moving image.

  (1) The present invention is a photographing apparatus that generates a composite image corresponding to a given layout pattern including at least one photographed image reflecting region that is a region in which a photographed image is reflected based on a photographed image photographed by the imaging unit. Then, the captured image of the subject being photographed by the imaging unit or the background extracted image obtained by extracting the background from the captured image and the confirmation background image corresponding to the given layout pattern or the captured image reflection area are synthesized in real time. A confirmation moving image generating means for generating a confirmation moving image to be displayed on the display unit, and a control for capturing a captured image at a given timing or a background extracted image obtained by extracting a background from the captured image as a captured image Rotating, changing the scale of the captured image and changing the scale according to the means to perform and the captured image reflection area of the layout pattern. A composite image that performs at least one of alignment and generates a composite image corresponding to a given layout pattern by combining one or a plurality of captured images and a background image corresponding to the layout pattern, or combining a plurality of captured images Generating means.

  The present invention also relates to a program that causes a computer to function as each of the above means. The present invention also relates to a computer-readable information storage medium that stores (records) a program that causes a computer to function as each of the above-described means. The present invention also relates to a method for controlling an imaging apparatus including steps for executing the above-described means.

  Here, the photographing device may be a photographic device or a photo sticker printing device, or an information device equipped with a photographing function (a portable communication device such as a mobile phone, a portable game machine, a portable information terminal, a personal computer, etc. For example, an imaging device such as an electronic camera may be used.

  The layout pattern includes at least one background image reflection area. In the background image reflection area, a captured image cut (masked) in a shape corresponding to the photographed image reflection area after being rotated and scaled corresponding to the photographed image reflection area is arranged.

  Photographing may be performed for each captured image reflection area. The confirmation moving image may be displayed for each shooting. For example, when there are a plurality of captured image reflection areas in the layout pattern, a different confirmation background image may be displayed for each captured image reflection area.

  The composite image generation unit rotates the captured image according to the attributes of the captured image reflection area (for example, the placement position, the rotation angle at the time of placement, the scale at the time of placement, etc.) that reflects the current shot image in the layout pattern. , At least one of scale change (reduction or enlargement) and alignment may be performed.

  The captured image reflection area may be an area framed by a frame image in the background image. In addition, a frame image or the like indicating the boundary between the captured image reflection area and another area may not be provided.

  The background image may include a pattern, a frame image, a pattern, a display object, or the like, or may be transparent.

  According to the present invention, it is possible to display a confirmation moving image using a confirmation background image corresponding to the background image reflection area for each background image reflection area.

  (2) In this photographing apparatus, the confirmation moving image generation means may generate a confirmation moving image using a confirmation background image that is different for each of the captured image reflection areas.

  (3) In this photographing apparatus, the confirmation moving image generating means includes the same pattern as a part of the background image to be combined with the captured image, and at least one of the arrangement, angle and scale of the pattern is A confirmation moving image may be generated using a confirmation background image different from the background image.

  (4) In this photographing apparatus, the confirmation moving image generation means rotates the same pattern as a part of the background image according to a relative angle of the photographed image reflection area with respect to the background image. A confirmation moving image may be generated.

  For example, the portion including the captured image reflection area of the background image is set to the same pattern as a part of the background image, and the captured image reflection area of the pattern is not inclined on the confirmation moving image display screen (for example, the live view screen) at the time of shooting. By displaying the confirmation moving image arranged by rotating the pattern so as to be in a state, the user does not tilt left and right depending on the angle of the captured image reflection area in the background image, and on the confirmation moving image screen The composite image of the background image and the photographed image can be confirmed.

  (5) In this photographing apparatus, the confirmation moving image generating means arranges the same pattern as the part of the background image in an enlarged or reduced manner according to the scale when reflecting the photographed image to the photographed image reflecting area. The confirmed moving image may be generated.

  (6) In this photographing apparatus, the confirmation moving image generation means may generate a confirmation moving image in which the photographed image reflection region is displayed in a predetermined region of the confirmation background image.

  For example, a portion including a captured image reflection region of a background image is set as a background image region included in the confirmation background image, and the region is aligned with the line of sight of the subject on a confirmation moving image display screen (for example, a live view screen) during photographing A confirmation moving image may be displayed so as to be located at a certain position.

  For example, if the shot image reflection area is at the top and bottom of the background image, when shooting an image corresponding to the top, a confirmation moving image is displayed so that the top of the background image is at the center of the screen, and at the bottom When shooting a corresponding image, a confirmation moving image in which the lower part of the background image is at the center of the screen may be displayed.

  In this way, the user can take a picture without standing or squatting depending on the position of the photographed image reflection area with respect to the background image, and can confirm a composite image of the background image and the photographed image on the confirmation moving image screen.

  (7) In this photographing apparatus, layout information including at least one of rotation information, reflection position information, and scale information when the captured image is combined with the background image is stored in association with the layout pattern or the captured image reflection region. The composite image generation means may combine with the background image by performing at least one of rotation and scale change on the captured image based on the layout information.

  (8) The photographing apparatus includes a confirmation background image storage unit that stores a confirmation background image to be displayed as a part of the confirmation moving image in association with the layout pattern or the captured image reflection region, The confirmation moving image generation means may use the confirmation background image stored in correspondence with the layout pattern or the captured image reflection area as the confirmation pattern corresponding to the layout pattern or the captured image reflection area. Good.

  In this way, the processing load is reduced as compared with the case where the confirmation background image is generated in real time.

  (9) In this photographing apparatus, the confirmation background image storage means includes the same pattern as a part of the background image, and at least one of the arrangement, angle and scale of the pattern is different from the background image. A confirmation background image may be stored.

  (10) In this photographing apparatus, the confirmation moving image generating means starts from a background image according to at least one of a relative position, rotation, and scale when reflecting the photographed image in the photographed image reflection area. A background image included in the confirmation background image is determined, a confirmation background image is generated by performing at least one of rotation and scale on the region, and the confirmation moving image is generated using the generated confirmation background image An image may be generated.

1 is a perspective view illustrating an example of an appearance of a photographic printing apparatus according to an embodiment. 1 is a perspective view illustrating an example of an appearance of a photographic printing apparatus according to an embodiment. 1 is a perspective view illustrating an example of an appearance of a photographic printing apparatus according to an embodiment. 1 is an example of a block diagram of a photo printing apparatus according to an embodiment. The figure for demonstrating the collage image of this Embodiment. FIGS. 6A to 6C are diagrams for explaining a shooting method in the collage shooting mode. The figure for demonstrating the imaging | photography method in collage imaging | photography mode. The figure for demonstrating the imaging | photography method in collage imaging | photography mode. FIGS. 9A to 9C are diagrams for explaining a background image for confirmation displayed corresponding to each captured image reflection area. 10A and 10B are diagrams for explaining layout information according to the present embodiment. The figure for demonstrating the method of arrange | positioning the captured image to a background image based on layout information. The figure for demonstrating the method of arrange | positioning the captured image to a background image based on layout information. The flowchart which shows the flow of a process of this Embodiment. The flowchart which shows the detailed flow of a captured image acquisition process. The flowchart which shows the detailed flow of a synthesized image generation process. 16A and 16B show captured images. The figure which shows a mode that a picked-up image is rotated and affixed. 18A and 18B show captured images. The figure which shows a mode that a captured image is reduced and affixed. The figure for demonstrating the imaging | photography method in collage imaging | photography mode. The figure for demonstrating the imaging | photography method in collage imaging | photography mode.

  Hereinafter, preferred embodiments of the present invention will be described with reference to the drawings.

1. Configuration of Imaging Device FIG. 1 is a perspective view illustrating an example of an appearance of an imaging device PM (an example of an image printing device) according to the present embodiment. As shown in FIG. 1, the photographing apparatus PM of the present embodiment includes a price receiving unit RM, an imaging unit CM, an editing operation receiving unit EM, and a photo sticker providing unit PB.

  The price accepting unit RM shown in FIG. 1 is the first contact with the user when using the imaging apparatus PM of the present embodiment, and is for inputting a price for using the imaging apparatus PM.

  The imaging unit CM illustrated in FIG. 1 is for capturing an image of a user who has input a price to the above-described price receiving unit RM, and is provided at a position different from the above-described price receiving unit RM. Yes. In the present embodiment, the imaging unit CM is covered with the sheet ST so that surrounding scenery does not appear in the background of the image, and so that the user being imaged cannot be seen from the surroundings.

  FIG. 2 is a perspective view showing the appearance of the imaging apparatus PM according to the present embodiment with the sheet ST removed, such that the front surface of the imaging unit CM is right front.

  As shown in FIG. 2, a camera CR that captures an image of a user located in the imaging area CA is provided at the upper front of the imaging unit CM. The camera CR can be realized by, for example, a CCD camera, a C-MOS camera, or a video camera. The camera CR captures a subject from the front direction and functions as an imaging unit.

  In addition, an imaging illumination unit CL that illuminates a user located in the imaging area CA is provided from the upper front to the middle of the imaging unit CM. The imaging illumination unit CL is controlled so as to be lit when the camera CR starts capturing an image of the user. Then, the camera CR is controlled to continue to illuminate the user while the user's image is captured.

  In addition, an imaging monitor CMN (an example of a display unit) that displays an image captured by the camera CR to a user located in the imaging area CA is provided in the front middle part of the imaging unit CM. In the present embodiment, when the camera CR starts capturing an image, the imaging monitor CMN displays the user's image located in the imaging area CA as a real-time moving image. As a result, the user can confirm the user's image captured by the camera CR on the imaging monitor CMN. The imaging monitor CMN is attached to the housing so that the display screen faces the user at a height looking down from the user's viewpoint. The video imaged by the camera CR is displayed on the imaging monitor CMN, and the user can check whether the user's own pose is appropriate by viewing the video image displayed on the imaging monitor CMN.

  The imaging monitor CMN is configured by a touch panel monitor in which touch sensor panels capable of detecting a contact position are stacked, and a user can input an operation by touching the imaging monitor CMN.

  Then, when a predetermined time has elapsed since the start of image capturing of the user, the image capturing of the user is terminated. Then, the user who is located in the imaging area CA is guided to move to the editing area EA (an example of a third area) that is the area on the front side of the editing operation reception unit EM.

  The editing operation accepting unit EM illustrated in FIG. 1 is for a user who has captured an image in the above-described imaging unit CM to perform an operation for editing the captured image, and is located at a position different from that of the above-described imaging unit CM. Is provided. In the present embodiment, the editing operation reception unit EM is covered with the sheet ST so that the user who is performing the editing operation cannot see from the surroundings. In particular, in the present embodiment, a first editing operation receiving unit EM1 and a second editing operation receiving unit EM2 are provided as the editing operation receiving unit EM. FIG. 3 is a perspective view showing the appearance of the photographing apparatus PM of the present embodiment in a state where the sheet ST is removed so that the front surface of the first editing operation accepting unit EM1 is right front.

  As shown in FIG. 3, a user located in an editing area EA1 (an example of a third area) that is an area on the front side of the first editing operation accepting unit EM1 is located at the front center part of the first editing operation accepting part EM1. On the other hand, there is provided a touch panel type editing monitor EMN that displays an image determined as a still image to be printed on a sticker and receives an operation for editing the displayed image.

  In the present embodiment, touch pens TP are prepared on the left and right of the editing monitor EMN, respectively, and the user combines the determined image with the determined image by bringing the tip of the touch pen TP into contact with an arbitrary position of the editing monitor EMN. Or select a pattern or design template (stamp image) and place it on the image to edit the still image to be printed on the sticker.

  In the present embodiment, a second editing operation receiving unit EM2 having the same configuration as the first editing operation receiving unit EM1 is provided on the back side of the first editing operation receiving unit EM1 shown in FIG. The image displayed while displaying the image determined as the still image to be printed on the sticker for the user located in the editing area EA2 (an example of the fourth area) that is the area on the front side of the editing operation accepting unit EM2 of No. 2 Accepts operations for editing.

  Then, when a predetermined time elapses after the acceptance of the image editing operation is started or the end of the editing operation is selected by the user's operation, the acceptance of the image editing operation is finished. Then, the user located in the editing area EA1 or the editing area EA2 moves to a take-out area PA (an example of a fifth area) that is a front side area of the photo sticker providing unit PB (an example of a printed matter providing unit). To guide.

  As shown in FIG. 3, the left side surface of the first editing operation accepting unit EM1 is provided with a photo sticker providing unit PB that provides a user with a photo sticker (an example of a printed material) in which an edited image is printed on a sticker. It has been. In the present embodiment, a photo sticker outlet PO is provided in front of the photo sticker providing part PB so as to provide a photo sticker to a user located in the take-out area PA.

  FIG. 4 is an example of a block diagram of the photographing apparatus PM of the present embodiment. Note that the imaging apparatus does not have to include all the components (each unit) in FIG. 4, and may have a configuration in which some of them are omitted.

  The imaging unit 198 is for photographing a subject and can be realized by, for example, a CCD camera, a CMOS camera, or a video camera.

  An operation unit 160 (a pointing device, a lever, a button, or the like) is used by the user to perform a selection operation or an editing operation of the editing content selection menu.

  The storage unit 170 (RAM) is a work area such as the processing unit 100 or the communication unit 196.

  An information storage medium 180 (a computer-readable medium such as a CD, DVD, HDD, or ROM) stores programs and data. The information storage medium 180 stores a program for causing a computer to function as each unit of the present embodiment (a program for causing a computer to execute processing of each unit).

  The display 190 displays an image, and is, for example, the imaging monitor CMN or the editing display shown in FIG.

  The imaging monitor CMN uses, for example, a touch panel or the like, and inputs various instructions (for example, selection of a collage background, selection of a shooting mode, etc.) when a user touches an input button displayed on a display with a finger. It is configured to be able to do.

  As the editing display, for example, a liquid crystal tablet, a touch panel, or the like is used, and a handwritten image can be input by drawing a character or a figure by bringing the tip of a touch pen (an example of a pointing device) into contact with the display surface. . In addition, a stamp image such as a star mark or a heart mark prepared in advance is selected, and the stamp image can be input by bringing the tip of the touch pen into contact with the display surface.

  The sound output unit 192 outputs sounds such as sounds and game sounds.

  The portable information storage device 194 stores user personal data, save data for editing, and the like.

  The printing unit 195 performs processing for printing the captured image on a print medium (seal paper, photographic paper, plastic plate, or recording layer). In this case, there are various printing methods such as a sublimation type, a thermal transfer type, an ink jet method, and a laser printing method. As a printer for printing, an ink jet printer, a laser printer, a sublimation printer, a thermal transfer printer, a melt printer, a thermal printer, a photographic paper printer, an instant film printer, or the like may be used.

The processing unit 100 (processor) performs various processes such as a game process, an image generation process, and a sound generation process based on operation data from the operation unit 160, a program, and the like. In this case, the processing unit 100 performs various processes using the main storage unit 172 in the storage unit 170 as a work area. The function of the processing unit 100 can be realized by hardware such as various processors (CPU, DSP, etc.) or ASIC (gate array, etc.) and a program (game program).

  The processing unit 100 includes a collage shooting control unit 110, an editing processing unit 120, an image generation unit 130, and a sound generation unit 140.

  The editing processing unit 120 performs editing processing on the captured image or the collage image generated by the collage shooting control unit 110 based on the input information for editing from the operation unit 160.

  The image generation unit 130 generates an image based on the result of shooting or editing, and outputs the image to the display 190.

  The sound generation unit 140 performs sound processing based on the results of various processes performed by the processing unit 100, generates game sounds such as BGM, sound effects, or sounds, and outputs the game sounds to the sound output unit 192.

  The collage shooting control unit 110 includes a background extracted image generation unit 112, a confirmation moving image generation unit 114, a captured image holding control unit 116, and a composite image generation unit 118. The confirmation moving image generation unit 114 captures a photographed image of the subject being photographed by the imaging unit 198 or a background extracted image obtained by extracting the background from the photographed image and a given layout pattern or a confirmation background corresponding to the photographed image reflection region. A confirmation moving image is generated to be combined with the image in real time and displayed on the display unit. The captured image holding unit 116 performs control to hold a captured image at a given timing or a background extracted image obtained by extracting a background from the captured image as a captured image. The composite image generation unit 118 performs at least one of rotation, scale change, and alignment on the captured image in accordance with the captured image reflection area of the layout pattern, and corresponds to one or a plurality of captured images and layout patterns. A composite image corresponding to a given layout pattern is generated by combining the background image or a plurality of captured images. In addition, the background extracted image generation unit 112 generates a background extracted image in which the background is extracted from the captured image.

  The confirmation moving image generation unit 114 may generate a confirmation moving image using a different confirmation background image for each captured image reflection area. The confirmation moving image generation unit 114 includes the same pattern as a part of the background image to be combined with the captured image, and at least one of the arrangement, angle, and scale of the part of the image is the background image. A confirmation moving image may be generated using different confirmation background images. Further, the confirmation moving image generation unit 114 generates a confirmation moving image in which the same pattern as the part of the background image is rotated according to the degree of relative rotation of the captured image reflection area with respect to the background image. May be. In addition, the confirmation moving image generation unit 114 generates a confirmation moving image in which the same pattern as the part of the background image is enlarged or reduced according to the scale when the captured image is reflected in the captured image reflection area. May be. The confirmation moving image generation unit 114 may generate a confirmation moving image in which the captured image reflection area is displayed in a predetermined area of the confirmation background image.

  In addition, layout information including at least one of rotation information, reflection position information, and scale information when the captured image is combined with the background image is stored in association with the layout pattern or the captured image reflection area (for example, stored in the storage unit 170). Alternatively, the composite image generation unit 118 may perform at least one of rotation and scale change on the captured image based on the layout information and combine it with the background image.

  Further, a confirmation background image storage unit (for example, the storage unit 170 may store the confirmation background image to be displayed as a part of the confirmation moving image in association with the layout pattern or the captured image reflection region thereof. The confirmation moving image generation unit 114 includes a confirmation background image stored in correspondence with the layout pattern or the captured image reflection area, as a confirmation background image corresponding to the layout pattern or the captured image reflection area. May be used. The storage unit 170 may store a confirmation background image that includes the same pattern as a part of the background image and that is different from the background image in at least one of the arrangement, angle, and scale of the pattern.

  The confirmation moving image generation means unit 114 includes a background image included in the confirmation background image from the background image in accordance with at least one of the relative position, rotation, and scale when reflecting the captured image. May be determined, at least one of rotation and scale may be performed on the region to generate a confirmation background image, and the confirmation moving image may be generated using the generated confirmation background image.

  The communication unit 196 performs processing for transmitting or receiving data via a network such as the Internet. More specifically, the communication unit 196 transmits data generated from the captured image of the camera to an external server or the like via the network. In this way, the user (subject) can download image data of the subject himself / herself from the server to the personal computer of his / her home. Based on the downloaded image data, it is possible to print and output an image on paper or a sticker using a printer at home. It is also possible to store an image captured by the user in a server, download it to the image capturing apparatus, print it, and output it.

  The function of the communication unit 196 can be realized by hardware such as a communication device (IC), a communication program, or the like. Note that either the printing unit 195 or the communication unit 196 may be omitted.

  Note that the photographing apparatus of the present embodiment may be a fixed housing or a portable device.

  Note that the subject photographed by the photographing apparatus of the present embodiment is not limited to a human being.

  Further, the photographing apparatus of the present embodiment can be used for a game apparatus of a type that uses a user's photographed image for a game. That is, it is possible to cause the photographing apparatus of this embodiment to function as an image capturing device of a game device.

2. Collage shooting process 2-1. Collage Image In this embodiment, a collage shooting mode is provided in which a shot image is combined with a background image to generate a collage image.

  FIG. 5 is a diagram for explaining the collage image of the present embodiment.

  The collage image 330 according to the present embodiment is a composite image created by combining one or a plurality of captured images 310 and 320 and a background image 300 corresponding to the layout pattern of the collage image. The background image 300 may be an image with a pattern, a picture, or a frame image (303 and 305 in FIG. 5), or may be a transparent image. The background image includes one or a plurality of captured image reflection areas 302 and 304. The captured image reflection areas 302 and 304 are areas (for example, areas to be pasted) where the captured images 310 and 320 are reflected. Here, the captured image may be a captured image or a background extracted image obtained by extracting a background from the captured image. Moreover, the image obtained by reversing the left and right of the photographed image may be used.

  In this embodiment, the captured image can be pasted on the captured image reflection areas 302 and 304 by rotating or enlarging or reducing the captured image. Here, an example is shown in which the captured images 310 and 320 are rotated and pasted in the captured image reflection areas 302 and 304 of the background image 300 and reflected in the captured image reflection areas 332 and 334 of the collage image 330. In the captured image reflection area of the collage image, images obtained by rotating the captured images 310 and 320 and cutting out the frames (cut out by the frames) are pasted (see 332 and 334). The frame can be realized by preparing a mask image corresponding to the frame and making the portion of the captured image outside the frame transparent using the mask image.

2-2. Shooting Method in Collage Shooting Mode FIGS. 6A to 6C, 7 and 8 are diagrams for explaining the shooting method in the collage shooting mode.

  6A to 6C show sample images that can be selected in the collage shooting mode. The sample image is an image indicating a layout pattern and a completed image of a collage image to be created, and is an image in which a model image is combined with a background image according to the layout pattern. The user can select a desired sample image and create a collage image.

  FIG. 6A is a first collage sample image 210 of the first layout pattern. With the first collage sample image 210, the background image 212 and the completed image corresponding to the first layout pattern can be grasped. The background image 212 corresponding to the first layout pattern includes captured image reflection areas 214 and 216. Sample model images are displayed in the captured image reflection areas 214 and 216. Here, as described with reference to FIG. 5, captured images (310 and 320 in FIG. 5) corresponding to the captured images (subject images) are displayed. Etc.) is pasted (reflected). Here, the heart mark that outlines the captured image reflecting areas 214 and 216 is arranged in a rotated state (a state in which the heart mark in the reference state is rotated with respect to the vertical side of the rectangle of the outer frame of the background image). The captured image is rotated and pasted in the captured image reflection areas 214 and 216.

  FIG. 6B is a second collage sample image 220 showing the second layout pattern. By using the second collage sample image 220, the background image 222 and the completed image corresponding to the second layout pattern can be grasped. The background image 222 corresponding to the second layout pattern includes captured image reflection areas 224 and 226. Sample model images are displayed in the photographed image reflecting areas 224 and 226, and here, for example, a captured image (a photographed image of a subject or a background-extracted image thereof or a background image shown in FIGS. 16A and 16B) is displayed. An image obtained by horizontally inverting them is pasted (reflected). Here, the rectangle that forms the frame of the captured image reflection area 224 (here, the image with the upper right and lower right edges cut off) is rotated (referenced to the vertical side of the rectangle of the outer frame of the background image) 17, the captured image is rotated and pasted in the captured image reflection area 224 as shown in FIG. 17.

  FIG. 6C is a third collage sample image 230 of the third layout pattern. The third collage sample image 230 includes a background image 232 (including a box image 233) and captured image reflection areas 234 and 236. In the photographed image reflection areas 234 and 236, sample model images are displayed without being particularly framed. For example, captured images as shown in FIGS. 18A and 18B are pasted here. (Reflected). Here, since the captured image reflection area 234 and the captured image reflection area 236 have different scales when the captured image is pasted, the captured image is reduced and pasted to the captured image reflection area 236 as shown in FIG. .

  7 and 8 are diagrams for explaining a shooting screen when the first collage sample image is selected (images displayed on the imaging monitor CMN of FIG. 1 at the time of shooting. First Collage Sample) Since the image has two captured image reflection areas, at least two capturing operations are performed in order to capture the subject image to be reflected in each captured image reflection area.

  FIG. 7 shows the state of the first shooting screen 200-1 (the shooting screen when shooting the subject image to be reflected in the first shot image reflection area). A collage sample image 210-1 is displayed on the upper left side of the shooting screen 200-1. By viewing this, the user can imagine a completed image to be taken and pose it. The collage sample image 210-1 has two shot image reflection areas. However, since the subject image to be reflected in the first shot image reflection area 214-1 is shot at the first shooting, the first shot image reflection is performed. It is good also as a structure which displays the part corresponding to the area | region 214-1 in the display mode different from another area | region. For example, the first captured image reflection area 214-1 may be displayed normally and the other areas may be displayed in a shaded or translucent state.

  Reference numeral 250-1 denotes a live view image (an example of a confirmation moving image) at the time of the first shooting. In the live view image 250-1, a confirmation background image 252 corresponding to the first captured image reflection area 214-1 is displayed. In the captured image reflection area 254 of the confirmation background image 252, a background extracted image obtained by extracting the background from the currently captured subject image is displayed as a real-time moving image.

  The background extracted image may be generated by, for example, reversing the photographed image left and right. In this way, the background-extracted image becomes the same image as when there is a mirror in the front, so if the camera and the display unit are in front of the user, this way is better for the user's sense of position and direction. It is easy to align.

  The background extracted image may be generated in real time by capturing an image of a subject being photographed by the imaging unit as a current image with the extracted color as a background, and extracting the background portion by extracting from the current image.

  The extraction process of the extraction color may be realized by, for example, chroma key processing or may be realized by mask processing. The chroma key is a technique for extracting an area having a specific color (extracted color) from an image and embedding another image therein. The method of embedding another image is realized by setting the α value (a value indicating transparency) of an area having a specific color to 1, and performing translucent synthesis (for example, α blending processing) with another image. be able to.

  Mask processing is a process of creating a mask image in which all pixel values of a specific area of the image (extracted color area in the present invention) are set to 1, and all other parts are set to 0. In this method, the image A is embedded in the portion and the image B is embedded in the portion 1 of the mask image to synthesize both images.

  Here, the confirmation background image 252 corresponding to the first captured image reflection area 214-1 may be prepared in advance, or may be generated in real time based on the background image of the first layout pattern.

  The confirmation background image corresponding to the layout pattern or the captured image reflection area may be an image obtained by cutting and rotating a part of the background image and pasting it at a predetermined position on the display screen. One of the background images that has been rotated, aligned, reduced, or enlarged so that the captured image reflection area of the background image comes to a position that matches the line of sight of the subject on the moving image display screen for confirmation during shooting (for example, a live view screen) A background image for confirmation including a part may be displayed.

  Here, the first captured image reflection region 214-1 is a region near the upper left of the background image, but in the confirmation background image 252, the region 254 corresponding to the first captured image reflection region is the confirmation background image 252. It is displayed near the center of. Assuming that the subject is displayed near the center of the live view image with the subject facing the camera in a normal posture, the user views the user's image in a heart shape by facing the camera in the normal posture. Can fit in the frame. Therefore, it is not necessary for the user to turn his / her body in the upper left direction in accordance with the position of the first captured image reflection area 214-1 of the background image, and a user interface that is easy to use can be provided when capturing a collage image.

  The first captured image reflection area 214-1 of the background image is in a state where the heart-shaped frame is tilted, but the confirmation-type background image 252 is displayed in a state where the heart-shaped frame is not tilted (the background image The outer frame is displayed as a tilted image). As described above, in the present embodiment, when the captured image is rotated by a predetermined rotation angle θ and reflected in the captured image reflection area (see 214) as shown in the collage sample image 210-1, it is indicated by 250-1. Thus, in the live view image, the background image is displayed in a state tilted in the direction opposite to the predetermined rotation angle θ.

  Therefore, since the user can place his / her image in a heart-shaped frame that does not tilt by facing the camera in a normal state, the live view image and the background image can be displayed without tilting the body at the time of shooting. A composite image of the photographed image can be confirmed.

  FIG. 8 shows the state of the second shooting screen 200-2 (shooting screen when shooting a subject image to be reflected in the second shot image reflection area). For example, a collage sample image 210-2 is displayed on the upper left side of the shooting screen 200-2. By viewing this, the user can imagine a completed image to be taken and pose it. The collage sample image 210-2 has two shot image reflection areas. However, since the subject image to be reflected in the second shot image reflection area is shot in the second shooting, the second shot image reflection area 216- It is good also as a structure which displays the part corresponding to 2 in the display mode different from another area | region. For example, the second captured image reflection area 216-2 may be displayed normally and the other areas may be displayed in a shaded state or a translucent state.

  Reference numeral 250-2 denotes a live view image at the time of the second shooting. In the live view image, a confirmation background image 256 corresponding to the second captured image reflection area 216-1 is displayed. In a region 258 corresponding to the captured image reflection region of the background image 256, a background extracted image obtained by extracting the background from the currently captured subject image is displayed.

  The background extracted image may be generated by, for example, reversing the photographed image left and right. In this way, the background-extracted image becomes the same image as when there is a mirror in the front, so if the camera and the display unit are in front of the user, this way is better for the user's sense of position and direction. It is easy to align.

  Here, the confirmation background image 256 corresponding to the second captured image reflection area 216-1 may be prepared in advance, or may be generated in real time based on the background image of the first layout pattern.

  Here, the second captured image reflection area 216-2 is an area near the lower right of the background image. In the confirmation background image 256, the area 258 corresponding to the second captured image reflection area is the confirmation background image. It is displayed near the center of 256. Assuming that the subject is displayed near the center of the live view image with the subject facing the camera in a normal posture, the user views the user's image in a heart shape by facing the camera in the normal posture. Can fit in the frame. Therefore, it is not necessary for the user to turn his / her body in the lower right direction according to the position of the second captured image reflection area 216-2 of the background image, and a user interface that is easy to use can be provided at the time of collage image capturing.

  As described above, according to the present embodiment, for example, when the captured image reflection areas are located above and below the background image, an image corresponding to the upper captured image reflection area 214-1 as illustrated in FIG. 7 is captured. In such a case, a live view image in which the upper part of the background image is in the center is displayed, and when an image corresponding to the lower captured image reflection area 216-2 as shown in FIG. A live view image is displayed in such a manner that the vicinity of the captured image reflection area is at the center. Therefore, when the user performs shooting corresponding to the upper captured image reflection area, the user does not perform squatting or the like in the live view image when performing shooting corresponding to the lower captured image reflection area. The composite image of the background image and the photographed image can be confirmed.

  The captured image reflection areas 214-1 and 216-2 are in a state where the heart-shaped frame is tilted, but in the confirmation background image 258, the heart-shaped frame is displayed without being tilted (the outer frame of the background image). Is displayed as a tilted image). As described above, in the present embodiment, when the captured image is rotated by a predetermined rotation angle θ and reflected in the captured image reflection area (see 214) as shown in the collage sample image 210-1, it is indicated by 250-1. Thus, in the live view image, the background image is displayed in a state tilted in the direction opposite to the predetermined rotation angle θ.

  Therefore, since the user can place his / her image in a heart-shaped frame that does not tilt by facing the camera in a normal state, the live view image and the background image can be displayed without tilting the body at the time of shooting. A composite image of the photographed image can be confirmed.

  FIGS. 9A to 9C are diagrams for explaining the background image for confirmation displayed corresponding to each captured image reflection area.

  Reference numeral 350 in FIG. 9A denotes a background image corresponding to the first layout pattern. FIG. 9B shows a first confirmation background image 360 corresponding to the first captured image reflection area 352 of the first layout pattern. The first confirmation background image 360 includes an image (a portion surrounded by a frame 357 and a region including the first captured image reflection region 352) corresponding to the first layout pattern (FIG. 9). (See (A) 353). The first confirmation background image 360 includes a pattern indicated by 357 in FIG. However, at least one of the arrangement, angle, and scale of the same pattern as a part of the background image in the first confirmation background image 360 is different from the background image 350.

  FIG. 9C shows a second confirmation background image 370 corresponding to the second captured image reflection area 354 of the first layout pattern. The second confirmation background image 370 includes an image (FIG. 9) including a part of the background image 350 corresponding to the first layout pattern (a region surrounded by the frame 359 and including the second captured image reflection region 354). (See (A) 355). The second confirmation background image 370 includes a pattern indicated by 355 in FIG. However, at least one of the arrangement, angle, and scale of the same pattern as a part of the background image in the second confirmation background image 370 is different from the background image 350.

  The pattern of the portion surrounded by the frames 357 and 359 (hereinafter referred to as “patterns 357 and 359”) is determined in accordance with at least one of the relative position and angle of the captured image reflecting area with respect to the background image and the scale when reflecting the captured image. You may make it do.

  Further, using the background image 350 stored in association with the layout pattern when the layout pattern is selected, the relative position and angle of the captured image reflection areas 352 and 354 with respect to the background image 350 and the scale at which the captured image is reflected. Depending on at least one of the patterns, the patterns 357 and 359 may be determined, and at least one of rotation and scale change may be performed on the patterns 357 and 359 to generate the confirmation background images 360 and 370.

  Further, for example, a confirmation background image corresponding to the captured image reflection area may be created in advance by the above method and stored in association with the layout pattern or the captured image reflection area.

  20 and 21 are diagrams for explaining a shooting screen when the third collage sample image is selected (an image displayed on the imaging monitor CMN of FIG. 1 at the time of shooting. Third Collage Sample) Since the image has two captured image reflection areas, at least two capturing operations are performed in order to capture the subject image to be reflected in each captured image reflection area.

  FIG. 20 shows the state of the first shooting screen 600-1 (shooting screen when shooting a subject image to be reflected in the first shot image reflection area). For example, a collage sample image 610-1 is displayed on the upper left side of the shooting screen 600-1. By viewing this, the user can imagine a completed image to be taken and pose it. The first collage sample image 610-1 has two captured image reflection areas. However, since the subject image to be reflected in the first captured image reflection area is captured at the time of the first shooting, the first captured image reflection is performed. It is good also as a structure which displays the part corresponding to the area | region 614-1 in the display mode different from another area | region. For example, the first captured image reflection area 614-1 may be displayed normally and the other areas may be displayed in a shaded state or a semi-transparent state.

  Reference numeral 650-1 denotes a live view image at the time of the first shooting. In the live view image 650-1, a confirmation background image 652 (background image on which a box image is displayed) corresponding to the first captured image reflection area 614-1 is displayed. In the area corresponding to the captured image reflection area 654 of the confirmation background image 652, a background extracted image obtained by extracting the background from the currently captured subject image is displayed.

  The background extracted image may be generated by, for example, reversing the photographed image left and right. In this way, the background-extracted image becomes the same image as when there is a mirror in the front, so if the camera and the display unit are in front of the user, this way is better for the user's sense of position and direction. It is easy to align.

  FIG. 21 shows a state of the second shooting screen 600-2 (shooting screen when shooting a subject image to be reflected in the second shot image reflection area). For example, a collage sample image 610-2 is displayed on the upper left side of the shooting screen 600-2. By viewing this, the user can imagine a completed image to be taken and pose it. The collage sample image 610-2 has two photographed image reflecting areas. However, since the subject image to be reflected in the second photographed image reflecting area is photographed at the second photographing, the second photographed image reflecting area 616- is taken. It is good also as a structure which displays the part corresponding to 2 in the display mode different from another area | region. For example, the second captured image reflection area 616-2 may be displayed normally and the other areas may be displayed in a shaded state or a translucent state.

  Reference numeral 650-2 denotes a live view image at the time of the second shooting. In the live view image, a confirmation background image 656 corresponding to the second captured image reflection area 616-2 is displayed. In a region 658 corresponding to the captured image reflection region of the confirmation background image 656, a background extracted image obtained by extracting the background from the currently captured subject image is displayed.

  The background extracted image may be generated by, for example, reversing the photographed image left and right. In this way, the background-extracted image becomes the same image as when the front is bent, so that the user's position and direction sense are better (when there is a camera and display in front of the user). It is easy to align.

  As shown in the collage sample image 610-2, the image captured here is pasted in the captured image reflection area in a state of being reduced compared to the first image. For this reason, the box object is displayed larger in the second live view image 650-2 than in the first live view image 650-1. For this reason, the user can confirm the composite image of the captured image in the completed image in the live view image.

2-3. Method for Combining Layout Information and Collage Image FIGS. 10A and 10B are diagrams for explaining layout information according to the present embodiment. FIGS. 11 and 12 are diagrams for explaining a method of synthesizing a captured image and a background image based on layout information.

  In this embodiment, as shown in FIG. 10A, background image data 422 constituting the background of each layout pattern is stored in association with the layout pattern 410 of the collage image. A background image ID 420 for identifying the background image data 422 may be stored in association with the corresponding layout pattern 410.

  Further, layout information corresponding to each captured image reflection area 430 may be stored in association with the layout pattern 410 and the captured image reflection area 430. The reflection position information 440, the rotation information 450, the scale information 460, the mask information 470, and the confirmation background image 580 may be stored as layout information. For example, as shown in FIG. 11, the reflection position information 440 is given by the coordinates C11 and C12 of the position where the upper left vertex of the captured image 510 is arranged in the local coordinate system with the upper left vertex of the background image of the layout pattern as the origin O. You may make it (refer FIG. 11 (A)). The rotation information 450 is a rotation angle θ1 when the captured image 510 is arranged on the background image (for example, the rotation angles 510 and 510 ′ when the captured image 510 is rotated to be in the state 510 ′ and arranged on the background image). You may make it give by (theta) 1) (refer FIG. 11 (B)). The scale information 460 is a value indicating how much the image is scaled with respect to the reference image when the captured image is pasted on the background image. For example, 1 may be set when pasting on the background image at the same magnification, less than 1 may be set when pasting after reduction, and a larger value may be set when pasting after enlargement. For example, when the captured image 560 is used as a reference image (assuming that the reference image is the same size as the background image) and a collage image as shown in FIG. 19 is generated, the captured image 560 is reduced as shown in FIG. The scale-changed image (here, the reduced image 560 ′) is pasted. The ratio of the reference image corresponding to the scale change image at this time may be stored as scale information.

  Also, for example, when reflecting a captured image in each captured image reflection area, when determining the reflection area according to the shape of the frame, for specifying the reflection area from the captured image or the captured image according to the shape of the frame Mask information 470 may be stored. The mask information 470 may be stored as flag information for specifying reflection on / off corresponding to each pixel of the captured image.

  The confirmation background image 480 is an image prepared for each captured image reflection area, and includes, for example, the same pattern as a part of the background image as shown in FIGS. 9B and 9C. At least one of the arrangement, angle and scale of the image is an image different from the background image. For example, an image in which the same pattern as a part of the background image is rotated and arranged according to the relative angle of the captured image reflection area with respect to the background image may be used. Further, for example, an image in which the same pattern as the part of the background image is enlarged or reduced in accordance with the scale at the time of reflecting the captured image to the captured image reflection area may be used. Further, for example, the captured image reflection area may be an image arranged in a predetermined area of the confirmation background image.

  FIG. 13 is a flowchart showing the processing flow of the present embodiment.

  A layout pattern selection screen on which sample images of a plurality of layout patterns are displayed is displayed, and a desired layout pattern selection input is accepted on the layout pattern selection screen (step S10). Next, according to the selected layout pattern, an image of the subject is taken and a captured image is acquired (step S20). Next, based on the layout information corresponding to the selected layout pattern, a composite image is generated by combining the background image and the captured image (step S30). Next, on the editing screen on which the composite image is displayed, an editing input is accepted (step S40), and a completed image is generated based on the editing input (step S50).

  FIG. 14 is a flowchart showing a detailed flow of the captured image acquisition process in step S20 of FIG.

  Reads the background image for confirmation corresponding to the captured image reflection area of the selected layout pattern, and synthesizes the background image for confirmation and the background extracted image obtained by extracting the background portion from the image of the subject being captured by the imaging unit in real time Then, a live view image is generated (step S110).

  When there is a captured image confirmation input (step S120), the background extracted image at the captured image confirmation input is held as a captured image corresponding to the captured image reflection area of the selected layout pattern (step S130).

  Until the subject images corresponding to all the captured image reflection areas of the selected layout pattern are captured, the processes of steps S110 to S130 are performed for the captured image reflection areas for which the corresponding captured image has not yet been acquired (step S140). .

  FIG. 15 is a flowchart showing a detailed flow of the composite image generation process in step S30 of FIG.

  The captured image held corresponding to the captured image reflection area of the selected layout pattern is reduced or enlarged based on the scale information stored corresponding to the captured image reflection area, and the captured image reflection area is displayed. Cut based on the mask information stored correspondingly, rotate based on the rotation information stored corresponding to the captured image reflection area, and reflect stored corresponding to the captured image reflection area of the background image Image composition arranged at the position is performed (step S210).

  Until the captured image corresponding to all the captured image reflection areas of the selected layout pattern is reflected, the process of step S210 is performed on the captured image reflection area where the captured image is not yet reflected (step S220).

  The present invention is not limited to that described in the above embodiment, and various modifications can be made. For example, terms cited as broad or synonymous terms in the description in the specification or drawings can be replaced with broad or synonymous terms in other descriptions in the specification or drawings.

  In the above-described embodiment, the photographing apparatus has been described by taking a business photo printing apparatus as an example, but the present invention is not limited thereto. For example, a photographic device or a photo sticker printing device may be used, or an information device (such as a portable communication device such as a mobile phone, a portable game machine, a portable information terminal, or a personal computer) equipped with a photographing function may be used. For example, an imaging device such as an electronic camera may be used.

  The present invention can also be applied to various game systems such as a business game system, a home game system, a portable game system, and a system board for generating a game image.

PM imaging device, RM price receiving unit, CM imaging unit, EM editing operation receiving unit, PB photo sticker providing unit, SP speaker, RA receiving area, RMN receiving monitor, CI coin slot, RL receiving lighting unit, MR mirror MR, BT imaging transition button, CA imaging area, ST sheet, CR camera, CL imaging illumination unit, CMN imaging monitor, EA editing area, EMN editing monitor, TP touch pen, PA extraction area, PO photo sticker exit, 100 processing unit, 110 collage shooting control unit, 112 background extracted image generation unit, 114 confirmation moving image generation unit, 116 captured image holding control unit, 118 composite image generation unit, 120 editing processing unit, 130 image generation unit, 140 sound Generation unit, 160 operation unit, 170 storage unit, 174 drawing buffer, 180 information Information storage medium, 190 display, 192 sound output unit, 194 portable information storage device, 195 printing unit, 196 communication unit, 198 imaging unit

Claims (6)

An imaging apparatus that generates a composite image corresponding to a given layout pattern including at least one captured image reflection area that is an area to reflect a captured image based on a captured image captured by an imaging unit,
A display unit that synthesizes in real time a captured image of a subject being imaged by an imaging unit or a background extracted image obtained by extracting a background from the captured image and a background image for confirmation corresponding to a given layout pattern or the captured image reflection area A confirmation moving image generating means for generating a confirmation moving image to be displayed on the display;
Means for controlling to hold a captured image at a given timing or a background extracted image obtained by extracting a background from the captured image as an image;
Depending on the captured image reflection area of the layout pattern, at least one of rotation, scale change, and alignment is performed on the captured image, and one or a plurality of captured images and a background image corresponding to the layout pattern are combined, or a synthetic image generating means for generating a composite image corresponding to the given layout pattern by combining a plurality of captured images, only including,
The confirmation moving image generating means includes
A confirmation moving image is generated using a confirmation background image that includes the same pattern as a part of the background image to be combined with the captured image, and at least one of the arrangement, angle, and scale of the pattern is different from the background image. An imaging apparatus characterized by generating. An imaging apparatus that generates a composite image corresponding to a given layout pattern including at least one captured image reflection area that is an area to reflect a captured image based on a captured image captured by an imaging unit,
A display unit that synthesizes in real time a captured image of a subject being imaged by an imaging unit or a background extracted image obtained by extracting a background from the captured image and a background image for confirmation corresponding to a given layout pattern or the captured image reflection area A confirmation moving image generating means for generating a confirmation moving image to be displayed on the display;
Means for controlling to hold a captured image at a given timing or a background extracted image obtained by extracting a background from the captured image as an image;
Depending on the captured image reflection area of the layout pattern, at least one of rotation, scale change, and alignment is performed on the captured image, and one or a plurality of captured images and a background image corresponding to the layout pattern are combined, Or a composite image generating means for generating a composite image corresponding to a given layout pattern by combining a plurality of captured images;
A confirmation background image storage means for storing a confirmation background image to be displayed as a part of the confirmation moving image in association with the layout pattern or the captured image reflection area;
The confirmation moving image generating means includes
As a confirmation background image corresponding to the layout pattern or the captured image reflection area, a confirmation moving image is generated using the confirmation background image stored corresponding to the layout pattern or the captured image reflection area,
The confirmation background image storage means includes:
An imaging apparatus comprising: a confirmation background image that includes the same pattern as a part of the background image and at least one of the arrangement, angle, and scale of the pattern is different from the background image. A program for generating a composite image corresponding to a given layout pattern including at least one captured image reflection area that is an area to reflect a captured image based on a captured image,
A confirmation moving image is obtained by combining in real time a photographed image of a subject being photographed or a background extracted image obtained by extracting a background from the photographed image and a confirmation background image corresponding to a given layout pattern or the photographed image reflection area. A confirmation moving image generating means for generating;
Means for controlling to hold a captured image at a given timing or a background extracted image obtained by extracting a background from the captured image as an image;
Depending on the captured image reflection area of the layout pattern, at least one of rotation, scale change, and alignment is performed on the captured image, and one or a plurality of captured images and a background image corresponding to the layout pattern are combined, Alternatively, the computer functions as a composite image generation unit that generates a composite image corresponding to a given layout pattern by combining a plurality of captured images.
The confirmation moving image generating means includes
A confirmation moving image is generated using a confirmation background image that includes the same pattern as a part of the background image to be combined with the captured image, and at least one of the arrangement, angle, and scale of the pattern is different from the background image. A program characterized by generating. In claim 3,
  The confirmation moving image generating means includes
  A program for generating a confirmation moving image in which the same pattern as a part of the background image is rotated and arranged according to a relative angle of the captured image reflection area with respect to the background image.   In claim 3 or 4,
  The confirmation moving image generating means includes
  A program for generating a confirmation moving image in which a same pattern as a part of the background image is enlarged or reduced in accordance with a scale when reflecting a captured image to the captured image reflection area. An image composition method for generating a composite image corresponding to a given layout pattern including at least one photographed image reflection region that is a region for reflecting a photographed image based on a photographed image,
A confirmation moving image is obtained by combining in real time a photographed image of a subject being photographed or a background extracted image obtained by extracting a background from the photographed image and a confirmation background image corresponding to a given layout pattern or the photographed image reflection area. A confirmation moving image generation step to be generated;
A step of performing control of capturing a captured image at a given timing or a background extracted image obtained by extracting a background from the captured image as a captured image;
Depending on the captured image reflection area of the layout pattern, at least one of rotation, scale change, and alignment is performed on the captured image, and one or a plurality of captured images and a background image corresponding to the layout pattern are combined, Or a composite image generation step of generating a composite image corresponding to a given layout pattern by combining a plurality of captured images ,
In the confirmation moving image generation step,
A confirmation moving image is generated using a confirmation background image that includes the same pattern as a part of the background image to be combined with the captured image, and at least one of the arrangement, angle, and scale of the pattern is different from the background image. An image composition method characterized by generating .