JP5657942B2 - Imaging apparatus, program, information storage medium, and imaging method - Google Patents

Description

  The present invention relates to a photographing apparatus, a program, an information storage medium, and a photographing method.

  2. Description of the Related Art Conventionally, a photographing apparatus such as a photo sticker manufacturing machine (an example of a photographic printing apparatus) that edits and prints an image of a subject photographed by a photographing unit such as a CCD camera is known.

  A photographing device such as a photo sticker manufacturing machine can edit a photographed image photographed by a plurality of friends to generate and enjoy a photograph sticker having a high degree of freedom different from a normal photograph.

  Recently, it is also popular as a communication tool for exchanging edited images over a network and publishing them on the Web without printing as photo stickers.

JP 2005-277476 A

  There is a need for an image acquired by a photographing device such as a photo sticker to have a face as close as possible. In order to shoot images that meet such needs, it is necessary for a plurality of users to shoot as close as possible.

  However, it is difficult to pose a plurality of users close to each other in a short shooting time, and it is difficult to perform shooting with the face portion close up.

  The present invention has been made to solve the above-described problems. According to the first aspect of the present invention, it is possible to provide an imaging device that can be guided so that the face portions of a plurality of users are close to each other. Moreover, according to the 2nd aspect of this invention, the imaging device which can provide the image with a high up degree of the face part of a some user can be provided.

(1) The present invention
An imaging device including an imaging unit that images a user as a subject,
Acquiring face position data indicating the position of the face of a subject in a photographed image photographed by the imaging unit or a composite image obtained by combining a background extracted image obtained by extracting a background from the photographed image and a given background image; Based on the position data, in the captured image or composite image, a cutout area setting unit that sets a rectangular area that includes the face portion of the subject and that has a predetermined vertical and horizontal ratio as a cutout area;
The present invention relates to a photographing apparatus including a provision image generating unit that generates a provision image based on a cut-out image of a photographed image or a composite image.

  The present invention also relates to a program that causes a computer to function as each of the above-described units. The present invention also relates to a computer-readable information storage medium that stores (records) a program that causes a computer to function as each unit. The present invention also relates to a method for controlling an imaging apparatus including steps for executing the above-described units.

  Here, the photographing device may be a photographic device or a photo sticker printing device, or an information device equipped with a photographing function (a portable communication device such as a mobile phone, a portable game machine, a portable information terminal, a personal computer, etc. For example, an imaging device such as an electronic camera may be used.

  This photographing apparatus needs to have a face position data acquisition function, but any function that can obtain information indicating the face position in any form is acceptable, and the configuration of the face position data acquisition function is not particularly limited. .

  As a method for detecting face position data, for example, a rectangular face region is extracted based on skin color information, feature points such as eyes, mouth, nose, etc., and a rectangle (rectangle) set corresponding to the face is extracted. It is possible to use a method in which the vertex data is the face position data.

  In addition, the photographing apparatus may have an editing function for performing an editing process on the captured image. Further, it may have a print function or a communication function of a captured image or an edited image obtained by performing editing processing on the captured image.

  Whether or not the face position has a predetermined positional relationship may be determined by the center coordinates of the face position, or a figure (a rectangle such as a circle or rectangle formed along the face outline) obtained from the face position data. Or a polygon).

  The number of the faces of the subject included in the captured image or the composite image may be one or plural. According to the present invention, it is possible to generate a provided image from a captured image or a composite image based on a partial image of a rectangle including a face portion of a subject and having a predetermined vertical and horizontal setting ratio.

  According to the present invention, it is possible to generate a provided image having the same content as the captured image or the composite image and having a higher degree of face up of the subject than the captured image or the composite image.

(2) In this photographing apparatus, program, information storage medium and photographing method,
The cut area setting unit
When the captured image or the composite image includes a plurality of faces, based on the face position data corresponding to the plurality of faces, set a cutting area individually for the plurality of faces,
The provided image generation unit
When cut areas are individually set for a plurality of faces, a provided image may be generated by combining images of the plurality of cut areas.

(3) In this photographing apparatus, program, information storage medium and photographing method,
The cut area setting unit
When the captured image or the composite image includes a plurality of faces, based on the face position data corresponding to the plurality of faces, a clipping region including a plurality of faces is set,
The provided image generation unit
When a cutout area including a plurality of faces is set, a provided image may be generated based on an image of the cutout area including a plurality of faces.

  One clipping region including a plurality of faces may be set for the photographed image or the composite image, or a plurality of clipping regions may be set. For example, when l (l = n + m) faces are included, a cutout area including n faces and a cutout area including m faces may be set. Further, a cutout area including n−1 faces and a cutout area including one face may be set.

(4) In this photographing apparatus, program, information storage medium and photographing method,
The cut area setting unit
When the captured image or the composite image includes a plurality of faces, it is determined whether or not the plurality of faces are in a predetermined positional relationship based on face position data corresponding to the plurality of faces, and the predetermined positional relationship is not satisfied. When a plurality of faces are set individually for a plurality of faces, and when the plurality of faces are in a predetermined positional relationship, one cutting area including a plurality of faces is set,
The provided image generation unit
When cut areas are individually set for a plurality of faces, a provided image is generated by combining the images of the plurality of cut areas, and one cut area is set for the plurality of faces. May generate the provided image based on the image of one cutout region.

  In this way, an optimal up image can be generated as a provided image according to the proximity of the subject's face.

(5) In this photographing apparatus, program, information storage medium and photographing method,
The cut area setting unit
When cutting areas are set individually for n faces, the horizontal ratio of the rectangle that becomes each cutting area is set to a value of 1 / n when one cutting area including a plurality of faces is set. Set,
The provided image generation unit
The provided image may be generated by arranging the images of the n cutout regions in the horizontal direction.

(6) In this photographing apparatus, program, information storage medium and photographing method,
The provided image generation unit
As the provided image, an up image in which the face of the subject is a close-up image may be generated based on the image of at least one cut-out area.

(7) In this photographing apparatus, program, information storage medium and photographing method,
The cut area setting unit
A determination rectangle corresponding to each face may be set based on the face position data, and a cutout region may be set based on the determination rectangle.

(8) In this photographing apparatus, program, information storage medium and photographing method,
The cut area setting unit
A determination rectangle corresponding to each face may be set based on the face position data, and based on the determination rectangle, it may be determined whether or not a plurality of faces have a predetermined positional relationship.

  When the face position data is given as, for example, vertex data of a rectangle (rectangle) set corresponding to the face, the rectangle may be used as a determination rectangle.

(9) In this photographing apparatus, program, information storage medium and photographing method,
The cut area setting unit
A rectangle that circumscribes the face rectangle set corresponding to the face position data and whose vertical and horizontal sides are parallel to the vertical and horizontal sides of the confirmation image or a rectangle obtained by adding a predetermined offset area to the rectangle is set as a determination rectangle. May be.

  For example, it may be determined that a plurality of faces have a predetermined positional relationship when a determination rectangle corresponding to the plurality of faces overlaps with another determination rectangle by a predetermined amount or more.

(10) In this photographing apparatus, program, information storage medium and photographing method,
The photographed image or the synthesized image and the provided image generated corresponding to either of them may be provided as an image for editing, and an editing processing unit that performs a given editing process may be included.

1 is a perspective view illustrating an example of an appearance of a photographic printing apparatus according to an embodiment. 1 is a perspective view illustrating an example of an appearance of a photographic printing apparatus according to an embodiment. 1 is a perspective view illustrating an example of an appearance of a photographic printing apparatus according to an embodiment. 1 is an example of a block diagram of a photo printing apparatus according to an embodiment. The figure for demonstrating the cursor display control in a confirmation image. 6A, 6B, and 6C are diagrams for explaining the cursor image of the present embodiment. The figure for demonstrating the method of investigating the positional relationship of a some face. The figure for demonstrating the method of investigating the positional relationship of a some face. The figure for demonstrating the method of investigating the positional relationship of a some face. The figure for demonstrating the method of determining the state in which the some face is separated in the depth direction. FIGS. 11A, 11 </ b> B, and 11 </ b> C are diagrams for explaining the positional relationship of the determination face rectangle in the confirmation image of FIG. 5. The flowchart which shows the flow of the process of this Embodiment. An example of the guidance image displayed on a display part by guidance processing. FIGS. 14A to 14E are views for explaining processing for generating a cursor image using a plurality of cursor reference images. The flowchart which shows the flow of a cursor display control process. The figure for demonstrating the automatic up function of this Embodiment. The figure for demonstrating the cutting synthetic | combination function of this Embodiment. 18A and 18B are diagrams for explaining the aspect ratio of the cut-out area. FIGS. 19A to 19F are diagrams for explaining an example of a cutting area setting method. 20A and 20B are diagrams for explaining a case where there are a plurality of cut regions. FIGS. 21A and 21B are diagrams for describing a case where a horizontally long provided image with one cutout region is generated. 22A and 22B are diagrams for explaining a case where there are a plurality of cut regions. The figure for demonstrating the provision image provided in this Embodiment. The flowchart which shows the flow of the process which performs automatic up or cut-out composition, and produces | generates a provision image. The figure for demonstrating the case where the cutting area containing a some face is set.

  Hereinafter, preferred embodiments of the present invention will be described with reference to the drawings.

1. Configuration of the imaging device
FIG. 1 is a perspective view illustrating an example of an appearance of the photographing apparatus PM (an example of an image printing apparatus) according to the present embodiment. As shown in FIG. 1, the photographing apparatus PM of the present embodiment includes a price receiving unit RM, an imaging unit CM, an editing operation receiving unit EM, and a photo sticker providing unit PB.

  The price accepting unit RM shown in FIG. 1 is the first contact with the user when using the imaging apparatus PM of the present embodiment, and is for inputting a price for using the imaging apparatus PM.

  The imaging unit CM illustrated in FIG. 1 is for capturing an image of a user who has input a price to the above-described price receiving unit RM, and is provided at a position different from the above-described price receiving unit RM. Yes. In the present embodiment, the imaging unit CM is covered with the sheet ST so that surrounding scenery does not appear in the background of the image, and so that the user being imaged cannot be seen from the surroundings.

  FIG. 2 is a perspective view showing the appearance of the imaging apparatus PM according to the present embodiment with the sheet ST removed, such that the front surface of the imaging unit CM is right front. As shown in FIG. 2, a camera CR that captures an image of a user located in the imaging area CA is provided at the upper front of the imaging unit CM. The camera CR can be realized by, for example, a CCD camera, a C-MOS camera, or a video camera. The camera CR captures a subject from the front direction and functions as an imaging unit.

  In addition, an imaging illumination unit CL that illuminates a user located in the imaging area CA is provided from the upper front to the middle of the imaging unit CM. The imaging illumination unit CL is controlled so as to be lit when the camera CR starts capturing an image of the user. Then, the camera CR is controlled to continue to illuminate the user while the user's image is captured.

  In addition, an imaging monitor CMN (an example of a display unit) that displays an image captured by the camera CR to a user located in the imaging area CA is provided in the front middle part of the imaging unit CM. In the present embodiment, when the camera CR starts capturing an image, the imaging monitor CMN displays the user's image located in the imaging area CA as a real-time moving image. As a result, the user can confirm the user's image captured by the camera CR on the imaging monitor CMN. The imaging monitor CMN is attached to the housing so that the display screen faces the user at a height looking down from the user's viewpoint. The video imaged by the camera CR is displayed on the imaging monitor CMN, and the user can check whether or not his / her pose is appropriate by viewing the video image displayed on the imaging monitor CMN.

  The imaging monitor CMN is configured by a touch panel monitor in which touch sensor panels capable of detecting a contact position are stacked, and a user can input an operation by touching the imaging monitor CMN.

  Then, when a predetermined time has elapsed since the start of image capturing of the user, the image capturing of the user is terminated. Then, the user who is located in the imaging area CA is guided to move to the editing area EA (an example of a third area) that is the area on the front side of the editing operation reception unit EM.

  The editing operation accepting unit EM illustrated in FIG. 1 is for a user who has captured an image in the above-described imaging unit CM to perform an operation for editing the captured image, and is located at a position different from that of the above-described imaging unit CM. Is provided. In the present embodiment, the editing operation reception unit EM is covered with the sheet ST so that the user who is performing the editing operation cannot see from the surroundings. In particular, in the present embodiment, a first editing operation receiving unit EM1 and a second editing operation receiving unit EM2 are provided as the editing operation receiving unit EM. FIG. 3 is a perspective view showing the appearance of the photographing apparatus PM of the present embodiment in a state where the sheet ST is removed so that the front surface of the first editing operation accepting unit EM1 is right front.

  As shown in FIG. 3, a user located in an editing area EA1 (an example of a third area) that is an area on the front side of the first editing operation accepting unit EM1 is located at the front center part of the first editing operation accepting part EM1. On the other hand, there is provided a touch panel type editing monitor EMN that displays an image determined as a still image to be printed on a sticker and receives an operation for editing the displayed image.

  In the present embodiment, touch pens TP are prepared on the left and right of the editing monitor EMN, respectively, and the user combines the determined image with the determined image by bringing the tip of the touch pen TP into contact with an arbitrary position of the editing monitor EMN. Or select a pattern or design template (stamp image) and place it on the image to edit the still image to be printed on the sticker.

  In the present embodiment, a second editing operation receiving unit EM2 having the same configuration as the first editing operation receiving unit EM1 is provided on the back side of the first editing operation receiving unit EM1 shown in FIG. The image displayed while displaying the image determined as the still image to be printed on the sticker for the user located in the editing area EA2 (an example of the fourth area) that is the area on the front side of the editing operation accepting unit EM2 of No. 2 Accepts operations for editing.

  Then, when a predetermined time elapses after the acceptance of the image editing operation is started or the end of the editing operation is selected by the user's operation, the acceptance of the image editing operation is finished. Then, the user located in the editing area EA1 or the editing area EA2 moves to a take-out area PA (an example of a fifth area) that is a front side area of the photo sticker providing unit PB (an example of a printed matter providing unit). To guide.

  As shown in FIG. 3, the left side surface of the first editing operation accepting unit EM1 is provided with a photo sticker providing unit PB that provides a user with a photo sticker (an example of a printed material) in which an edited image is printed on a sticker. It has been. In the present embodiment, a photo sticker outlet PO is provided in front of the photo sticker providing part PB so as to provide a photo sticker to a user located in the take-out area PA.

  FIG. 4 is an example of a block diagram of the photographing apparatus PM of the present embodiment. Note that the imaging apparatus PM does not have to include all the components (parts) shown in FIG. 4 and may have a configuration in which some of them are omitted. The imaging unit 198 is for photographing a subject and corresponds to the CR in FIG. 2 and can be realized by, for example, a CCD camera, a CMOS camera, or a video camera.

  An operation unit 160 (a pointing device, a lever, a button, or the like) is used by the user to perform a selection operation or an editing operation of the editing content selection menu.

  The storage unit 170 (RAM) is a work area such as the processing unit 100 or the communication unit 196.

  An information storage medium 180 (a computer-readable medium such as a CD, DVD, HDD, or ROM) stores programs and data. The information storage medium 180 stores a program for causing a computer to function as each unit of the present embodiment (a program for causing a computer to execute processing of each unit).

  The display 190 displays an image, and is, for example, the imaging monitor CMN or the editing display shown in FIG.

  The imaging monitor CMN uses, for example, a touch panel or the like, and inputs various instructions (for example, selection of a collage background, selection of a shooting mode, etc.) when a user touches an input button displayed on a display with a finger. It is configured to be able to do.

  As the editing display, for example, a liquid crystal tablet, a touch panel, or the like is used, and a handwritten image can be input by drawing a character or a figure by bringing the tip of a touch pen (an example of a pointing device) into contact with the display surface. . In addition, a stamp image such as a star mark or a heart mark prepared in advance is selected, and the stamp image can be input by bringing the tip of the touch pen into contact with the display surface.

  The sound output unit 192 outputs sounds such as sounds and game sounds.

  The portable information storage device 194 stores user personal data, save data for editing, and the like.

  The printing unit 195 performs processing for printing the captured image on a print medium (seal paper, photographic paper, plastic plate, or recording layer). In this case, there are various printing methods such as a sublimation type, a thermal transfer type, an ink jet method, and a laser printing method. As a printer for printing, an ink jet printer, a laser printer, a sublimation printer, a thermal transfer printer, a melt printer, a thermal printer, a photographic paper printer, an instant film printer, or the like may be used.

  The processing unit 100 (processor) performs various processes such as a game process, an image generation process, and a sound generation process based on operation data from the operation unit 160, a program, and the like. In this case, the processing unit 100 performs various processes using the main storage unit 172 in the storage unit 170 as a work area. The function of the processing unit 100 can be realized by hardware such as various processors (CPU, DSP, etc.) or ASIC (gate array, etc.) and a program (game program).

  The processing unit 100 includes a background extracted image generation unit 110, a confirmation image display control unit 112, a cursor image display control unit 114, an image capture control unit 116, a guidance processing unit 118, a cut area setting unit 122, a provided image generation unit 124, an editing A processing unit 120, an image generation unit 130, and a sound generation unit 140 are included.

  The background extracted image generation unit 110 performs a process of extracting a predetermined color (background color) from a captured image captured using a predetermined color (background color) as a background. The extraction process of a predetermined color (background color) may be realized by, for example, chroma key processing or mask processing.

  The chroma key is a technique for extracting an area having a specific color (for example, background color) from an image and embedding another image therein. The method of embedding another image is realized by setting the α value (a value indicating transparency) of an area having a specific color to 1, and performing translucent synthesis (for example, α blending processing) with another image. be able to.

  Mask processing is a process of creating a mask image in which all pixel values of a specific area of the image (background color area in the present invention) are set to 1, and all other parts are set to 0. In this method, the image A is embedded in the portion and the image B is embedded in the portion 1 of the mask image to synthesize both images.

  The confirmation image display control unit 112 performs processing for displaying a confirmation image corresponding to the photographed image captured by the imaging unit on the display unit in real time, and the cursor image display control unit 114 performs processing for displaying the face of the subject in the confirmation image. Face position data indicating the position of the subject, and a cursor image indicating the face of the subject of the confirmation image is displayed based on the photographed image and the face position data, and the image capture control unit 116 captures timing that occurs during photographing. Control is performed to store an image corresponding to the confirmation image in the storage unit as a captured image.

  When the confirmation image includes a plurality of faces, the cursor image display control unit 114 determines whether or not the plurality of faces have a predetermined positional relationship based on the face position data. If it is determined, a first cursor image showing a plurality of faces individually is displayed on the confirmation image, and if it is determined that there is a predetermined positional relationship, a plurality of faces are shown integrally on the confirmation image. A cursor integrated display process for displaying the second cursor image is performed.

  Further, the confirmation image display control unit 112 generates, in real time, a composite image obtained by combining a background extracted image obtained by extracting a background from a photographed image of the subject photographed by the imaging unit and a given background image as the confirmation image, The image capture control unit 116 may store a composite image corresponding to the confirmation image at the capture timing generated during shooting as a captured image.

  The cursor image display control unit 114 may generate a confirmation image on which a frame-like cursor image surrounding the face is displayed as the cursor image.

  In addition, the cursor image display control unit 114 sets a determination rectangle corresponding to each face based on the face position data, and determines whether or not a plurality of faces have a predetermined positional relationship based on the determination rectangle. Also good.

  The cursor image display control unit 114 circumscribes the determination rectangle set corresponding to the face position data, and a rectangle whose vertical and horizontal sides are parallel to the vertical and horizontal sides of the confirmation image or a predetermined offset area in the rectangle. The added rectangle may be set as the determination rectangle.

  The cursor image display control unit 114 determines whether or not the degree of close-up of the face satisfies a predetermined standard when a plurality of faces are in a predetermined positional relationship, May display a third cursor image of a special specification that shows a plurality of faces in a predetermined positional relationship as one body.

  In addition, when the confirmation image includes a plurality of faces including a first face and a second face, the cursor image display control unit 114 determines the determination rectangle set corresponding to the first face and the It is determined whether or not the size of the determination rectangle set corresponding to the second face is greater than a predetermined reference. If it is determined that the size is different, the integrated cursor display process is not performed. Good.

  The guidance processing unit 118 causes the display unit to display a guidance image including a description regarding the cursor integrated display processing. The guidance processing unit 118 may determine the number of photographers based on face position data acquired at the time of photographing, and may explain the cursor integrated display processing when there are a plurality of photographers.

  Further, the guidance processing unit 118 may omit the description regarding the cursor integrated display processing when the cursor integrated display processing is performed when displaying the confirmation image in the shooting after the start of the game.

  The cursor image display control unit 114 prepares a plurality of cursor reference images having similar shapes and different sizes in steps, and selects and selects one of the cursor reference images according to the degree of close-up of the captured image. The cursor image may be generated by performing at least one of enlargement, reduction, and scale change on the cursor reference image.

  Further, the image capture control unit 116 performs processing for counting the remaining time until the capture timing during shooting, and the confirmation image display control unit 112 ends the display of the cursor image when the remaining time becomes a predetermined value or less. May be.

  The cut region setting unit 122 is a face position indicating the position of the face of the subject in the captured image captured by the image capturing unit, or a composite image obtained by combining a background extracted image obtained by extracting a background from the captured image and a given background image. Data is acquired, and based on the face position data, in the photographed image or the composite image, a rectangular area including a face portion of the subject and having a predetermined horizontal and vertical ratio is set as a cut-out area, and the provided image generation unit 124 The provided image is generated based on the image of the cut area of the image or the composite image.

  In addition, when the captured image or the composite image includes a plurality of faces, the cut region setting unit 122 sets the cut regions individually for the plurality of faces based on the face position data corresponding to the plurality of faces. The provided image generation unit 124 may generate the provided image by combining the images of the plurality of cut regions when the cut regions are individually set for the plurality of faces.

  In addition, when the captured image or the composite image includes a plurality of faces, the cut region setting unit 112 sets a cut region including a plurality of faces based on face position data corresponding to the plurality of faces, and provides a generated image The unit 124 may generate the provided image based on the image of the cut area including the plurality of faces when the cut area including the plurality of faces is set.

  In addition, when the captured image or the composite image includes a plurality of faces, the cutout region setting unit 122 determines whether the plurality of faces have a predetermined positional relationship based on face position data corresponding to the plurality of faces. And when it is not in a predetermined positional relationship, set a cutting area individually for a plurality of faces, and when a plurality of faces are in a predetermined positional relationship, set one cutting area including a plurality of faces, The provided image generation unit 124 generates a provided image by combining the images of the plurality of cut regions when the cut regions are individually set for the plurality of faces, and generates one cut image for the plurality of faces. When the region is set, the provided image may be generated based on the image of one cutout region.

  In addition, when the cut area setting unit 122 sets cut areas individually for n faces, the cut area setting unit 122 sets one cut area including a plurality of faces with the horizontal ratio of each cut area. In this case, the provision image generation unit 124 may generate the provision image by arranging the images of the n clipped regions in the horizontal direction.

  Further, the provided image generation unit 124 may generate an up image in which the face of the subject is a close-up based on the image of at least one cut-out area as the provided image.

  The cut area setting unit 122 may set a determination rectangle corresponding to each face based on the face position data, and set a cut area based on the determination rectangle.

  The cutout area setting unit 122 also sets a determination rectangle corresponding to each face based on the face position data, and determines whether or not a plurality of faces have a predetermined positional relationship based on the determination rectangle. Good.

  Further, the cutout area setting unit 122 circumscribes the face rectangle set corresponding to the face position data, and adds a predetermined offset area to the rectangle whose vertical and horizontal sides are parallel to the vertical and horizontal sides of the confirmation image. A rectangle may be set as the determination rectangle.

  The editing processing unit 120 performs editing processing on the captured image based on input information for editing from the operation unit 160. The editing processing unit 120 may provide the captured image or the composite image and the provided image generated corresponding to either of them as an image for editing, and perform a given editing process.

  The image generation unit 130 generates an image based on the result of shooting or editing, and outputs the image to the display 190.

The sound generation unit 140 performs sound processing based on the results of various processes performed by the processing unit 100, generates game sounds such as BGM, sound effects, or sounds, and outputs the game sounds to the sound output unit 192.
The communication unit 196 performs processing for transmitting or receiving data via a network such as the Internet. More specifically, the communication unit 196 transmits data generated from the captured image of the camera to an external server or the like via the network. In this way, the user (subject) can download image data of the subject himself / herself from the server to the personal computer of his / her home. Based on the downloaded image data, it is possible to print and output an image on paper or a sticker using a printer at home. It is also possible to store an image captured by the user in a server, download it to the image capturing apparatus, print it, and output it.

  The function of the communication unit 196 can be realized by hardware such as a communication device (IC), a communication program, or the like. Note that either the printing unit 195 or the communication unit 196 may be omitted.

  The communication unit 196 functions as a user information acquisition unit that acquires user information regarding the user who is shooting, and the cursor image display control unit 114 prepares a plurality of patterns of cursor images, and based on the acquired user information. Alternatively, one cursor image may be selected from a plurality of patterns of cursor images, and cursor display may be performed using the selected cursor images.

  Note that the photographing apparatus of the present embodiment may be a fixed housing or a portable device. Further, the photographing apparatus of the present embodiment can be used for a game apparatus of a type that uses a user's photographed image for a game. That is, it is possible to cause the photographing apparatus of this embodiment to function as an image capturing device of a game device.

2. Display control of cursor image
FIG. 5 is a diagram for explaining cursor display control in a confirmation image.

  In the present embodiment, during shooting, a confirmation image (for example, a live view image) corresponding to a captured image (or a composite image) captured by the imaging unit is displayed in real time on a display (CNN in FIG. 3) as a moving image (live view). indicate.

  Reference numerals 200-t1, 200-t2, and 200-t3 are confirmation images. The confirmation images 200-t1, 200-t2, and 200-t3 may be captured images themselves that are currently being captured, or may be composite images obtained by combining a background extracted image obtained by extracting a background from a captured image and a given background image.

  The background extracted image may be generated by, for example, reversing the photographed image left and right. In this way, the background-extracted image becomes the same image as when there is a mirror in the front (when there is a camera and a display unit in front of the user). Easy to do.

  The background extracted image is obtained by capturing the image of the subject being captured by the imaging unit as a current image with a predetermined color (background color) as the background, and extracting the background portion by extracting the predetermined color (background color) from the current image. Images may be generated in real time.

  Since the confirmation image is displayed as a real-time moving image on the display unit during shooting, the user aligns or poses while viewing the confirmation image, and the captured image (captured image or composite image) Can be confirmed.

  In the present embodiment, face position data indicating the position of the face of the subject in the confirmation image (which may be a captured image) is received. Then, at the time of live view display, based on the acquired face position data, the cursor image 220 indicating that the face is on the face portions 212-1 and 212-2 of the subjects 210-1 and 210-2 of the confirmation image 200-t1. -1, 220-2 are displayed. In a state where the face portions 212-1 and 212-2 of the subjects 210-1 and 210-2 are separated to some extent (when a plurality of faces are not in a predetermined positional relationship), as shown in the confirmation image 200-t1, First cursor images 220-1 and 220-2 showing the faces 212-1 and 212-2 are displayed. Note that when displaying the first cursor image, an image, text, or the like that prompts the distance between a plurality of subjects to be reduced may be displayed in the vicinity of the first cursor.

  Then, as shown in the confirmation image 200-t2, when the face portions 212-1 and 212-2 of the subject approach more than a certain amount (when a plurality of faces are in a predetermined positional relationship), the faces of the subjects 210-1 and 210-2 A second cursor 230 showing the portions 212-1 and 212-2 as a unit is displayed. When displaying the second cursor, an image or text that prompts the subject to approach the camera more may be displayed in the vicinity of the second cursor.

  Further, as shown in the confirmation image 200-t3, when the degree of increase in the face portions 212-1 and 212-2 of the subject further increases (when the subject approaches the camera), the third cursor image 230 is replaced with the third cursor image 230. A cursor image 240 is displayed. The third cursor image 240 is a star image around an image obtained by decorating the second cursor image 230 (for example, a frame of a heart indicating the cursor (this has the same shape as the second cursor image) as indicated by 240). The image may be an image in which 242 is interspersed, or may be a cursor image having a shape or pattern completely different from that of the second cursor image.

  Note that the second and third cursors are shaped like a heart symbol or an inverted triangle so that the bottom of the cursor is narrowed, so that the two subjects can approach the chin and the two subjects can bring their faces close to each other without a sense of incongruity. be able to.

  In this way, the distance between the face parts of a plurality of subjects approaches a certain distance, and the subject approaches the camera more than a certain distance. In addition, it is possible to take an image in which the face of the subject is shown larger.

  In addition, in a photographing apparatus in which photographing timing is automatically set, a time for a user (subject) to perform alignment is often limited, but while looking at a cursor image displayed on a confirmation image, It is possible to adjust the distance between a plurality of subjects and the distance between the camera and the subject, thereby enabling easy alignment.

  6A, 6B, and 6C are diagrams for explaining the cursor image of the present embodiment. 260A and 260-2 in FIG. 6A are other examples of the first cursor image, 270 in FIG. 6B is another example of the second cursor image, and FIG. ) 290 is another example of the third cursor image. The cursor images (first cursor image, second cursor image, and third cursor image) are shown in FIGS. 5, 6A, 6B, and 6C, as shown in FIGS. A frame surrounding 212-2 (a closed frame such as 260-1 and 260-2 in FIG. 6A or an open frame such as 220-1 and 220-2 in FIG. 5) The image of

  In this way, since a frame-like cursor image surrounding the face is displayed, it is possible to prevent the face from becoming difficult to see due to the cursor image. Therefore, it is possible to generate a confirmation image that does not hinder the user's image confirmation.

  7 to 9 are diagrams for explaining a method of examining the positional relationship between a plurality of faces.

  In the present embodiment, face position data indicating the position of the face of the subject in the confirmation image (which may be a captured image) is received.

  For example, as shown in FIG. 7, the face position data is the vertex data (302-1, 304-1, 306) of the face rectangles 300-1, 300-2 set corresponding to the faces 212-1, 212-2. −1, 308-1 coordinate values), (302-2, 304-2, 306-2, 308-2 coordinate values).

  Based on this face position data, as shown in FIGS. 8A and 8B, determination rectangles 340-1340-2 corresponding to each face are set, and a plurality of faces 212-1 are set based on the determination rectangles. , 212-2 may be determined to be in a predetermined positional relationship. As the determination rectangle, the face rectangles 300-1 and 300-2 in FIG. 7 may be used, or a rectangle set based on the face rectangles 300-1 and 300-2 as described in FIG. 9 is used. Also good.

  As shown in FIG. 8A, when the determination rectangles 340-1 and 340-2 do not have overlapping portions (not overlapped), it is determined that a plurality of faces do not satisfy the predetermined positional relationship, and FIG. As shown in (B), when the determination rectangles 340-1 and 340-2 have overlapping portions (overlap), it may be determined that a plurality of faces satisfy a predetermined positional relationship. In addition, it may be added to the condition satisfying the predetermined positional relationship that not only the overlap but the ratio of the overlapping portion is a predetermined ratio or more. For example, when the ratio a / b of the length (a) of the overlapping portion to the length (b) of the side 342 of the determination rectangle 340-1 constituting the overlapping portion is equal to or greater than a predetermined ratio, a plurality of faces are predetermined. It may be determined that the positional relationship is satisfied.

  As shown in FIG. 9A, when the face is inclined, the facing sides of the obtained face rectangles 300- and 300-2 are not parallel. Therefore, as shown in FIG. 9B, a rectangle (circumscribing each of the face rectangles 300-1 and 300-2 and having a vertical and horizontal side parallel to the vertical and horizontal sides of the confirmation image 200 (which may be a captured image or a composite image). The second face rectangles 320-1 and 320-2 may be set as the determination rectangles 340-1 and 340-2. In addition, a rectangle (third face) added with a predetermined offset area (the vertical side is extended by l1 vertically and the horizontal side is extended by l2 left and right) to the second face rectangles 320-1 and 320-2. (Rectangular) 330-1 and 330-2 may be set as the determination rectangles 340-1 and 340-2.

  FIG. 10 is a diagram for explaining a method for determining a state in which a plurality of faces are separated in the depth direction.

  As shown in FIG. 10, the subjects 210-1 and 210-2 are separated in the depth direction even if the faces 212-1 and 212-2 of a plurality of subjects are close to each other on the two-dimensional plane that is the confirmation image 200. There are cases. In such a case, a plurality of faces may not be indicated by an integrated cursor.

  That is, when the confirmation image 200 includes a plurality of faces including the first face 212-1 and the second face 212-2, the radius rectangle 340-1 set corresponding to the first face If it is determined whether or not the size (side length or area) of the determination rectangle 340-2 set corresponding to the second face differs by more than a predetermined reference, The cursor integrated display process may not be performed.

  Whether the sizes of the determination rectangles 340-1 and 340-2 are different from a predetermined standard or more may be determined based on the area of the determination rectangle, or may be determined based on the length of the sides of the determination rectangle. Also good.

  11A, 11B, and 11C are diagrams for explaining the positional relationship of the determination rectangles in the confirmation image of FIG.

  Reference numerals 340-1 and 340-2 in FIG. 11A indicate determination rectangles set for each face in the confirmation image 200-t1 in FIG. Here, since the determination rectangles 340-1 and 340-2 do not satisfy a predetermined positional relationship (for example, a relationship having an overlapping portion as described with reference to FIG. 7B), as indicated by 200-t 1 in FIG. 5. The first cursor is individually displayed for each face. The determination rectangle is set in real time based on the acquired face position data. Therefore, the determination rectangle that is set when the pose or position of the subject changes also changes. Accordingly, it is determined in real time whether each determination rectangle has a predetermined positional relationship.

  Reference numerals 340-1 and 340-2 in FIG. 11B indicate determination rectangles set for each face in the confirmation image 200-t2 in FIG. Here, since the determination rectangles 340-1 and 340-2 satisfy the predetermined positional relationship, the integrated rectangle 350 is set, and the second rectangle is set with respect to the integrated rectangle as indicated by 200-t <b> 1 in FIG. 5. A cursor image 230 is set.

  Reference numerals 340-1 and 340-2 in FIG. 11C indicate determination rectangles set for each face in the confirmation image 200-t3 in FIG. Here, since the determination rectangles 340-1 and 340-2 satisfy a predetermined positional relationship, the integrated rectangle 350 is set. In addition, the subject is approaching the camera while the face of the subject is approaching, and the subject's face is close-up (up) compared to FIG. Therefore, as shown in FIG. 11B, the ratios of the determination rectangles 340-1 and 340-2 and the integrated rectangle 350 in the confirmation image are large. If this ratio is greater than or equal to a predetermined value, it may be determined that the degree of close-up of the face satisfies a predetermined criterion. Further, when the lengths of the sides of the determination rectangles 340-1 and 340-2 and the integrated rectangle 350 are equal to or larger than a predetermined ratio of the vertical and horizontal lengths of the confirmation image, the degree of the close-up of the face satisfies a predetermined criterion. You may judge.

  The degree of close-up (up) may be determined by the horizontal length of the integrated rectangle 350 (which may be a ratio to the horizontal length of the confirmation image), or the diagonal length of the integrated rectangle 350 (the confirmation image). The ratio may be a ratio to the diagonal line. Alternatively, it may be determined by the ratio of closing the entire confirmation image of the area of the integrated rectangle 350.

3. Guidance processing
Next, the guidance process to be displayed on the guidance image display unit including the explanation related to the cursor integrated display process will be described. FIG. 12 is a flowchart showing a processing flow of the present embodiment. FIG. 13 is an example of a guidance image displayed on the display unit in the guidance process.

  Before the game starts, an attract image is displayed (step S10), and the game starts when credits such as coins are inserted. When the game starts (step S20), various setting screens are displayed, and shooting mode setting processing and the like are performed (step S30). Selection of a background image or the like may also be performed here. When the various setting processes are completed, the photographing process (step S40) is started. The photographing process can be performed n times within a predetermined period in one game. Accordingly, the photographing process (step S50) is repeated until the specified number is reached. When the photographing process is started, the camera CR starts imaging, and the imaging monitor CMN displays a user image (live view image, corresponding to the confirmation image of the present application) located in the imaging area CA as a real-time moving image ( Live view display). A cursor image as shown in FIGS. 5 and 6 is displayed on the live view image at this time. When a predetermined time elapses from the start of imaging, the image capture timing is reached, and the captured image (or a composite image with the background image) at this time is stored as a captured image in the storage unit and is subjected to later editing processing.

  When the current photographing process is finished, if the photographing is not finished (if the number of photographing has not reached the specified number and there is a remaining time) (N in step S50), if the first photographing is finished (step S50) In the case of Y in S70) After the guidance process (step S80) is performed, the process goes again to the photographing process (step S40). If the first shooting is not completed (N in step S70), the guidance process is not performed, and the process again goes to the shooting process (step S40). In the guidance process (step S80), for example, a guidance image as shown in FIG. 13 is displayed to prompt the user to recognize the cursor integration process. When the number of photographers of the current game is determined based on the face position data acquired in the past shooting (which may be the previous shooting) performed since the start of the current game, and it is determined that there are not a plurality of photographers The guidance image display process related to the cursor integrated display process may be omitted.

  In this way, since it is possible to omit the guidance image output process that is not related to the current shooting, the overall processing time can be shortened, the operating rate can be improved, and a wasteful waiting time for the user. You can enjoy the game without getting tired.

  If the cursor integrated display process has already been performed in the shooting after the game is started (may be the first shooting), the description regarding the cursor integrated display process may be omitted. For example, if n shots are possible in one game, and the cursor integrated display process is performed during the first shooting after the game starts, the description of the cursor integrated display process is omitted. Also good.

4). Cursor image creation method
FIGS. 14A to 14E are views for explaining processing for generating a cursor image using a plurality of cursor reference images. FIGS. 14A, 14B, 14C, and 14D show a plurality of confirmation images with different degrees of face up of the subject, and FIG. 14E shows a cursor image and its scale. 14A, 14B, 14C, and 14D, the ratio of one predetermined side of the acquired face rectangle 300-1 is h1: h2: h3: h4 (h1 <h2 <h3 <h4). It shall be. Cursor images 370-1, 370-2, 370-3, and 370-4 in FIG. 14E are obtained by arranging cursor images having sizes corresponding to those in FIGS. is there.

  In the present embodiment, the cursor display is realized by placing the cursor image on the face portion of the confirmation image. Here, the size of the cursor image changes according to the size of the face image (face rectangle) in the confirmation image. That is, the size (or scale) of the cursor image also changes in proportion to the size (or side length) of the face rectangle of the confirmation image.

  In this embodiment, a plurality of cursor reference images 360-1 and 360-2 having similar shapes and different sizes are prepared, and the cursor reference image 360 to be used is used according to the degree of close-up (up) of the confirmation image. -1, 360-2 are changed. The cursor reference image may be enlarged or reduced to generate and display cursor images of various sizes in real time.

  For example, in the case of FIG. 14A, the cursor reference image 360-1 may be reduced to generate the cursor image 370-1. In the case of FIG. 14B, the cursor reference image 360-1 may be used as it is as the cursor image 370-2. In the case of FIG. 14C, the cursor reference image 360-2 may be reduced to generate a cursor image 370-3, or the cursor reference image 360-1 may be enlarged to generate a cursor image. 370-3 may be generated. In the case of FIG. 14D, the cursor reference image 360-2 may be used as it is as the cursor image 370-4.

  In this way, it is possible to prevent deterioration of the image quality of the cursor image as compared with the case where the cursor image corresponding to all the up degrees is generated by enlarging or reducing one cursor reference image.

5. Modification of cursor image display control
If the capturing device is configured so that the capturing timing comes during the capturing after the start of capturing, the remaining time from the capturing to the capturing timing is counted, and the remaining time is set to a predetermined value. Then, the display of the cursor image may be terminated. In this way, the cursor display ends immediately before the capture timing, so that the user can easily confirm his / her facial expression.

  In addition, when the photographing apparatus includes a unit that acquires user information about the user (for example, when acquiring user information stored in a server via a network, or when user information stored in a user's portable information device is networked) In the case of obtaining via (), the cursor image to be used may be changed based on the user information. For example, a plurality of patterns of cursor images are prepared, one cursor image is selected from a plurality of patterns of cursor images based on the acquired user information, and cursor display is performed using the selected cursor images. Also good.

  For example, the same cursor image may be used by all users for the first cursor image, and the second cursor image may be selected based on user information of a plurality of users. For example, as the second cursor image, three types of cursor images are prepared: a cursor image used for male friends, a cursor image used for female friends, and a cursor image used for male and female couples. Any cursor image may be selected and displayed based on the user information of the user to be integrated.

  Further, for example, the first cursor image may be selected based on the user information. For example, a cursor image for men, a cursor image for women, and the like may be prepared, and one of the cursor images may be selected and displayed based on user information.

6). Flow of cursor image display control processing
FIG. 15 is a flowchart showing the flow of the cursor display control process.

  When shooting is started, the following processing is performed at a predetermined interval (for example, every frame of an image to be generated) (step S110).

  A synthesized image obtained by synthesizing the background extracted image obtained by extracting the background from the photographed image of the photographed subject and the given background image is generated and output as a confirmation image in a live view (step S120). Next, face position data is acquired (step S130).

  When the confirmation image includes a plurality of faces (Y in step S140), a determination rectangle corresponding to the face is obtained based on the face position data (step S150).

  If the determination rectangle corresponding to the face is in a predetermined positional relationship (in the case of Y in step S160), it is determined whether the degree of close-up (up) of the face satisfies a predetermined standard and satisfies (In the case of Y in step S170), a specially designed third cursor image showing a plurality of faces in a predetermined positional relationship as a single unit is displayed on the confirmation image (step S180).

  If the degree of close-up (up) of the face does not satisfy a predetermined standard (N in step S170), a second cursor image showing a plurality of faces in a predetermined positional relationship as a single unit is confirmed. (Step S190).

  If the determination rectangle corresponding to the face has a predetermined positional relationship (N in step S160), a first cursor image individually indicating a plurality of faces is displayed on the confirmation image (step S200).

  By doing so, a cursor image corresponding to the change of the face position in real time can be displayed. When the faces are in a predetermined positional relationship, the cursor is integrated, so that the user can naturally guide the user's face closer by aligning while viewing the confirmation image. Further, when the degree of improvement of the user's face increases, a specially specified third cursor is displayed, so that the user can be guided naturally in the direction of increasing the degree of improvement of the user's face.

7). Auto-up function and cut-out composition function
FIG. 16 is a diagram for explaining the automatic up function of the present embodiment.

  The automatic up function of the present embodiment is a function provided when an image for editing is provided, and an automatic up process is performed on the captured image 400 captured at the capture timing that occurs during shooting. Here, the captured image 400 may be a captured image captured by the imaging unit, or may be a composite image obtained by combining a background extracted image obtained by extracting a background from the captured image and a given background image. The captured image corresponds to the confirmation image displayed at the capture timing.

  Face position data (coordinate values of 302-1, 304-1, 306-1, 308-1) indicating the position of the face of the subject in the captured image 400 (which may be a captured image or a composite image), (302-2) , 304-2, 306-2, and 308-2), and based on the face position data, in the captured image 400, a rectangular region that includes the face of the subject and that has a predetermined vertical and horizontal setting ratio is cut out. The provided image 420 is generated based on the image of the cutout area 410.

  For example, as shown in FIG. 16, the face position data is the vertex data (302-1, 304-1, 306) of the face rectangles 300-1, 300-2 set corresponding to the respective faces 212-1, 212-2. −1, 308-1 coordinate values), (302-2, 304-2, 306-2, 308-2 coordinate values). Then, a determination rectangle (not shown) corresponding to each face may be set based on the face position data, and a cut region 410 having a predetermined vertical / horizontal ratio may be set based on the determination rectangle.

  When a plurality of subjects are photographed close to each other as shown in FIG. 16, a plurality of subjects are integrated to set one cut area. By generating an image for editing based on the cut region set in this way, it is possible to provide a provided image in which a plurality of subjects are up.

  FIG. 25 is a diagram for describing a case where a cutout region including a plurality of faces is set. In FIG. 16, the faces 212-1 and 212-2 of a plurality of subjects are not close to each other as shown in FIG. 16, but even in such a case, one cutout region including the faces 212-1 and 212-2 of the subjects May be set.

  Face position data (302-1, 304-1, 306-1, 308-1 coordinates) indicating the positions of the subject's faces 212-1 and 212-2 in the captured image 400 (which may be a captured image or a composite image) Value), (coordinate values of 302-2, 304-2, 306-2, 308-2), or a plurality of faces based on a plurality of face rectangles 300-1, 300-2 specified by the face position data. Alternatively, a rectangular area including a face rectangle and having a predetermined setting ratio in the vertical and horizontal directions may be determined as the cut area 410, and the provided image 420 may be generated based on the image of the cut area 410.

  In FIG. 25, the case where one cutout area including a plurality of faces is set as an example for the captured image or the composite image has been described as an example. However, the present invention is not limited to this, and a plurality of cutout areas including a plurality of faces are set. May be. For example, when the photographed image or the composite image includes l (l = n + m) faces, a cutout area including n faces and a cutout area including m faces may be set. In addition, a cutout area including a plurality of faces (for example, a cutout area including n−1 faces) and a cutout area including one face may be set.

  FIG. 17 is a diagram for explaining the cut-out composition function of the present embodiment.

  In the captured image 400 of FIG. 17, since a plurality of subjects are not close to each other as shown in FIG. 16, when one cut region is set by integrating a plurality of subjects, the area of the face portion of the subject occupying the cut region is reduced. The provided image in which the faces of a plurality of subjects are up cannot be generated.

  Therefore, in such a case, face position data (coordinate values of 302-1, 304-1, 306-1, 308-1) corresponding to a plurality of faces, (302-2, 304-2, 306-2). , 308-2), the cut areas 410-1 and 410-2 are determined individually for a plurality of faces.

  For example, as shown in FIG. 17, the face position data is the vertex data (302-1, 304-1, 306) of the face rectangles 300-1, 300-2 set corresponding to the faces 212-1, 212-2. −1, 308-1 coordinate values), (302-2, 304-2, 306-2, 308-2 coordinate values). Then, a determination rectangle (not shown) corresponding to each face is set based on the face position data, and based on the determination rectangle, a cutout area 410 having a predetermined ratio between the vertical and horizontal directions for each of the faces 212-1 and 212-2. −1, 410-2 may be determined.

  Then, the provided image 420 is generated by combining the images of the plurality of cut areas 410-1 and 410-2. In this way, even when a plurality of subjects are not close to each other, it is possible to generate a provided image in which each subject is close up.

  As described above, in the present embodiment, when the captured image (the captured image or the composite image) includes a plurality of faces, the plurality of faces have a predetermined positional relationship based on the face position data corresponding to the plurality of faces. If the predetermined positional relationship is not satisfied, as shown in FIG. 17, cut areas are individually set for a plurality of faces, and a provided image is generated by combining the images of the plurality of cut areas. . Further, when a plurality of faces are in a predetermined positional relationship, one cut area including a plurality of faces may be set as shown in FIG. 16, and a provided image may be generated based on an image of one cut area. .

  Here, whether or not a plurality of faces are in a predetermined positional relationship can be performed by a method similar to the method described with reference to FIGS. That is, a determination rectangle (rectangle) corresponding to each face is set based on the face position data, and it can be determined whether or not a plurality of faces are in a predetermined positional relationship based on the determination rectangle. When face position data is given by rectangular vertex data set corresponding to each face as shown in FIG. 9 (A), it circumscribes the rectangle as shown in FIG. A rectangle parallel to the vertical and horizontal sides of the captured image or the composite image may be set as a determination rectangle, or a rectangle obtained by adding a predetermined offset area to the rectangle as shown in FIG. It may be set.

  18A and 18B are diagrams for explaining the aspect ratio of the cut-out area.

  In the present embodiment, the cut-out area is a rectangle having a predetermined setting ratio in the vertical and horizontal directions including the face of the subject. The predetermined vertical / horizontal ratio may be set in advance, or may be set to be changeable in accordance with an external setting input or a change in predetermined parameters.

  Here, when the predetermined vertical / horizontal ratio is set as a: b, when one cut area 420 is set as shown in FIG. 18A, the vertical / horizontal ratio of the one cut area 420 is a. : B.

  Further, when two cut areas 420-1 and 420-2 are set as shown in FIG. 18B, the aspect ratio of the two cut areas 420-1 and 420-2 is a: b / 2. Become.

  In this way, when the cut areas are individually set for the n faces, the horizontal ratio of the rectangle to be the cut areas is set to 1 / n when one cut area including a plurality of faces is set. The provided image may be generated by arranging the images of the n cut regions in the horizontal direction.

8). Set cropping area
FIGS. 19A to 19F are diagrams for explaining an example of the setting method of the cut region.

  FIG. 19A shows the face position data (corresponding values of the vertices 302-1, 304-1, 306-1, and 308-1, the vertices 302− given to the faces 212-1 and 212-2. 2, 304-2, 306-2, and 308-2), the face rectangles 300-1 and 300-2 are set.

  In FIG. 19B, for each of the face rectangles 300-1 and 300-2, the face rectangles 300-1 and 300-2 are circumscribed, and the vertical and horizontal sides are captured images 400 (may be captured images or composite images). The rectangles (second face rectangles) 320-1 and 320-2 parallel to the vertical and horizontal sides are set. This may be set as a determination rectangle.

  FIG. 19C shows a state in which rectangles (third face rectangles) 330-1 and 330-2 obtained by adding a predetermined offset area to the second face rectangles 320-1 and 320-2 are set. . This may be set as a determination rectangle.

  In FIG. 19D, a plurality of faces are in a predetermined positional relationship (the determination rectangles overlap) based on the determination rectangles (here, the third face rectangles 330-1 and 330-2) of each face. It is determined that there is a rectangle, and a rectangle (fourth rectangle) 350 is set in order to set one cutout region for cutting out as two integrated faces.

  FIG. 19E shows a state in which a rectangle (fifth rectangle) 360 expanded to have an aspect ratio of a: b with respect to the fourth rectangle 350 is set. Here, a margin for a predetermined pixel may be provided on the top.

  FIG. 19F shows a provided image generated based on the fifth rectangle 360. The provided rectangle 420 may be generated by setting the fifth rectangle 360 as a cutout area from the captured image, cutting out the image of the cutout area, and enlarging the image to a predetermined size.

  20A and 20B are diagrams for explaining a case where there are a plurality of cut-out areas. As shown in FIG. 20A, when the determination rectangles 330-1 and 330-2 for each face do not overlap, it is determined that the plurality of faces are not in a predetermined positional relationship, and the two faces are individually detected. Two cut areas 410-1 and 410-2 for cutting into two are set.

  Then, as shown in FIG. 20B, the images of the two cut areas 410-1 and 410-2 are cut, combined (joined), and enlarged to generate a provided image.

  FIGS. 21A and 21B are diagrams for explaining a case where a horizontally long provided image with one cutout region is generated. As shown in FIG. 21A, since the determination rectangles 330-1 and 330-2 for each face are overlapped, it is determined that a plurality of faces are in a predetermined positional relationship, and the two faces are cut out as a unit. One horizontally long cut area 410 ′ is set for this purpose. Here, the aspect ratio a: b (a <b) of the cut-out area is set to be horizontally long.

  Then, as shown in FIG. 21B, an image of one cutout area 410 'is cut out and enlarged to generate a provided image.

  22A and 22B are diagrams for explaining the case where there are a plurality of cut-out areas. As shown in FIG. 22A, when the determination rectangles 330-1 and 330-2 of each face do not overlap, it is determined that the plurality of faces are not in a predetermined positional relationship, and the two faces are individually set. Two cutout areas 410′-1 and 410′-2 are set (in which the aspect ratio is longer than in the case of FIG. 20A).

  Then, as shown in FIG. 22B, the images of the two cut areas 410'-1 and 410'-2 are cut out, combined (joined), and enlarged to generate a provided image. Here, the aspect ratio a: b (a <b) of the cut-out area is set to be horizontally long.

  FIG. 23 is a diagram for describing a provided image according to the present embodiment.

  When the photographing process is completed, the photographed image or the composite image is stored as a captured image, and the user can select a desired image from the captured image and perform editing such as “doodle” or decoration.

  In the present embodiment, captured images (captured images or synthesized images) 510 and 520 and provided images 512 and 522 generated corresponding to these images are displayed in a list on the display unit, and the user selects a desired image. It can be so. In this way, the user can select the captured images themselves 510 and 520, or select the image 522 that has been subjected to the automatic up process and the image 512 that has been subjected to the cut-out synthesis process. It is also possible to make various selections according to the user's preference.

  FIG. 24 is a flowchart showing a flow of processing for generating a provided image by performing automatic up or cut-out synthesis.

  This process is performed on a captured image (a captured image or a background extracted image obtained by extracting a background from the captured image).

  First, face position data of a captured image is acquired (step S210). Based on the face position data, a determination rectangle corresponding to the face is obtained (step S220). Based on the face position data, it is determined whether or not the captured image includes a plurality of faces. If not (NO in step S230), the process proceeds to step S250. If it is included (Y in step S230), it is determined whether or not the determination rectangle has a predetermined positional relationship. If it is in the predetermined positional relationship (Y in step S240), a plurality of faces are determined. One cut-out area of the first aspect ratio (a: b) for cutting out the image as a unit is set (step S250), and a provided image is generated based on the image of the set one cut-out area (step S260). When not in a predetermined positional relationship (in the case of N in step S240) or when the captured image does not include a plurality of faces (in the case of N in step S230), to cut out a plurality (n) of faces individually N cut areas having the second aspect ratio (a: b / n) are set (step S270), and images for the set n cut areas are synthesized to generate an editing image (step S280). ).

  In the above-described embodiment, the photographing apparatus has been described by taking a business photo printing apparatus as an example, but the present invention is not limited thereto. For example, a photographic device or a photo sticker printing device may be used, or an information device (such as a portable communication device such as a mobile phone, a portable game machine, a portable information terminal, or a personal computer) equipped with a photographing function may be used. For example, an imaging device such as an electronic camera may be used. The present invention can also be applied to various game systems such as a business game system, a home game system, a portable game system, and a system board for generating a game image.

PM imaging device, RM price receiving unit, CM imaging unit, EM editing operation receiving unit, PB photo sticker providing unit, SP speaker, RA receiving area, RMN receiving monitor, CI coin slot, RL receiving lighting unit, MR mirror MR, BT imaging transition button, CA imaging area, ST sheet, CR camera, CL imaging illumination unit, CMN imaging monitor, EA editing area, EMN editing monitor, TP touch pen, PA extraction area, PO photo sticker exit, 100 Processing Unit, 110 Background Extracted Image Generation Unit, 112 Confirmation Image Display Control Unit, 114 Cursor Image Display Control Unit, 116 Image Capture Control Unit, 118 Guidance Processing Unit, 120 Editing Processing Unit, 122 Clipping Area Setting Unit, 124 Provided Image Generator, 130 image generator, 140 sound generator, 160 operation Department, 170 storage unit, 174 drawing buffer, 180 information storage medium, 190 a display, 192 a sound output unit, 194 portable information storage device 195 printing unit, 196 communication unit, 198 imaging unit

Claims (9)

An imaging device including an imaging unit that images a user as a subject,
Photographed image taken by the shooting unit, or acquires the face position data indicating the position of the face of the subject in the composite image obtained by combining the background extraction image and given background image was drawn background from the captured image, wherein Based on the face position data, a cut-out area setting unit that sets a rectangular area including a face portion of the subject and having a predetermined vertical and horizontal ratio as a cut-out area in the captured image or the composite image;
A provision image generation unit that generates a provision image based on an image of a cutout area of a captured image or a composite image, and
The cut area setting unit
When the captured image or the composite image includes a plurality of faces, it is determined whether or not the plurality of faces are in a predetermined positional relationship based on face position data corresponding to the plurality of faces, and the predetermined positional relationship is not satisfied. In such a case, a cutting area is set for each of a plurality of faces, and when the plurality of faces are in a predetermined positional relationship, one cutting area including the plurality of faces is set.
The provided image generation unit
When cut areas are individually set for a plurality of faces, a provided image is generated by combining the images of the plurality of cut areas, and one cut area is set for the plurality of faces. Is a photographing apparatus that generates a provided image based on an image of one cut-out area. In claim 1,
The cut area setting unit
When cutting areas are set individually for n faces, the horizontal ratio of the rectangle that becomes each cutting area is set to a value of 1 / n when one cutting area including a plurality of faces is set. Set,
The provided image generation unit
An imaging apparatus comprising: n cut-out area images arranged in a horizontal direction to generate a provided image. In claim 1 or 2,
The provided image generation unit
An imaging device, wherein the provided image is generated based on an image of at least one cut-out area and a close-up of the face of the subject. In any one of Claims 1 thru | or 3,
The cut area setting unit
An imaging apparatus, wherein a determination rectangle corresponding to each face is set based on the face position data, and a cutout region is set based on the determination rectangle. In any one of Claims 1 thru | or 4,
The cut area setting unit
An imaging apparatus, wherein a determination rectangle corresponding to each face is set based on the face position data, and it is determined whether a plurality of faces are in a predetermined positional relationship based on the determination rectangle. In claim 4 or 5 ,
The cut area setting unit
A rectangle that circumscribes the face rectangle set corresponding to the face position data and whose vertical and horizontal sides are parallel to the vertical and horizontal sides of the confirmation image or a rectangle obtained by adding a predetermined offset area to the rectangle is set as a determination rectangle. An imaging apparatus characterized by that. In any one of Claims 1 thru | or 6.
An imaging apparatus, comprising: an edited processing unit that provides the captured image or the synthesized image and a provided image generated corresponding to any one of them as an image for editing and performs a given editing process. A program for controlling a photographing device including a photographing unit that photographs a user as a subject,
Photographed image taken by the shooting unit, or acquires the face position data indicating the position of the face of the subject in the composite image obtained by combining the background extraction image and given background image was drawn background from the captured image, wherein Based on the face position data, a cut-out area setting unit that sets a rectangular area including a face portion of the subject and having a predetermined vertical and horizontal ratio as a cut-out area in the captured image or the composite image;
A provision image generation unit that generates a provision image based on an image of a cutout area of a captured image or a composite image, and causing the computer to function
The cut area setting unit
When the captured image or the composite image includes a plurality of faces, it is determined whether or not the plurality of faces are in a predetermined positional relationship based on face position data corresponding to the plurality of faces, and the predetermined positional relationship is not satisfied. In such a case, a cutting area is set for each of a plurality of faces, and when the plurality of faces are in a predetermined positional relationship, one cutting area including the plurality of faces is set.
The provided image generation unit
When cut areas are individually set for a plurality of faces, a provided image is generated by combining the images of the plurality of cut areas, and one cut area is set for the plurality of faces. Is a program for generating a provided image based on an image of one cutout region . A method of controlling an imaging apparatus including an imaging unit that images a user as a subject,
Photographed image taken by the shooting unit, or acquires the face position data indicating the position of the face of the subject in the composite image obtained by combining the background extraction image and given background image was drawn background from the captured image, wherein A cutout area setting step for setting a rectangular area including a face portion of the subject and having a predetermined vertical and horizontal ratio as a cutout area based on the face position data,
A provision image generation step for generating a provision image based on the image of the cut-out area of the captured image or the composite image ,
In the cutting area setting step,
When the captured image or the composite image includes a plurality of faces, it is determined whether or not the plurality of faces are in a predetermined positional relationship based on face position data corresponding to the plurality of faces, and the predetermined positional relationship is not satisfied. In such a case, a cutting area is set for each of a plurality of faces, and when the plurality of faces are in a predetermined positional relationship, one cutting area including the plurality of faces is set.
In the provided image generation step,
When cut areas are individually set for a plurality of faces, a provided image is generated by combining the images of the plurality of cut areas, and one cut area is set for the plurality of faces. Is a method for controlling a photographing apparatus, wherein a provided image is generated based on an image of one cutout region .