JP5983717B2 - Imaging apparatus, imaging method, and program thereof - Google Patents

Description

  The present invention relates to a photographing apparatus, and more particularly to a photographing apparatus that photographs a user with a camera and outputs a composite image generated based on the photographed image as a photograph or the like.

  2. Description of the Related Art Conventionally, there is known an amusement photo creation device that is a photographing device that photographs a user with a camera and outputs the photographed image as a photo sticker or a photo card. Such a play photo creation device is highly playable or entertaining, and therefore, a variety of background images and foreground images (foreground images) prepared in advance according to the user's preference are combined with images to be synthesized. For example, a frame image or a stamp image is often selected, and a user can freely draw using a touch pen. Since such an operation by the user is performed on the photographed image, it is called “graffiti”.

  In such an amusement photo creation apparatus, a wide-angle lens is often used for a camera in order to take a picture in a limited shooting room. This wide-angle lens is known to generate distortion called barrel distortion, and various techniques have been conventionally used for correcting the deformation of a captured image caused by such distortion.

  For example, Japanese Patent Application Laid-Open No. 2001-311993 discloses a configuration of a photo booth that softly corrects a photographed user's image after zooming in or out using a camera.

JP 2001-311993 A

  Here, when a person is photographed in the photographing room in the play photo creating apparatus, the person's foot portion may appear unnaturally. That is, typically, the position of the camera provided in the play photo creation device is usually near the front of a standing person and is at or above the chest of the person, and the person is photographed from that position. Then, a person's foot may appear unnaturally short.

  Further, the part from the ankle to the toe of a standing person is usually substantially parallel to the ground. For this reason, the way in which this part and the part above the ankle that is substantially perpendicular to the ground are different, and as a result, it may appear unnatural.

  Even if such an unnatural image is subjected to known image processing for correcting the barrel distortion, geometric distortion, etc., on the image taken by the game photo creation device, the fundamental improvement is not achieved. I can't hope.

  Therefore, an object of the present invention is to provide a photographing apparatus, a photographing method, and a program for performing image correction so that a foot portion of a user to be photographed is not unnaturally photographed.

The present invention, a user is photographed as the subject, so that the image ranging photographed upper limit position of the ankle or et predetermined subject in the captured image is enlarged in the vertical direction, deformation processing on the captured image Is to do. In addition, the present invention captures a user as a subject, detects bending of the foot of the subject included in the captured image, and performs deformation processing on the captured image based on the detection result. It is.

In addition, as a structure relevant to this invention, the following can be considered.
A first configuration is an automatic photo creation apparatus including a photographing unit that captures an image including a user who is a subject, and an output unit that outputs a composite image including a captured image obtained by the photographing unit as a photograph. ,
Image synthesis means for generating the synthesized image so as to reduce a toe image corresponding to a photographed image that should include an image from the vicinity of the user's ankle to the toe included in the photographed image in a predetermined direction; It is characterized by.

The second configuration is the same as the first configuration,
The image composition means enlarges a leg image that should include an image from the vicinity of the user's ankle to a predetermined upper limit position in the direction so that the reduced foot image and the enlarged leg image are in contact with each other. And generating the composite image.

The third configuration is the second configuration,
The image compositing means has, as a lower side, a line segment including a predetermined position where an ankle should be captured in the captured image, among line segments connecting from the left end to the right end of the captured image in the left-right direction of the captured image, In the captured image, a rectangular area having a line segment including a predetermined upper limit position where the knee should be captured as an upper side is defined as the leg image, the foot image is reduced in the downward direction in the captured image, and the leg image is displayed. Is expanded in the downward direction.

The fourth configuration is the second or third configuration,
The image compositing means has, as an upper side, a line segment including a predetermined position where an ankle should appear in the photographed image among line segments connecting the left and right ends of the photographed image in the left-right direction of the photographed image, A rectangular region having a lower side of a line segment including the lower end of the captured image or a line segment including a predetermined position where the foot tip should be captured in the captured image is defined as the foot image, and the foot image is defined in the captured image. The image is reduced in the downward direction and the leg image is enlarged in the downward direction.

The fifth configuration is any one of the second to fourth configurations,
The image composition means matches the lower end position of the foot image before reduction in the captured image with the lower end position of the reduced foot image, and the upper end of the leg image before enlargement in the captured image. The foot image is reduced and the leg image is enlarged so that the position matches the upper end position of the enlarged leg image.

The sixth configuration is the second configuration,
Face detection means for determining whether or not the image of the user's face is included in the leg image;
The image composition means enlarges the leg image in the direction at a first magnification when the face detection means determines that the face image is not included, and the face detection means If it is determined that an image is included, the leg image is not enlarged, or the leg image is enlarged in the direction at a second enlargement factor smaller than the first enlargement factor. .

The seventh configuration is the second configuration,
A background presenting means that is a subject to be photographed by the photographing means and has a surface to be photographed with a predetermined color;
Mask creating means for creating a chroma key mask corresponding to the color of the area occupied by the image of the surface to be photographed included in the photographed image;
A user image extracting means for extracting a user image including only the user image by using the chroma key mask created by the mask creating means;
The image synthesizing unit generates a new synthesized image by performing a chroma key process for synthesizing the synthesized image including the user image and a predetermined background added image.

The eighth configuration is the seventh configuration,
An ankle detection means for detecting the position of the user's ankle based on the user image extracted by the user image extraction means;
The image synthesizing unit determines the foot tip image and the leg image based on an ankle position detected by the ankle detection unit.

The ninth configuration is the eighth configuration,
The image synthesizing unit is calculated based on a position of at least one ankle of the user detected by the ankle detection unit among line segments connecting the left end and the right end of the captured image in the left-right direction of the captured image. Within a rectangular area or a rectangular area having a line segment that includes a position where the tip of the image is taken as the upper side and a line segment that includes a predetermined position where the foottip is to be captured in the captured image as the lower side. The corresponding portion of the user image included is defined as the foot image, and the foot image is reduced in the downward direction in the captured image and the leg image is expanded in the downward direction.

The tenth configuration is the eighth or ninth configuration,
Further comprising knee detecting means for detecting the position of the user's knee based on the user image extracted by the user image extracting means;
The image synthesizing unit is calculated based on a position of at least one knee of the user detected by the knee detecting unit among line segments connecting the left end and the right end of the captured image in the left-right direction of the captured image. A rectangular area having a line segment including a position calculated on the basis of the position of at least one ankle of the user detected by the ankle detection means as a lower side or within the rectangular area The corresponding portion of the user image included is defined as the leg image, and the foot image is reduced in the downward direction in the captured image, and the leg image is enlarged in the downward direction.

The eleventh configuration is the eighth configuration,
The ankle detection means detects the position of each ankle of one or more users based on the user image,
The image synthesizing unit determines a corresponding part of the user image including an image of each ankle and an image of a toe corresponding to each ankle based on the position of each ankle detected by the ankle detection unit. And corresponding portions of the user image including the images of the ankles and the upper limit position corresponding to the ankles are defined as leg images corresponding to the ankles. The composite image is generated such that the reduced foot image corresponding to the image and the enlarged leg image are in contact with each other at the respective ankles.

The twelfth configuration is the eleventh configuration,
Based on the user image extracted by the user image extraction means, further comprising knee detection means for detecting the position of each knee of one or more users,
The image composition means defines the leg image with the position of each knee detected by the knee detection means as the upper limit position corresponding to each ankle.

A thirteenth configuration is an automatic photo creation method including a photographing step of photographing an image including a user who is a subject, and an output step of outputting a composite image including a photographed image obtained by the photographing step as a photograph. ,
An image compositing step for generating the composite image so as to reduce a toe image corresponding to a captured image that should include an image from the vicinity of the user's ankle to the toe included in the captured image in a predetermined direction. It is characterized by.

In the fourteenth configuration, an automatic photo creation device, which is a computer,
A shooting step for shooting an image including a user as a subject;
An output step of outputting a composite image including a photographed image obtained by the photographing step as a photograph;
Executing an image composition step of generating the composite image so as to reduce a foot image corresponding to a captured image that should include an image from the vicinity of the user's ankle to the toe included in the captured image in a predetermined direction. It is a program characterized by making it carry out.

  According to the above invention, it is possible to prevent the user's foot part to be photographed from being unnaturally captured.

  According to the first configuration, the composite image is generated so that the foot image corresponding to the captured image that should include the image from the vicinity of the user's ankle to the toe is reduced in a predetermined direction. It is possible to prevent the target user's feet from appearing unnaturally (so as not to look unnaturally long).

  According to the second configuration, the foot image is reduced in a predetermined direction, the leg image that should include an image from the vicinity of the user's ankle to the predetermined upper limit position is enlarged in the direction, and the reduced foot Since the composite image is generated so that the image and the enlarged leg image are in contact with each other, the continuity of the user's image is maintained and the foot portion of the user to be photographed is not unnaturally captured ( It can be seen naturally long), and the user's legs can be shown long in style.

  According to the third configuration, the rectangular region including the line segment including the predetermined position where the ankle should be captured in the photographed image as the lower side and the line segment including the predetermined upper limit position where the knee should be captured as the upper side. Since it is defined as a leg image and the foot image is reduced downward and the leg image is expanded downward, the user's foot part is unnaturally reflected by a simple configuration using a predetermined position. Can not be. In particular, it can be said that the position of the user (for example, the height of the ankle etc.) in the limited photographing room is almost constant, and thus such a simple configuration is possible.

  According to the fourth configuration, a line segment including a predetermined position where the ankle should be captured is an upper side, and a line segment including the lower end of the photographed image or a line segment including a predetermined position where the foot tip is to be captured The rectangular area as the lower side is defined as the foot image, and the foot image is reduced downward and the leg image is enlarged downward, so that the user's foot can be obtained with a simple configuration using a predetermined position. You can prevent the part from appearing unnatural. Similarly, it can be said that the position of the user (for example, the height of an ankle etc.) in a particularly limited photographing room is almost constant, and thus such a simple configuration is possible.

  According to the fifth configuration, the lower end position of the foot image before reduction and the lower end position of the reduced foot image are matched, and the upper end position of the leg image before enlargement is enlarged. Therefore, the continuity of the user's image can be easily maintained, and the user's foot can be prevented from appearing unnaturally.

  According to the sixth configuration, when the face detection unit determines that the image of the user's face is included in the leg image, the leg image is not enlarged or is smaller than the first enlargement factor. Since the leg image is magnified at an enlargement ratio of 2, the user's foot part to be photographed is not unnaturally photographed (so as not to look unnaturally long), and The uncomfortable feeling that the face is deformed can be eliminated or reduced.

  According to the seventh configuration, the chroma key mask corresponding to the color of the area occupied by the image of the imaged surface is generated, and the user image including only the user image is extracted by using the generated chroma key mask. Since the synthesized image including the user image and the predetermined background added image are synthesized, a highly playable synthesized image can be obtained, and the foot image can be easily obtained by using the extracted user image. The leg image can be enlarged by reducing the size.

  According to the eighth configuration, the position of the user's ankle is detected based on the user image, and the foot image and the leg image are determined based on the detected position of the ankle. Based on the position of the ankle, an accurate range of foot image can be obtained. Therefore, it is possible to prevent the user's foot portion from appearing unnaturally more accurately.

  According to the ninth configuration, a rectangular area determined based on the position of at least one ankle of the user detected based on the user image or a corresponding part of the user image included in the rectangular area is defined as a foot image. Therefore, it is possible to obtain a toe image in a more accurate range than when based on a predetermined ankle position. Therefore, it is possible to prevent the user's foot portion from appearing unnaturally more accurately.

  According to the tenth configuration, a rectangular area determined based on the position of at least one knee of the user detected based on the user image or a corresponding portion of the user image included in the rectangular area is defined as a leg image. Therefore, a leg image in a more accurate range than that based on a predetermined knee position can be obtained. Therefore, it is possible to prevent the user's foot portion from appearing unnaturally more accurately.

  According to the eleventh configuration, the position of each ankle of one or more users is detected based on the user image, the foot image is determined based on the position of each ankle, and the leg image corresponding to each ankle Therefore, even if the position of each foot of each user is different, a foot image corresponding to each ankle position can be obtained, and each foot image can be appropriately reduced. it can. Therefore, it is possible to prevent the user's foot portion from being unnaturally captured without leakage.

  According to the twelfth configuration, the position of each knee of one or more users is detected based on the user image, and the leg image is determined based on the position of each knee. Even if they are different from each other, it is possible to obtain foot images corresponding to the respective ankle positions and leg images corresponding to the respective knee positions. Therefore, it is possible to prevent the user's foot portion from appearing unnaturally and more reliably and accurately.

  According to the thirteenth configuration, an effect similar to the effect of the first invention as the device invention can be achieved in the method.

  According to the fourteenth configuration, an effect similar to the effect of the first invention as the device invention can be achieved in the program.

It is a figure which shows the external appearance of the photo production apparatus for games which is an automatic photo production apparatus which concerns on one Embodiment of this invention. It is a front view of the imaging unit in the embodiment. It is a front view of the edit unit in the said embodiment. It is a front view of the output unit in the said embodiment. It is a schematic diagram which shows the display structure of the touchscreen for imaging operations in the said embodiment. It is a block diagram which shows the structure which looked at the principal part of the photo production apparatus for play concerning the said embodiment from the functional surface. It is a flowchart which shows the procedure of the imaging | photography process in the said embodiment. It is a flowchart which shows the procedure of the graffiti edit process in the said embodiment. In the said embodiment, it is a figure which shows simply the positional relationship of user U1, U2 and the background seen from the camera position vicinity. In the said embodiment, it is a figure which shows simply the positional relationship of the user U and the background seen from the imaging | photography room side surface direction. In the said embodiment, it is a figure which shows the predetermined ankle position and knee position in a picked-up image. In the said embodiment, it is a figure which shows the image obtained as a result of the process by the 1st aspect of a leg balance process. It is a figure which shows the ankle position and knee position of each user which were detected in the picked-up image in the said embodiment, and a corresponding foot tip image and leg part image. It is a figure which shows the user image containing the ankle position and knee position of each user after the process in the 3rd aspect of the leg balance process in the said embodiment, and a corresponding foot tip image and leg part image. In the said embodiment, it is a figure which shows the ankle position and knee position of each user which were detected in the picked-up image different from the example shown in FIG. 13, and a corresponding foot tip image and leg part image.

Hereinafter, an embodiment of the present invention will be described with reference to the accompanying drawings.
<1. Overall configuration>
FIG. 1 is a diagram showing an appearance of a game photo creation device that is an automatic photo creation device according to an embodiment of the present invention. More specifically, FIG. 1 (a) is an external side view of the play photo creation apparatus as viewed from the side, and FIG. 1 (b) is an external plan view as viewed from above. This play photo creation device accepts a shooting room 2 where a user enters, a shooting unit 3 that takes a picture of the user and receives selection of a background image and a foreground image, and an editing operation including graffiti (drawing operation) by the user. An editing unit 4 that generates a combined image combined with the photographed image and an output unit 5 that outputs the combined image are provided. FIG. 2 is a front view of the photographing unit 3, FIG. 3 is a front view of the editing unit 4, and FIG. 4 is a front view of the output unit 5. Hereinafter, the overall configuration of the game photo creation device according to the present embodiment will be described with reference to FIGS. 1 to 4.

  The photographing room 2 has a substantially rectangular parallelepiped shape, and the photographing unit 3 is arranged along the front surface of the photographing room 2 which is a front surface when viewed from the user entering the inside. Note that an opening for a user to enter and exit, and a light-shielding curtain that covers a part or all of the opening are provided on a part of each of the left and right side surfaces of the photographing room 2. In addition, a single color (in this case, blue or green) for chroma key composition processing is attached to the rear surface, which is the rear surface as viewed from the user who enters the photographing room 2. These colors are also attached to the floor surface.

  The photographing unit 3 includes a camera 10 as an imaging unit for photographing a user, strobes 11, 12, 13 L, 13 R, and 14 that are arranged at the top, bottom, left, and right positions of the camera 10, and below the camera 10. It is provided with a shooting operation touch panel 20 that is arranged and receives operations from a user, displays shot images, and the like.

  The camera 10 is typically a digital camera that generates a digital image signal using a CCD (Charge Coupled Device), captures a user, and outputs an image signal representing the captured image. Here, the camera 10 uses a wide-angle lens, and it is assumed that barrel-shaped distortion occurs in the captured image. The flashes 11 to 14 emit a flash toward the user in order to obtain sufficient light for photographing. The shooting operation touch panel 20 is configured to provide an operation screen for accepting various operations by the user during shooting, and to display an image based on the image signal in real time.

  FIG. 5 is a schematic diagram showing a display configuration of the touch panel 20 for shooting operation. As shown in FIG. 5, the shooting operation touch panel 20 has a real-time preview area 201 for displaying a shot image in real time, a pose sample display area 202 for displaying a pose sample, and a graffiti obtained by shooting. A graffiti target image display area 203 for displaying the target image is included.

  The photographing unit 3 is built around a computer and incorporates a control device for controlling each part, an I / O control device, a network adapter for communicating with the editing unit 4, and the like. The photographing unit 3 includes a coin insertion slot 26 at the lower front side.

  The editing unit 4 is configured around a computer similar to the photographing unit 3, and includes a control device that controls each part and the like, and a network adapter that communicates with the photographing unit 3 and the like. Further, as shown in FIG. 1B, the editing unit 4 is divided into two units 4a and 4b so that two sets of users can play. One of the units 4a includes an editing operation touch panel 400 that includes a graffiti area, an area for displaying a menu for graffiti, a tool, and the like and functions as a GUI (Graphical User Interface) display means, and the editing operation touch panel 400 Four touch pens 49L1, 49L2, 49R1, and 49R2 are provided as pointing devices used for the operation on the. The other unit 4b has the same configuration. Here, the unit 4a seen in the foreground in FIG. 1A is referred to as “graffiti booth A”, and the unit 4b on the opposite side (the back side in the figure) is referred to as “graffiti booth B”.

  The touch panel 400 for editing operation in the present embodiment has a display configuration in which each of the units 4a and 4b typically allows two users to simultaneously doodle. In this description, components used by the user on the left side are given reference signs including “L”, and components used by the user on the right side are assigned reference signs including “R”. ing.

  In addition, the display screen displayed on the editing operation touch panel 400 in the present embodiment is typically divided into two parts on the left and right sides so as to be suitable for the operation of two users. Hereinafter, the left operation screen of the display screen is referred to as a left operation screen, and the right operation screen is referred to as a right operation screen.

  Further, the editing operation touch panel 400 is configured to be able to simultaneously detect the operation positions of the four touch pens 49L1, 49L2, 49R1, and 49R2, and the detected operation positions are those of 49L1, 49L2, 49R1, and 49R2. It is possible to detect which of these operations is supported. For example, when a capacitive touch panel is used as the editing operation touch panel 400, simultaneous detection of such a plurality of operation positions and identification of the operated touch pen are possible.

  Two touch pens for left operation screen (hereinafter referred to as “left screen pen”) 49L1 and 49L2, or two touch pens for right operation screen (hereinafter referred to as “right screen pen”) 49R1 and 49R2, respectively. Although it can be used (input operation) at the same time, it is controlled so that only the touch pen that is active first (operation input is accepted first) and is continuously active can be input. Also good. By controlling in this way, the drawing response can be maintained at high speed.

  The number of touch pens is not particularly limited, and instead of the touch pen, a known pointing device capable of simultaneously detecting coordinates (for example, position determination based on a trackball, cursor keys, camera images, etc.) is used as an input operation means. May be.

  As shown in FIG. 4, the output unit 5 typically has a material image such as a Deco-mail (registered trademark) image or a composite image in which a photographed image is doodled by an infrared communication port built in a mobile phone terminal. Output operation touch panel 30 that is operated by the user when the user is transferred to the user's mobile phone terminal, and the material image or the like that is arranged below the output operation touch panel 30 is directly directed to the mobile phone terminal as an infrared signal. An infrared communication port (non-contact communication port) 31 for transmission and a speaker 32 for notifying the user of operation methods and sound effects necessary for the communication are provided. Further, the output unit 5 includes an outlet 33 for taking out a photo sticker, a photo card, or the like on which a composite image that has been edited by the editing unit 4 is printed.

  When the user wants to view the photo sticker on which the composite image is printed on the mobile phone terminal, the output operation touch panel 30 transmits the photo sticker to the mobile phone terminal having the infrared communication function (or other proximity wireless communication function). It is configured to provide an operation screen for accepting various operations necessary for this.

  Such an output unit 5 is also configured around a computer similar to the photographing unit 3, and includes a control device for controlling each part, a network adapter for communicating with the editing unit 4, etc. A network printer 35 for printing an image as a photo sticker or the like is provided.

  In the configuration as described above, after shooting in the shooting room 2, the user uses the graffiti booth A of the editing unit 4 or the touch panel 400 for the editing operation of the graffiti booth B to generate based on the shot image. Graffiti is performed on the graffiti target image. Then, the user prints the composite image generated by the graffiti by the network printer, transmits the image to the mobile phone terminal having the infrared communication function, and displays the image received by the mobile phone terminal on the terminal screen. .

<2. Functional configuration>
FIG. 6 is a block diagram showing the configuration of the main part of the play photo creating apparatus according to the present embodiment as seen from the functional aspect. As shown in FIG. 6, this game photo creation device functionally has a shooting processing unit 7 for mainly performing a process of shooting a user (shooting process), and a user's graffiti mainly for a graffiti target image. Edit processing unit 8 for editing the graffiti target image according to the operation, and output the graffiti target image subjected to the editing process as a photo sticker or the like, or output it to a mobile phone terminal using infrared communication And an output processing unit 9 that performs processing (output processing).

  The imaging processing unit 7 includes a first control unit 70, an imaging unit 71, a first display / operation unit 72, an I / O control unit 73, an illumination unit 74, and a first communication unit 75. It is configured. The edit processing unit 8 includes a second control unit 80, second display / operation units 81 and 82, and a second communication unit 83. The output processing unit 9 includes a third control unit 90, a third display / operation unit 91, a print output unit 92, an audio output unit 93, a non-contact communication unit 94, and a third communication unit 95. It is constituted by. The first, second, and third communication units 75, 83, and 95 that are network adapters can communicate with each other via a network 6 that is a LAN (Local Area Network).

  The imaging unit 71 corresponds to the camera 10 configured using an imaging element such as a CCD, and captures an image in real time and outputs an image signal representing the image (captured image). This image signal is input to the first control unit 70 and temporarily stored as photographed image data in its internal memory. The captured image data is supplied as a captured image signal from the first control unit 70 to the first display / operation unit 72, and a captured image based on the captured image signal is displayed in real time.

  The first display / operation unit 72 corresponds to the shooting operation touch panel 20 and is used for selecting a background image and a foreground image to be combined with a shot image and determining a layout of a photo to be output. The user's operation and shutter operation are accepted. Signals indicating these operations are input to the first control unit 70 as operation signals. Here, when a predetermined process for photographing the user (corresponding to the selected photographing menu) is started, guidance for the user is displayed on the first display / operation unit 72, and thereafter Based on an instruction from the first control unit 70, a flash is emitted from the flashes 11 to 14 in the shooting direction of the camera 10 after a predetermined time of about several seconds. At that time, an image signal output from the imaging unit 71 as a signal representing a photographed image of the user is input to the first control unit 70, a memory in the first control unit 70, a hard disk device as an auxiliary storage device, or the like. Stored as captured image data.

  The illumination unit 74 corresponds to the strobes 11, 12, 13 L, 13 R, and 14 disposed at the top, bottom, left, and right positions of the camera 10, and is turned on / off by the I / O control unit 73 based on an instruction from the first control unit 70. Turning off and dimming are controlled. The I / O control unit 73 corresponds to an I / O control device built in the photographing unit 3, and controls the illumination unit 74 based on an instruction from the first control unit 70. Further, an input signal such as a detection signal from a coin detection unit (not shown) described later is transferred to the first control unit 70. The first communication unit 75 corresponds to a network adapter built in the photographing unit 3 and functions as an interface for data transmission / reception via the network 6.

  The first control unit 70 is built in the photographing unit 3 and corresponds to a control device mainly composed of a computer including a CPU, a memory, a frame buffer, a timer, an auxiliary storage device and the like, and is stored in a predetermined memory stored in an internal memory. When the CPU executes the program, an instruction is given to each unit as described above in order to control each unit based on the operation signal or the like input as described above. A graffiti target image is generated based on the photographed image. The generated graffiti target image is written in the frame buffer and displayed on the first display / operation unit 72. Further, the first control unit 70 generates a composite image that is an image in which a predetermined background image or foreground image is drawn on the graffiti target image. As will be described in detail later, this graffiti target image has a background color that is a key color used to create a mask, and based on the well-known chroma key composition processing method, the image includes only the user's image. An image part is extracted and synthesized so that the photographed image part is inserted into the background image. Various correction processes described later are performed on a captured image portion including only the captured image or the user image. The composite image generated in this way is displayed on the first display / operation unit 72 by being written in the frame buffer. When the shooting and the generation of the composite image are completed in this way, the generated composite image is appropriately selected according to the input operation of the user, and then the second corresponding to the editing unit 4 via the first communication unit 75. It is sent to the control unit 80.

  In addition to the above-described components, the photographing unit 3 is further provided with a coin detection unit (not shown) for detecting coins inserted into the coin insertion slot 26 in the photographing unit 3, and the first control unit 70 includes Based on the detection result of the coin detection unit, each unit is controlled so as to allow the user to play with the present photo creation device, such as shooting, selection of the background image and foreground image, and graffiti for a predetermined time. Since the detection operation by the coin detection unit and the control operation by the first control unit 70 based on the detection result are the same as those of the conventional game photo creation device and are well known, detailed description thereof will be omitted.

  The second control unit 80 is a built-in editing unit 4 and corresponds to a control device that is configured around a computer including a CPU, a memory, a frame buffer, an auxiliary storage device, and the like, and is a predetermined program stored in an internal memory. Is executed by the CPU to perform overall control related to the graffiti editing process. In addition, the second control unit 80 generates a composite image that is an image obtained by drawing a predetermined image on the graffiti target image based on an operation signal for performing graffiti processing on the captured image sent from the first control unit 70. Generate. The generated composite image is displayed on the corresponding second display / operation unit 81 or 82 in accordance with a user instruction. When the generation of the composite image as described above is completed, the composite image is sent to the output unit 5. If the print output unit 92 is outputting another composite image, a message to that effect is displayed and sent after waiting for completion.

  The second display / operation units 81 and 82 correspond to the editing operation touch panel 400 that mainly functions as a GUI display unit for graffiti, and accepts a user's operation using a touch pen. The second communication unit 83 corresponds to a network adapter built in the editing unit 4 and functions as an interface for data transmission / reception via the network 6.

  The third control unit 90 is built in the output unit 5 and corresponds to a control device mainly composed of a computer including a CPU, a memory, a frame buffer, a timer, an auxiliary storage device, and the like, and is stored in an internal memory. When the CPU executes a predetermined program, overall control relating to output processing is performed. The third control unit 90 stores the composite image sent from the second control unit 80 in the memory as composite image data. The print output unit 92 corresponds to the network printer 35 built in the output unit, and prints a plurality of composite image data stored in the memory as a photo sticker (or photo card) (appropriately laid out). The printed photo sticker or the like is taken out from an outlet 33 provided at the lower front of the output unit 5.

  Further, the third control unit 90 can start printing processing such as a photo sticker based on the composite image sent from the editing unit 4 and simultaneously download the composite image data created by the user to the mobile phone terminal. As described above, an operation screen for accepting a user's input operation is displayed on the third display / operation unit 91. The third display / operation unit 91 corresponds to the output operation touch panel 30 and functions as an input unit that inputs the received input operation to the third control unit 90 as an operation signal.

  Based on the input operation signal, the third control unit 90 issues an instruction to the non-contact communication unit 94 to transmit image data to the user's mobile phone terminal. The non-contact communication unit 94 corresponds to the infrared communication port 31. The image data is directly transmitted from the non-contact communication unit 94 to the user's mobile phone terminal as a non-contact communication infrared signal. The third communication unit 95 corresponds to a network adapter built in the output unit 5 and functions as an interface for data transmission / reception via the network 6.

  Here, the third control unit 90 writes a demo image (demonstration image) or a mini game image stored in advance in the auxiliary storage device in the frame buffer while the input operation is not performed. Is displayed on the third display / operation unit 91. The audio output unit 93 corresponds to the speaker 32. The audio output unit 93 explains the input operation method to the user in conjunction with the operation screen displayed on the third display / operation unit 91, and the third display / operation unit 91 displays a demo image or a mini game. Play music, sound effects, etc. when displayed.

  Here, the predetermined program executed in each control device is provided by, for example, a DVD-ROM which is a recording medium on which the program is recorded. That is, a DVD-ROM as a recording medium for the predetermined program is attached to a DVD-ROM driving device built in the control device as an auxiliary storage device, and the predetermined program is read from the DVD-ROM as an auxiliary storage device. Installed on the hard disk drive. The predetermined program may be provided via a recording medium (CD-ROM or the like) other than a DVD-ROM or a communication line. Then, when an operation for starting the present photo creating device is performed, the predetermined program installed in the hard disk device is transferred to the memory in the control device and temporarily stored therein, It is executed by the CPU. Thereby, the control process of each part by the control device is realized.

  The first to third control units 70, 80, 90 have been described as corresponding to devices including different computers built in different units, but such a configuration is an example, and The first to third control units 70, 80, 90 may be realized by two or less devices or four or more devices. In this case, a program corresponding to the function to be realized is executed in each device. In addition, the photographing unit 3, the editing unit 4, and the output unit 5 may be configured by two or less units or four or more units.

<3. Processing procedure in a photo creation device for play>
As described above, the play photo creating apparatus includes the photographing unit 3, the editing unit 4, and the output unit 5. The photographing unit 3 performs photographing processing, the editing unit 4 performs later-described graffiti editing processing, and the output unit 5 performs output processing. It should be noted that when a certain user is playing with the photographing unit 3, another user can play with the editing unit 4, and yet another user can output a composite image with the output unit 5. ing. In other words, this game photo creation device can perform shooting processing, graffiti editing processing, and output processing in parallel. Below, the outline | summary of the procedure of an imaging | photography process, graffiti edit processing, and an output process is demonstrated.

<3.1 Shooting process>
FIG. 7 is a flowchart showing the procedure of the photographing process in the present embodiment. When the play photo creation device is not used (when no play is performed), a demonstration image is displayed on the shooting operation touch panel 20. When the user inserts a coin into the coin insertion slot 26 while displaying the demo image, play is started (step S100).

  When the play is started, the first control unit 70 accepts selection of a shooting mode by the user (step S110). In step S110, for example, image quality (specifically, one of clear image quality with high contrast, soft image quality with softness, or cool image quality with transparency) is selected, brightness is selected, Whether to shoot automatically or manually is selected. When shooting automatically, a shooting theme is selected. In this case, the first control unit 70 displays on the shooting operation touch panel 20 a screen for allowing the user to select one or more shooting themes from a plurality of shooting themes prepared in advance. The selection operation by the person is accepted. Then, the first control unit 70 acquires selection information based on a user's selection operation, and determines a combination of a frame and a background to be used for shooting based on the selected shooting theme. When shooting manually, the user can freely determine the frame and background. Thereafter, the process proceeds to step S120, and photographing is performed. By this shooting, the shot image data is stored in the memory of the first control unit 70.

  In step S <b> 130, a graffiti target image (an image including a captured image) generated based on the captured image is displayed on the capturing operation touch panel 20. Specifically, each time the process of step S130 is performed, graffiti target images are additionally displayed sequentially in the graffiti target image display area 203 of the photographing operation touch panel 20 shown in FIG. Thereafter, the process proceeds to step S140, and the first control unit 70 determines whether or not a predetermined number of images have been taken. As a result of the determination, if the number of images has been shot, the process proceeds to step S150. If the number of images has not been shot, the process returns to step S120. In practice, a time limit for photographing (for example, 3 minutes) is provided.

  In step S150, an image to be actually graffitied is selected (by the user) from a plurality of graffiti subject images. Specifically, the first control unit 70 displays a list of graffiti target images on the shooting operation touch panel 20 in order to allow the user to select an image to be used for graffiti and printing, and performs a selection operation by the user. Accept. After step S150 ends, the process proceeds to step S160.

  In step S160, a correction process is performed on the selected image to be graffitied to eliminate known distortion, typically barrel distortion caused by the camera 10 using a wide-angle lens. In this correction process, for example, a reduction deformation process is performed such that a square image is deformed into a pincushion shape. Since such correction processing is well known, detailed description thereof is omitted. Further, here, a process of correcting geometric distortion, for example, distortion caused by a relatively high shooting position of the camera for a user in a standing posture (such as a leg that looks short) may be performed. After step S160 ends, the process proceeds to step S170.

  In step S170, a chroma key composition process is performed on the image to be graffiti corrected for distortion. This chroma key composition process generally includes a process for creating a chroma key mask and a process for composing a background image. The chroma key mask is, for example, a well-known composite mask (alpha channel), and an area having a key color (in this case, blue or green) in a target photographed image, that is, an area occupied by an image on the back of the photographing room 2. Is data for making transparent and making other areas non-transparent. In this composite mask, a value α indicating transparency of 0 (completely transparent) or 255 (completely non-transparent) is generally set for each pixel of the target image. By using such a chroma key mask, the background image region corresponding to the non-transparent region (α = 255) of the photographed image is hidden (masked), and the background image corresponding to the transparent region (α = 0). This area is synthesized so that it can be seen. Therefore, a composite image can be obtained as if a non-transparent area portion of the target captured image is cut out and the cut-out portion is pasted on the background image. With such chroma key processing, it is possible to create a photo with high playability. The background image is appropriately selected by the user during shooting, but may be selected according to the user's selection operation during the graffiti process described later.

  In step S180, a leg balance correction process, which is a characteristic process of the present embodiment, is performed on the graffiti target image that has been subjected to the chroma key composition process so that the foot portion of the user to be photographed does not appear unnatural. Is called. Instead of the graffiti target image, a leg balance correction process may be performed on the photographed image itself, or a photographed image portion including only the user image corresponding to the transparent area before the chroma key composition (hereinafter referred to as a “compressed image”). Leg balance correction processing may be performed on the “user image”. The leg balance process may have a plurality of processing modes, and details of the process will be described later. Then, the first control unit 70 sends the image corrected as described above to the second control unit 80 as an actual graffiti target image. After step S180 ends, the process proceeds to step S190.

  In step S190, a guidance screen is displayed. Specifically, the first control unit 70 displays a screen for guiding the user to any one of the editing units 4 (4 a or 4 b) on the photographing operation touch panel 20. Thereby, the photographing process is completed.

<3.2 Graffiti editing process>
FIG. 8 is a flowchart showing the procedure of graffiti editing processing in the present embodiment. The graffiti editing process is realized by the second control unit 80 operating as shown in FIG. 8 based on a predetermined program. In this process, after the above-described shooting process ends, the second control unit 80 sends the graffiti target image sent from the first control unit 70 via the network (LAN) 6 (by the user in step S150 in FIG. 7). The image selected and corrected in steps S160 and S170) is acquired (step S200). Thereafter, the timer 46 is set to a predetermined time (specifically, a time during which graffiti is permitted), and a countdown is started (step S210).

  After the count-down of the timer 46 is started, a graffiti editing operation screen is displayed on the editing operation touch panel 400 of each of the units 4a and 4b constituting the editing unit 4, and a graffiti operation (drawing operation) by the user is accepted (step S232). ~ S236). At the time of this graffiti editing process, the editing unit 4 may accept an editing operation for creating a material image such as a decomail image in addition to an editing operation on the graffiti target image based on the photographed image.

  Specifically, when one of the buttons and tools in the editing operation screen is touched with the touch pen, the coordinate value is input (S232), and the corresponding graffiti screen processing (S234) is performed. Thereafter, in step S236, it is determined whether the graffiti has ended by the remaining time of the timer 46 becoming zero (or performing an operation to be ended by the user), and if not completed (in the case of No in S236), step Returning to S232, the process is repeated until the graffiti is completed, and when the graffiti is completed (Yes in S236), the process proceeds to step S240.

  In step S240, the division pattern of the photo to be output is selected. Specifically, the second control unit 80 displays on the editing operation touch panel 400 a screen for allowing the user to select one of the plurality of division patterns prepared in advance, and the user can select the division pattern. The selection operation by is accepted. And the 2nd control part 80 acquires selection information based on a user's selection operation. After the process of step S240 is complete | finished, it progresses to the process of step S250.

  In step S250, a guidance screen is displayed. Specifically, the second control unit 80 displays a screen for guiding the user to the output unit 5 on the editing operation touch panel 400. Thereby, the graffiti editing process ends. As described above, when the graffiti editing process is completed, the output unit 5 starts a printing process such as a photo sticker based on the composite image sent from the editing unit 4.

  Among the processes described above, as described above, the leg balance correction process in step S180 has a characteristic content, and a plurality of processing modes can be considered. Hereinafter, it demonstrates concretely in order. The leg balance correction process (S180) is performed by the first control unit 70 immediately after the end of shooting, but may be performed by the second control unit 80, or may be performed by, for example, the user during the graffiti operation. The configuration may be performed in response to an instruction.

<3.3 Specific contents of leg balance processing>
<3.3.1 First Processing Mode>
FIG. 9 is a diagram simply showing the positional relationship between the users U1 and U2 and the background viewed from the vicinity of the camera position, and FIG. 10 shows the positional relationship between the user U and the background viewed from the side of the photographing room. It is a figure shown simply.

  As shown in FIG. 9, the users U1 and U2 are standing on the floor surface 120 with the chroma key color with the back surface 110 with the chroma key color inside the photographing room 2 as the back. Of the back surface (background) 110 and the floor surface 120, the photographing range 101 photographed by the camera 10 is indicated by a dotted line. Here, it is assumed that the users U1 and U2 are standing side by side, and the distance from the back to the ankle is almost the same. Therefore, in FIG. 10, the users U1 and U2 are shown as the user U without being particularly distinguished.

  As can be seen from FIG. 10, depending on the angle of view of the camera 10, an object within a region surrounded by a dotted line (including the user U) is photographed by the camera 10. The user's U from the ankle to the toe is positioned parallel to the floor surface 120. If the angle between the line segment indicated by the alternate long and short dash line and the floor 120 in the figure from the camera 10 to the ankle is greater than 45 degrees, the portion from the ankle to the toe of the user U photographed by the camera 10 The length of is longer than the actual length. Even when the angle is 45 degrees or less, the virtual (or psychological) viewpoint position when viewing a portrait is placed at a position farther from the person than the actual camera position. There are many cases. In particular, in order to prevent the foot from appearing short (photographed) due to the shooting position of the camera 10 being high from the floor, when performing geometric distortion correction and the like in step S160 described above, In many cases, a virtual viewpoint is placed at a position farther (a little below) from the actual camera position. For this reason, it may be felt that the length from the ankle to the toe of the user U included in the captured image is unnaturally extended regardless of the angle.

  Therefore, in the first aspect of the leg balance process shown in Step 180, the image portion including the ankle to the toe of the user U (hereinafter referred to as “foot image”) is reduced downward, and the user U's The above-mentioned unnatural appearance is eliminated by enlarging an image portion including the knee to the ankle (hereinafter referred to as “leg image”) downward.

  A configuration in which the length from the ankle to the toe of the user U is reduced to such an extent that it is not felt that the user U is unnaturally extended, and the process of enlarging the leg image is not considered. With this configuration, it is possible to prevent the foot portion of the user to be photographed from appearing unnaturally (so as not to look unnaturally long). However, with this enlargement process alone, if only the length from the ankle to the toe is reduced, there may be a sense of incongruity due to the reduction in only that part. In addition, even if a process for correcting the leg so that it does not appear short by performing geometric distortion correction or the like in step S160, if only the length from the ankle to the toe is reduced, the leg is shortened as a whole. It may be visible. Further, by enlarging the leg image so as to extend the length from the knee of the user U to the ankle, an effect of making the user U's style look better (than actual) can be obtained. For this reason, it is preferable that the leg image is enlarged together with the reduction of the foot image.

  In addition, it is preferable to extend the length from the vicinity of the user U's knee to the vicinity of the ankle in order to make it look good, but for example, the user's U thigh, the base of the foot, or the vicinity of the waist (or that) Even if the length from an upper predetermined position) to the vicinity of the ankle is extended, the overall appearance can be improved. Therefore, the leg image is not limited to the image portion including the vicinity of the knee to the vicinity of the ankle as described above, and may be an image portion including approximately all or a part of the foot of the user U.

  Here, in the first processing mode, the ankle position and the knee position of the user U are determined to be predetermined positions. That is, when the user takes a standing posture, the positions of the hands and elbows change variously, so it is almost impossible to determine in advance. However, among the positions of the ankles and knees of the average height user, it is difficult to predetermine the (standing) position in the left-right direction as viewed from the camera 10, but in the vertical direction as viewed from the camera 10 ( It is relatively easy to predetermine the standing position. This is because a user who faces the back in the limited shooting room 2 often stands at an approximately equal distance from the back, and therefore, unless he takes a posture such as swinging his leg up, This is because the ankle position and knee position of U are substantially constant positions (heights). Therefore, based on a predetermined ankle position and knee position that is estimated (with relatively high accuracy) as the ankle position and knee position of the user U in the photographed image, the reduction of the foot image and the leg image are performed. Enlarge and do. Hereinafter, the contents of the first processing mode will be described with reference to FIGS. 11 and 12.

  In the following, the positions of ankles, knees, toes, and the like are positions that do not consider the position in the vertical direction (Y-axis direction) (that is, the position in the horizontal direction (X-axis direction)) in the captured image (or user image) ).

  FIG. 11 is a diagram illustrating a predetermined ankle position and knee position in a captured image. In FIG. 11, the range (outer periphery) of the photographed image 111 is indicated by a dotted line as being equivalent to the photographing range 101. As described above, since the ankle position and the knee position of the user U are substantially constant positions (heights), the knee position determined in advance based on this is shown as a line segment Pa in the drawing, and the ankle position Is shown as a line segment Pb in the figure.

  Therefore, the leg image is assumed to include the ankle positions of the users U1 and U2 and is assumed to include the knee positions of the users U1 and U2, that is, the line segment Pa is the upper side. The line segment Pb is taken as the lower side, and it can be defined as an area surrounded by these upper and lower sides and the left end (left side) and right end (right side) of the photographed image.

  In addition, it is estimated that the foot image includes the ankle positions of the users U1 and U2, and the image area that is estimated to include the foot positions of the users U1 and U2, that is, the line segment Pb is the upper side. The lower end (lower side) of the photographed image is the lower side, and it can be defined as a region surrounded by these and the left end (left side) and right end (right side) of the photographed image.

  Here, in the case where the lower side of the foot image is matched with the lower end (lower side) of the captured image, the possibility of including the foot position of the users U1 and U2 is very high, but the actual foot position is the position of the captured image. Often above the lower end. That is, as described above, a user who faces the back in the limited imaging room 2 often stands at a substantially equal position from the back, and therefore, similarly to the ankle position and knee position of the user U. The toe position is also substantially constant (height). Therefore, since it is possible to set a line segment that should include the foot position as in the case of the line segments Pa and Pb, the line segment that should include the foot position is replaced with the lower end of the captured image, and the foot image It can also be the lower side.

  Next, since the foot image and the leg image determined as described above are part of the user's image, they must be reduced or enlarged so as to maintain continuity as the user's image. I must. Therefore, the leg image is fixed with the upper side, which is the line segment Pa, expanded downward, and the foot image is fixed, with the lower side, which is the lower end of the captured image (or the line segment that should include the foot position), fixed downward. And the image transformation process is performed so that the lower side of the enlarged leg image and the upper side of the reduced foot image touch each other. An image obtained by this processing is an image finally obtained by the first processing mode of the leg balance processing.

  FIG. 12 is a diagram illustrating an image obtained as a result of the processing according to the first aspect of the leg balance processing. As shown in FIG. 12, the leg image has no change in the position of the upper side (here, line segment Pa) before and after enlargement, and the foot image has the lower side (here, the lower side of the photographed image) before and after enlargement. There is no change in the position, and the position of the side (shown as line segment Pc in the figure) where the leg image expanded by the image deformation process and the reduced foot image are in contact with the position of the line segment Pb It is moving downward.

  Specifically, the toe image is reduced downward by a factor of 0.8 (or any other magnification, of course), and the leg image is enlarged to a position that contacts the reduced foot image. Yes. In this way, the lower end position of the foot image before reduction in the photographed image matches the lower end position of the reduced foot image, and the upper end position of the leg image before enlargement in the photograph image and the enlarged leg portion If the foot image is reduced and the leg image is enlarged so as to match the upper end position of the image, the continuity of the image of the user after the image transformation processing is maintained, and from the ankle to the toe of the user U Can be unnaturally long (photographed), and the legs of the users U1 and U2 can be shown long in a good style. Here, when the degree of reduction of the foot image (reduction ratio) is determined, the degree of enlargement of the leg image (enlargement ratio) is naturally determined as long as the size of the captured image does not change before and after image deformation. The size of the photographed image may be changed before and after the image deformation, and the enlargement ratio may be determined regardless of the reduction ratio.

  The foot image is a rectangular region portion of the photographed image with the line segment Pb as the upper side and the lower end of the photographed image or the line segment that should include the foot tip position as the lower side, and the leg image has the line segment Pa as the upper side. Although described as being a rectangular area portion of the captured image with the line segment Pb as the lower side, it may be a user image portion (extracted by chroma key processing) included in these rectangular areas. This is because there is no difference between the case where the chroma key process is performed after the image deformation process is performed on the photographed image and the case where the image deformation process is performed on the user image after the chroma key process.

  Further, as described above, the leg image does not have to include the vicinity of the user U's knees to the vicinity of the ankles, and therefore, for example, a captured image portion including the vicinity of the waist of the general user U to the vicinity of the ankles. May be a leg image. In this case, the position of the line segment Pa in FIG. 11 is not the knee position (knee height) (of the general user) but the waist position (waist height). For this reason, when a user who is considerably shorter than a general user (typically a child) or a user in a squatting position is included in the shooting target, the leg image is converted by the above image deformation process. When the image is enlarged, a composite image that produces a feeling of strangeness may be generated. In particular, when the face portion of the user is located between the line segments Pa and Pb, and the face image is included in the leg image, the shape of the face is deformed by the enlargement of the image, so that the user is particularly uncomfortable. Will occur.

  Therefore, as a modification of the first aspect of the leg balance process shown in step 180, if the leg image includes an image of the face of the user U, the leg image is not enlarged at all downward, Or the structure which makes small the grade (magnification ratio) to enlarge can be considered. If comprised in this way, the uncomfortable feeling that a face deform | transforms can be eliminated or reduced. Specifically, the first control unit 70 uses a well-known pattern recognition or the like to determine whether or not a captured image between the line segments Pa and Pb that are leg images includes a human face image. If it is determined by face recognition processing based on the technique and it is not determined that a face image is included by the face recognition processing, the same processing as in the first aspect is performed and it is determined that it is included. In this case, the leg image is not enlarged at all in the downward direction, or is enlarged after setting the enlargement ratio to be small, and the ankle image is reduced. In this way, it is possible to eliminate or reduce the uncomfortable feeling that the user's face appears in the captured image is deformed while eliminating the state in which the ankle to toe of the user U looks unnaturally long. For the face recognition, hardware such as a face detection LSI chip may be used. Then, the face detection can be speeded up as compared with the case of software processing.

  Since the image deformation process in the first aspect of the leg balance process as described above is not based on the distortion aberration correction in step S160 or the chroma key composition process in step S170, these processes may be omitted. It may be performed during or before these processes. Moreover, the structure performed simultaneously with distortion aberration correction in step S160 may be sufficient.

  Further, a configuration in which the chroma key composition process in step S170 is associated with the image deformation process and the (actual) ankle position and knee position of the users U1 and U2 are detected based on the user image obtained by the chroma key composition process is also conceivable. By doing so, an accurate ankle position and knee position can be obtained, so that it is possible to more reliably eliminate the state where the ankle to toe of the user U looks unnaturally long, and the legs of the users U1 and U2 can be improved in style. Can show long. Hereinafter, a second aspect of the leg balance process using the user image will be described.

<3.3.2 Second Processing Mode>
In the second processing mode, unlike the first processing mode, the actual ankle position and knee position of the users U1 and U2 are detected based on the user image. As a method for detecting the ankle position and the knee position, various methods such as an image recognition method based on a well-known pattern matching can be considered.

  For example, the ankle position and the knee position may be calculated based on the vertical length of the user image corresponding to each user (specifically, the length from the upper end of the head to the toes). This is because the human ankle position and knee position are often located at a predetermined ratio with respect to height.

  In addition, the waist position is detected from the width in the left-right direction of the user image (typically at the position where the width is the largest), and the waist position (and the position of the toe, etc.) is determined together with the vertical length. By setting the target, the ankle position and the knee position may be calculated. This is because the length of the foot is often determined from the position of the waist.

  Furthermore, by analyzing the shape of the user image, the first bending position from the toe in the image corresponding to each foot part of the user image is detected as the ankle position, and the second bending position is detected as the knee position. May be.

  Furthermore, the image patterns of many user images obtained by photographing users with various postures and their ankle positions and knee positions are stored in advance in association with each other, and the use obtained by the chroma key process is used. An image pattern that approximates the person image and its shape may be searched, and the ankle position and knee position corresponding to the image pattern may be detected as the ankle position and knee position in the both images.

  If the ankle position and the knee position are detected as described above, the line segments Pa and Pb in the first processing mode can be set. Therefore, the image deformation processing is performed in the same manner as in the first processing mode. Accordingly, the state where the ankle to the toe of the user U appear to be unnaturally long can be solved, and the legs of the users U1 and U2 can be shown long in a good style.

  In the second processing mode, as in the first processing mode, the toe image and the leg image need not be a rectangular area portion of the captured image, and are included in these rectangular areas (chroma key). It may be a user image portion (extracted by processing).

  Similarly to the modification in the first aspect, when the leg image includes an image of the face of the user U, the leg image is not enlarged at all or is not enlarged (magnification rate). ) May be reduced. If comprised in this way, the uncomfortable feeling that a face deform | transforms can be eliminated or reduced.

  Here, in the second processing mode, the ankle positions and the knee positions of the users U1 and U2 have been described as being at the same or substantially unchanged positions. Various methods are conceivable, such as averaging the positions of the two, averaging except the most different one, or determining based on one representative position.

  However, in order to perform a more accurate image deformation process, the ankle position and the knee position of the users U1 and U2 are individually detected and then individually (that is, for each user's one leg). Need to do. Therefore, a third aspect of leg balance processing in which individual image deformation processing is performed in this way will be described with reference to FIGS. 13 and 14.

<3.3.3 Third Processing Mode>
FIG. 13 is a diagram illustrating the detected ankle position and knee position of each user in the captured image, and the corresponding foot tip image and leg image. In the figure, the left foot (the right foot in the figure) of the user U2 is placed slightly behind the camera position (in the direction approaching the back of the photographing room 2), and as a result, the position of the ankle of the left foot of the user U2 (Height) is different from the position of both ankles of the user U1 and the position (height) of the ankle of the right foot of the user U2. In addition, such a difference is an example for simplifying the explanation, and the position of each ankle and the position of each knee of each user may be different at any one or more, and in fact all It may be the same.

  Also, in the figure, the knee positions of user U1 and U2 (including the position of the line segment) are indicated as ha, and the position of the line segment including (including the position of both ankles of user U1 and the ankle of the right foot of user U2) ) Is shown as hb, and the position of the ankle of the left foot of the user U2 (including the position of the line segment) is shown as hd.

Further, in the third processing mode, the positions of the toes are also detected, and the positions of the toes of both feet of the user U1 and the toes of the right foot of the user U2 (including the position of the line segment). Is indicated as hc, and the position (including the position of the line segment) of the tip of the left foot of the user U2 is indicated as he.

  As described above, the position of the toe can be easily detected because it is at each lower end of the user image, and the positions of the ankle and knee can be detected by various image recognition methods.

  Here, among the user images generated based on the chroma key process, the leg image A11a on the right foot (left foot in the figure) of the user U1 is a user having the knee position ha as the upper limit and the ankle position hb as the lower limit. The toe image A11b is a user image part having an ankle position hb as an upper limit and a foot position hc as a lower limit. Then, the toe image A11b is reduced 0.8 times downward toward the toe position hc, and the leg image A11a is lowered toward the toe position hc so as to be in contact with the reduced toe image A11b. Magnify in the direction.

  Further, regarding the leg image A12a and the toe image A12b on the left foot of the user U1, and the leg image A21a and the toe image A21b on the right foot of the user U2, the leg image A11a on the right foot of the user U1 and Since the ankle position and knee position are the same as those of the foot tip image A11b, the same enlargement / reduction process is performed.

  Further, the leg image A22a on the left foot of the user U2 is a user image part with the knee position ha as the upper limit and the ankle position hd as the lower limit, and the foot image A22b has the ankle position hd as the upper limit and the foot position he. Is a user image portion with a lower limit of. Since the ankle position hd and the toe position he are different from the ankle position hb and the toe position hc, the toe image A22b is reduced 0.8 times downward toward the toe position he and The partial image A22a is expanded downward toward the foottip position he so as to contact the reduced foottip image A22b. Even if the reduction rate in this image deformation process is the same as the others, the ankle position after the image deformation is the above-mentioned position (because the vertical lengths of the leg image A22a and the toe image A22b are different from each other). This is different from the case of the first processing mode.

  FIG. 14 is a diagram illustrating a user image including an ankle position and a knee position of each user after the image deformation process and a corresponding foot image and leg image. As shown in FIG. 14, the positions of both ankles of the user U1 and the right ankle of the user U2 after the image deformation are both ankle positions hf, but the position of the left ankle of the user U2 is the ankle position. hg, the ankle position is accurately detected for each foot of each user, and appropriate image transformation processing is individually performed. That is, when the position of a certain ankle is different from the positions of other ankles, if the above-described image deformation process is performed uniformly, the deformation of the leg image or the toe image including the ankle may not be appropriate. is there. According to the third processing mode, such inconvenience can be solved, and a state in which the other ankle to toe of the user U looks unnaturally long without leaking more reliably to each foot. It can be solved appropriately, and the legs of the users U1 and U2 can be shown longer in a better style.

  Here, for convenience of explanation, the explanation has been made such that the direction of enlargement or reduction is downward, but this is usually because each leg of a user in a standing posture extends perpendicularly to the floor. It is. Therefore, a direction extending for each foot of the user (for example, a direction along a line segment connecting the two most distant points included in the leg image and the toe image) is calculated, and the direction extending for each foot is calculated. It may be configured to perform enlargement and reduction in the image deformation process. Moreover, the structure which calculates each extending direction separately about the leg part image and foot image in each leg | foot, and performs expansion or reduction may be sufficient.

  In the above description, it has been described that the image deformation process is always performed on each foot. However, for example, when the foot is swung up largely as shown in FIG. 15, the toes are parallel or substantially parallel to the floor. If it is not (typically when the toes are extended away from the floor surface), if image deformation processing is performed on the toe image from the ankle to the toes, it may be uncomfortable. .

  FIG. 15 is a diagram illustrating an ankle position and a knee position of each user detected in a captured image different from the example illustrated in FIG. 13, and a corresponding foot tip image and leg image. As shown in FIG. 15, knee positions ha of users U1 and U2, both ankles of user U1 and ankle positions hb and toe positions hc of the right foot of user U2 are the same as those shown in FIG. However, the ankle position hi and the toe position hj of the left foot of the user U2 are further upward than the case shown in FIG. 13 and extend the toes (that is, from the knee to the toes are almost on the same straight line). In the form).

  Therefore, when the image deformation process in the second processing mode in which enlargement and reduction are performed in an appropriate direction for each foot is performed on each foot, the image of the foot tip on the left foot of the user U2 that should actually be stretched Is reduced, which may cause a sense of incongruity. Therefore, for example, when the foot position of the foot (here, the left foot) is above the ankle position of the other foot (here, the right foot) of the same user U (here, the user U2), the foot Since the tip is usually away from the floor surface, it is assumed that the tip is not parallel to the floor surface and is stretched, so that the image deformation process is not performed on the leg image and the foot image on the foot. .

  In this way, the user U's other ankles to toes appear to be unnaturally long without shrinking from ankles to toes that do not look unnaturally long (ie, should actually look long). It is also possible to make the legs of the users U1 and U2 look longer in style.

<4. Effect>
As described above, according to the present embodiment, the foot image is reduced downward, and the leg image is enlarged to a position in contact with the reduced foot image, while maintaining the continuity of the user's image, It is possible to prevent the foot portion of the user to be photographed from appearing unnaturally (so as not to look unnaturally long), and to make the user's foot look long in style.

3 ... Shooting unit 4 ... Editing unit 5 ... Output unit 6 ... Network (LAN)
DESCRIPTION OF SYMBOLS 10 ... Camera 20 ... Touch panel for photography operation 30 ... Touch panel for output operation 35 ... Network printer 31 ... Non-contact communication port (infrared port)
32 ... Speaker 49L1, 49L2 ... Left screen touch pen 49R1, 49R2 ... Right screen touch pen 70 ... First control unit 72 ... First display / operation unit (touch panel for photographing operation)
80 ... 2nd control part (GUI control apparatus)
81, 82: Second display / operation unit (touch panel for editing operation)
90 ... third control unit 91 ... third display / operation unit (touch panel for output operation)
92 ... Print output section (network printer)
101 ... shooting range 110 ... back surface 400 ... touch panel for editing operation U, U1, U2 ... user

Claims (7)

Photographing means for photographing a user as a subject;
As the image of the range to the upper limit position ankles or these predetermined the subject in the photographed image taken is expanded in the vertical direction by the imaging means, and deformation processing means for performing a modification process on the captured image,
An imaging device comprising: The deformation processing means uses a line segment including the ankle position as a lower side and a line segment including the upper limit position as an upper side among line segments connecting the left end and the right end of the captured image in the left-right direction of the captured image. The imaging device according to claim 1, wherein a leg image, which is a rectangular area, is enlarged downward.   The photographing apparatus according to claim 1, wherein when there are a plurality of subjects, the deformation processing unit performs deformation processing according to the state of the subject for each subject. Photographing means for photographing a user as a subject;
Deformation processing means for performing a deformation process for making the subject's foot included in the captured image look longer with respect to the captured image captured by the imaging means;
With
The photographing apparatus according to claim 1, wherein the deformation processing unit detects a bending of a foot of a subject included in the photographed image, and performs the deformation process on the photographed image based on the detection result. A shooting step for shooting a user as a subject;
As the image of the range to the upper limit position of the ankle or et predetermined the subject in the images picked up in the imaging step is enlarged in the vertical direction, and deformation processing step of performing deformation processing on the captured image,
A photographing method comprising: To the imaging device that is a computer,
A shooting step for shooting a user as a subject;
As the image of the range to the upper limit position of the ankle or et predetermined the subject in the images picked up in the imaging step is enlarged in the vertical direction, and deformation processing step of performing deformation processing on the captured image A program to be executed. On the computer,
A deformation processing step for performing a deformation process on a captured image including a person as a subject to make the subject's foot included in the captured image look longer;
And execute
In the deformation processing step, a bending of a foot of a subject included in the photographed image is detected, and the deformation process is performed on the photographed image based on the detection result.

Images