JP2017034409A - Image processing system, image processing method and computer program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an image processing system, an image processing method, and a computer program that are able to perform processing such that even if legs in a photographic image taken by a photographing game machine is highly unlikely stretched straight, a leg portion look in a natural and ideal figure.SOLUTION: An image processing system comprises: acquiring means for acquiring image data from a photographing device that photographs a subject person; and extracting means that extracts a subject person area from an image based on the acquired image data. The image processing system performs image processing for elongating a range of the image which includes the subject person area, in a direction of a height of the subject person. The image processing system further comprises: determining means for determining whether legs of the subject person in the image are bent or not; and elongating means for elongating the area of a leg portion of the subject person in a different pattern according to whether the determining means determines that the legs are bent or not.SELECTED DRAWING: Figure 5

Description

  The present invention relates to a shooting game machine that can take a photograph of a user, who is a person, and obtain a predetermined photo sticker sheet or photo data in which the shooting result is output, and has a preferable shooting result regardless of the user's pose. The present invention relates to an image processing apparatus, an image processing method, and a computer program in an obtained shooting game machine.

  An apparatus for discharging a photo sticker sheet obtained by photographing a plurality of users or a single user is provided as a shooting game machine. In a shooting game machine, a shooting device that automatically performs shooting, and a shooting device including a flash and a reflector are provided in a shooting space that is large enough for a plurality of users to enter.

  The shooting equipment in the shooting game machine has a depression angle and an angle of view respectively set when shooting a range above the waist of a person existing in the shooting space and when shooting the whole body from the front or obliquely from above. Yes. Especially when shooting the range from the waist and when shooting the whole body from diagonally above, increasing the depression angle raises the user's eyes of the subject, the effect of appearing on the small face and the effect of enlarging the eyes Is obtained.

  Furthermore, the shooting game machine performs correction processing on the photographic image obtained by shooting so that the subject looks cute, such as processing the contour so that it looks like a small face and processing that emphasizes it so that the eyes appear large. Do. As described above, when a whole body is imaged with a large depression angle, it is proposed to perform a process of extending the leg portion and shortening the trunk portion because the leg is easy to be captured (Patent Document 1).

  The user who is the subject is not always photographed in an upright posture in the shooting space. In particular, in a setting for photographing the whole body, the user often takes a pose with various movements. Patent Document 2 proposes eliminating the unnaturalness by not performing the extension process of the leg portion when the foot is away from the floor surface.

JP 2012-169987 A JP2013-162451A

  However, when photographing the whole body, the leg part often appears unnaturally short as disclosed in Patent Document 1, so the leg part must be stretched as disclosed in Patent Document 2. As a result, the degree of satisfaction with the obtained photo sticker sheet and photo data is reduced. Further, it is very difficult to specify the abdominal position and knee position as disclosed in Patent Documents 1 and 2 based on the person subject region extracted from the image. This is because there are various poses taken by the user who is the subject, and it is very difficult to specify the posture of crossing the legs, the posture of crossing the legs, and the position of the knee depending on the clothes.

  The present invention has been made in view of such circumstances, and the leg portion is naturally and ideal even in the case where there is a high possibility that the leg is not in an upright state in a photographic image photographed by a photographing amusement machine. An object of the present invention is to provide an image processing apparatus, an image processing method, and a computer program for realizing processing that looks like a body shape.

  An image processing apparatus according to the present invention includes an acquisition unit that acquires image data from an imaging device that captures a human subject, and an extraction unit that extracts a human subject region from an image based on the acquired image data. In an image processing apparatus that performs image processing for extending a range including a person subject area in a height direction of the person subject, a determination unit that determines whether or not a leg of the person subject in the image is bent; The image processing apparatus includes: an extension unit configured to extend the leg portion region of the person subject in a different pattern depending on whether or not the determination unit determines that the leg is bent.

  In the image processing apparatus according to the present invention, when the extension means determines that the leg is not bent by the determination means, an area corresponding to the lower knee portion of the person subject area is set as another area. A first extension means that extends longer than the first extension means, and a second extension that extends the area of the leg portion of the person subject in a plurality of places when the determination means determines that the leg is bent. Means.

  In the image processing apparatus according to the present invention, the determination unit includes a unit that specifies a contour of the person subject region, and the height in an image of a contour line within a predetermined range including a leg portion of the specified contour. Direction specifying means for specifying the direction with respect to the direction, and when the direction specified by the specifying means is determined to be lateral, it is determined that the leg is bent.

  In the image processing apparatus according to the present invention, the specifying unit includes a unit that specifies a plurality of points on the contour line at predetermined intervals, and a size of a tangent line of the contour line at the plurality of points in the horizontal direction in the image. Means for calculating the total and means for determining whether or not the calculated total is greater than a predetermined value, and when the means is determined to be larger, it is determined to be in the landscape orientation.

  The image processing apparatus according to the present invention is characterized in that the second extension means distributes the extension at a place other than the bent leg portion.

  The image processing method according to the present invention includes: an image processing apparatus including means for acquiring image data from a photographing apparatus that captures a person subject; and means for extracting a person subject region from an image based on the acquired image data. In an image processing method for performing image processing for extending a range including a person subject area in a height direction of the person subject, the image processing apparatus determines whether or not a leg of the person subject in the image is bent. According to the determination, it is characterized in that the leg part region of the person subject is extended in different patterns depending on whether or not it is determined that the leg is bent.

  A computer program according to the present invention acquires image data from the imaging device, extracts a human subject region from an image based on the image data, and includes the human subject region of the image in a computer connected to an imaging device. In a computer program for executing image processing for extending a range in the height direction of a person subject, the computer determines whether the leg of the person subject in the image is bent, and the leg is bent. The step of extending the leg part region of the person subject in a different pattern is executed according to whether or not the state is determined to be in a different state.

  In the present invention, the posture of the leg is estimated and determined as the pose of the human subject in the image, and the leg region is expanded in a different pattern depending on whether or not the leg is bent. . That is, an appropriate extension process is performed according to the posture of the leg.

  In the present invention, when it is determined that the leg is not bent, the leg part region is stretched so as to be particularly stretched in the lower knee part. On the other hand, if it is determined that the leg is bent, the extension of the leg portion region in the height direction is distributed to a plurality of locations. Thereby, when it is presumed that the leg is bent, the specific part of the leg is not extremely extended.

  In the present invention, whether or not the leg is bent is determined based on the direction of the contour line (the direction with respect to the height direction) within a predetermined range in which it is estimated that the leg in the human subject area will be reflected. . Since the outline of the person subject is also used in other processes, it is possible to reduce the load due to the process only for the determination process.

  In the present invention, the direction is specified based on whether or not the total value of the components of the tangent direction with respect to the contour line at a plurality of points on the contour line is larger than a predetermined value. By calculating the total value and comparing it with the predetermined value, a judgment is made based on the tendency of the direction of the contour line within the predetermined range where it is estimated that the legs in the human subject area will be reflected. Is accurately estimated.

  In the present invention, when it is determined that the legs are bent, the portions other than the bent legs are extended at a plurality of locations in the height direction of the human subject. Thereby, the leg other than the leg bent sideways with respect to the height direction is extended, and the effect of appearing as a leg length can be further expected.

  According to the present invention, when there is a high possibility that the human subject in the image is not in an upright state, the extended portions of the legs are distributed and extended at a plurality of locations, so that the human subject is natural and ideal. It can be corrected so that it looks like a figure.

It is a perspective view which shows the external appearance of the imaging | photography game machine in this Embodiment. It is a block diagram which shows the structure of the imaging | photography game machine in this Embodiment. It is a flowchart which shows an example of the whole process performed with the imaging | photography game machine of this Embodiment. It is a flowchart which shows an example of the procedure of a style correction process. It is a flowchart which shows an example of the process sequence of estimation judgment of the state of a leg. It is explanatory drawing which shows an example of the image obtained by whole body imaging. It is explanatory drawing which shows the process of presumed judgment of the state of a leg. It is explanatory drawing which shows the process of presumed judgment of the state of a leg. It is explanatory drawing which shows the process of presumed judgment of the state of a leg. It is explanatory drawing which shows the process of presumed judgment of the state of a leg. It is explanatory drawing which shows the difference of the two expansion | extension processes in a style correction process. It is explanatory drawing which shows the result of the style correction process with respect to the image shown in FIG.

  Hereinafter, the present invention will be specifically described with reference to the drawings showing embodiments thereof.

  FIG. 1 is a perspective view showing an external appearance of a photographing amusement machine according to the present embodiment. A photographing amusement machine as a whole is a box-shaped device having a substantially rectangular parallelepiped shape that a plurality of users can enter. A shooting game machine can be roughly divided into a shooting area A and an editing area B, and a plurality of devices are provided in each of the areas A and B.

  In the shooting area A, devices related to the pre-service section 1 and the shooting section 2 are provided. The pre-service section 1 includes a coin slot, a touch panel display, a speaker, and the like that are provided toward the outside of the shooting area A. Accept the operation. The photographing unit 2 is configured to include a camera 21, a display 22, a lighting device group 23, and a speaker 24 (see FIG. 2) installed on the wall surface inside the photographing area A so as to photograph the user. A guide is displayed, and an image is taken in a plurality of times while being guided by voice, and processing for outputting the taken image to the editing unit is performed.

  In the editing area B, devices related to the first editing unit 3a, the second editing unit 3b (see FIG. 2), and the printing unit 4 are provided. The first editing unit 3a is configured to include a device in which a multi-touch panel display 30a for displaying an operation screen is exposed at a central portion of the editing area B at a height that is easy for a user to operate. The first editing unit 3a further includes a speaker 33a, a lighting device, and touch pens 31a and 32a (see FIG. 2) for the display 30a. The first editing unit 3a accepts a graffiti operation on an image based on a photographed image taken by the user using the display 30a and the touch pens 31a and 32a, and executes image processing corresponding to the accepted operation. The second editing unit 3b is configured in the same manner as the first editing unit 3a (not shown), and is provided back to back with the first editing unit 3a. The printing unit 4 includes a printer provided in a housing of an apparatus including the first editing unit 3a and the second editing unit 3b in the editing area B, and includes a photo sticker sheet on which an image after image processing is printed. It exhales from the outlet provided on the surface of the body.

  FIG. 2 is a block diagram illustrating a configuration of the shooting game machine according to the present embodiment. The shooting game machine includes the above-described pre-serving section 1, the shooting section 2, the first editing section 3a, the second editing section 3b, and the printing section 4 in a range that is visible to the user in the shooting area A and the editing area B. In addition, the communication unit 5 and the control mechanism are provided at a location hidden on the wall surface in the shooting area A, or inside the housing of the device in the editing area B.

  The communication unit 5 is connected to an external network such as the Internet and realizes transmission / reception of data including images taken. The communication unit 5 is connected to a control mechanism, and the control mechanism can communicate with a server device on an external network via the communication unit 5 to transmit and receive data. In the shooting game machine in the present embodiment, the communication unit 5 transmits the image data obtained by shooting to a server device managed by the manufacturer of the shooting game machine. Thus, the user can download his / her captured image from the server device by an information terminal device such as a so-called smartphone, tablet terminal, PC (Personal Computer), etc., and store it in a storage device in the information terminal device.

  The control mechanism includes a PC that controls each hardware described above, a control board including the bus 100 to which each hardware is connected, and a power supply unit (not shown) that supplies power to each hardware. The PC may be composed of a single PC, but in order to facilitate processing, the PC that controls the pre-serving section 1, the PC that controls the photographing section 2, the PC that controls the first editing section 3a, the second Different PCs may be used for each control target, such as a PC that controls the editing unit 3b and a PC that controls the printing unit 4 and the communication unit 5. In the following description, a composite image creation process that is a pre-stage of processing in the printing unit 4 will be mainly described. For this reason, the description will be made by virtually separating the first PC and the second PC. The first PC corresponds to the control unit 6 that controls the processes in the prior customer service unit 1 and the photographing unit 2. The second PC controls image processing including predetermined processing for the image obtained by the photographing unit 2, acceptance of editing by the first editing unit 3a and the second editing unit 3b, and synthesis based on these, and printing unit 4 corresponds to the image processing unit 7 instructing the print output in 4 and the data transmission from the communication unit 5.

  The control unit 6 includes a CPU (Central Processing Unit) 60, a memory 61, an HDD (Hard Disk Drive) 62, and a capture / chroma keyboard 63 provided in the first PC. The memory 61 may be a RAM such as a DRAM (Dynamic Random Access Memory) or an SDRAM (Synchronous DRAM), and may be a flash memory or another nonvolatile storage device such as an SSD (Solid State Drive) instead of the HDD 62.

  The CPU 60 reads the control program 6P stored in the HDD 62, and executes the control program 6P while reading / writing various information in the memory 61, thereby realizing the following functions.

  The CPU 60 inputs a control signal output from the pre-service unit 1 when receiving a predetermined charge at the coin insertion slot of the pre-service unit 1, causes the touch panel display to output an operation screen, and guides from the speaker. Output audio. Further, the CPU 60 inputs information corresponding to the operation on the operation screen displayed on the display of the pre-service unit 1 from the pre-service unit 1. Specifically, the CPU 60 inputs the user name, information related to the transmission destination address from the communication unit 5 and the like by an operation on the display.

  The CPU 60 controls the capture unit 2 to input the monitor video signal obtained from the camera 21 to the capture / chroma keyboard 63 and acquires the image data of each frame included in the video signal from the capture / chroma keyboard 63. Then, a video signal of an image for a live screen is created based on the acquired frame image data, and the created image signal is output to the display 22. Further, the CPU 60 outputs an image signal of a shooting guide for notifying the user of shooting start and guidance to the display 22 and outputs an audio signal to the speaker 24.

  The CPU 60 performs photographing by synchronizing photographing by the camera 21 of the photographing unit 2 and flash lighting by the lighting device group 23. The CPU 60 executes photographing with the camera 21 a plurality of times. The CPU 60 acquires a plurality of image data of an image obtained by photographing from the camera 21, assigns identification information so that each can be distinguished, stores it in the HDD 62, duplicates it, and outputs it to the image processing unit 7.

  The HDD 62 stores various information referred to by the CPU 60. The HDD 62 stores a control program 6P executed by the CPU 60 in advance. In addition to the control program 6P, the HDD 62 displays images and font data to be displayed on the display of the pre-service section 1 and the display 22 of the photographing section 2, and sounds and music output from the speaker of the pre-service section 1 and the speaker 24 of the photographing section 2. Such data is stored in advance. The HDD 62 stores image data obtained by taking a picture in the photographing unit 2.

  The capture / chroma keyboard 63 is a circuit board having a capture function and a chroma key function. The capture / chroma keyboard 63 has a capture function of acquiring image data of 30 (29.97) frames per second from a video signal and outputting the image data as a still image. The capture / chroma keyboard 63 extracts pixels determined to have a predetermined color tone and brightness within a certain range from a still image, and sets an α value (transparency) for the extracted pixels. It has a chroma key function to make the color transparent and output. The capture / chroma keyboard 63 sequentially captures frames from the video signal for monitoring from the camera 21, executes chroma key processing on the still image obtained by the capture as necessary, and writes it in the built-in memory. The CPU 60 sequentially reads still images from a memory built in the capture / chroma keyboard 63 and creates an image signal of an image for a live screen to be displayed on the display 22 of the photographing unit 2.

  The image processing unit 7 includes a CPU 70, a memory 71, an HDD 72, and a graphic board 73 provided in the second PC. The memory 71 uses a RAM such as DRAM or SDRAM.

  The CPU 70 reads the image processing program 7P stored in the HDD 72, and executes image processing based on the image processing program 7P while reading / writing various information from / to the memory 71. Further, the CPU 70 stores the image data output from the control unit 6 in the memory 71 or the HDD 72 together with the assigned identification information, and performs processing based on the image data. The processing includes processing to make the face of the person in the image a small face, brightening the color of the cheeks of the face (blushing), processing to gloss the hair, processing to brighten the hair color, large eyes Various processes are included, such as a process for making the body line appear thin, and a process for making the leg look longer. Details of the processing will be described later. In some of these processing processes, the strength (somewhat) of the processing condition may be executed in a plurality of stages. The CPU 70 stores in the memory 71 or the HDD 72 image data duplicated by the number of steps for each type of processing, executes processing on each image data in each step, and stores a plurality of processed image data in the memory 71 or Store in the HDD 72.

  Further, the CPU 70 performs a graffiti operation in the first editing unit 3a or the second editing unit 3b based on a plurality of image data before and after the above-described processing processing obtained by photographing stored in the memory 71 or the HDD 72. Accept. The CPU 70 creates image data such as a stamp including handwritten characters or makeup to be displayed on a preview display section of a graffiti operation screen, which will be described later, according to the content of the operation accepted as the graffiti operation. The CPU 70 outputs the created image data to the display of the first editing unit 3a or the second editing unit 3b. Further, the CPU 70 creates composite image data according to the confirmed editing content displayed in the preview display portion of the graffiti operation screen. Further, the CPU 70 executes processing for printing out an image on a photographic sticker sheet based on the generated composite image data, and processing for transmitting the composite image data or image data before the composite processing from the communication unit 5 to the external server device.

  The HDD 72 stores various information referred to by the CPU 70. The HDD 72 stores an image processing program 7P executed by the CPU 70 in advance. The HDD 72 stores in advance data such as guidance character images to be displayed on the displays of the first editing unit 3a and the second editing unit 3b, characters, font data or images that are composition materials, music and voices to be output from a speaker, and the like. I remember it. The HDD 72 stores data such as characters, font data, a background image, and a foreground image, which are composition materials, in advance. The HDD 72 stores image data obtained by image processing based on accepted editing and image data of the combined image.

  The graphic board 73 is hardware that draws an image based on input image data and a drawing command and outputs image data of the drawn image. The CPU 70 uses the graphic board 73 to perform image processing on the image data acquired and stored from the control unit 6. Specifically, the CPU 70 gives image data and a drawing command for obtaining a target image to the graphic board 73 to perform processing, and the image data of the image obtained by the drawing processing is stored in a memory built in the graphic board 73. Enter from. The drawing process includes enlargement / reduction, rotation, movement, chroma key function, trimming function, alpha function, brightness or color tone adjustment function, filter function, composition function, and the like.

  FIG. 3 is a flowchart showing an example of the overall processing executed by the shooting game machine of the present embodiment. The following process is started when the CPU 60 of the control unit 6 detects an operation on the touch panel display of the advance customer service unit 1 and a coin insertion.

  The CPU 60 accepts input of a user name, selection of a shooting course, selection of an output sheet design (background), selection of an angle, or the like on the touch panel display of the customer service section 1 (step S101). The CPU 60 confirms the availability of the shooting area A. If the shooting area A is empty, the CPU 60 displays a movement guidance screen for prompting movement on the display and outputs a guidance voice (step S102).

  Next, the CPU 60 causes the display 22 of the photographing unit 2 in the photographing area A to display a guidance screen for photographing contents according to the photographing mode or the sheet design selected in step S <b> 101, and guide voice and BGM from the speaker 24. Output (step S103). At this time, the guidance screen may include a live video based on the monitor output of the camera 21 of the photographing unit 2 and a pose sample for each photographing time.

  The CPU 60 controls the camera 21 and the illumination device group 23 of the control unit 2 according to the shooting mode or the sheet design selected in step S101, and executes shooting a plurality of times (step S104). The CPU 60 stores the image data obtained by each shooting in the HDD 62 so as to be mutually distinguishable (step S105). In a plurality of times of shooting in step S104, the shooting from the waist up and the whole body shooting including the toes are distinguished according to the shooting mode or the seat design selected in step S101. In each shooting, it may be controlled so that the shooting is executed with an appropriate tilt of the camera 21 (an angle of depression), a field angle, and brightness of lighting depending on whether the shooting is an up-shooting or a whole-body shooting.

  When shooting is completed, the CPU 60 confirms the availability of the editing area B, and if it is available, the guidance screen prompts the user to move the editing area B to either the first editing unit 3a or the second editing unit 3b. Is displayed on the display 22 and a guidance voice is output from the speaker 24 (step S106), and the process in the control unit 6 is terminated. In the following description, the case where the first editing unit 3a is urged to move is described as an example. However, since the same processing is performed when the second editing unit 3b is urged, detailed description thereof is omitted.

  Next, the process proceeds to the processing in the image processing unit 7. The CPU 70 of the image processing unit 7 acquires the image data stored in the HDD 62 by the CPU 60 of the control unit 6 (step S107). The CPU 70 executes predetermined image processing for each of the plurality of acquired data (step S108). Here, the predetermined image processing is, for example, the following processing. In step S108, the CPU 70 specifies a person subject from each of the obtained images based on the plurality of image data, creates a mask image of the specified person subject, and performs trimming so that the person subject fits appropriately. Further, the CPU 70 specifies the face area (contour) and the eye, nose, and mouth areas in the face area based on the face recognition technique for each of the one or more extracted human subject areas. The CPU 70 processes the face to be small on the basis of the specified area, brightens the cheek color of the face (blush), processes to gloss the hair, processes to lighten the hair color, Processing such as processing to make the body line thin, processing to make the leg look longer, etc. is executed. Of these processing processes, the details of the style correction process including the process of making the body line appear thin and the leg long will be described later.

  After the image processing in step S108, the CPU 70 displays the next graffiti operation screen on the display 30a of the first editing unit 3a, and outputs guidance voice and music from the speaker 33a (step S109). The CPU 70 receives a graffiti operation with the display 30a and the touch pen 31a on the graffiti operation screen (step S110).

  The CPU 70 synthesizes the sheet design image selected in step S101 and the character or image received in the graffiti operation in step S109 with the image after the image processing in step S108 based on the image data obtained by shooting. Is executed (step S111). The CPU 70 displays a screen including a preview of the photo sticker sheet after the composition process on the display 30a (step S112).

  The CPU 70 confirms whether or not the printer of the printing unit 4 is in use, and if it is not in use, causes the printer of the printing unit 4 to start printing out a photo sticker sheet (step S113). While the print output process is being performed, the CPU 70 performs operations such as selecting an image that can be obtained from the information terminal device among a plurality of captured images and inputting an information transmission destination address. It is received by the display 30a of 3a (step S114). The CPU 70 transmits the image data of the image selected according to the accepted operation to the external server device (step S115). The CPU 70 displays a game standby screen on the display 30a of the first editing unit 3a until the print output process is completed (step S116). When the print output process is completed, the CPU 30 displays the screen before the print unit 4 on the display 30a. A guidance screen for prompting movement is displayed and guidance voice is output (step S117), and the process is terminated.

  By performing the processing shown in the flowchart of FIG. 3, the user can enjoy the shooting itself in the shooting game machine and obtain the printed photo sticker sheet and the image data via the external server device. it can.

  Of the predetermined image processing executed in step S108 described above, the style correction processing will be described. FIG. 4 is a flowchart illustrating an example of the procedure of style correction processing. CPU70 acquires the image obtained by whole body imaging | photography (step S21). The angle of view and depression angle for each number of times of shooting and the brightness of lighting are set according to the settings of the sheet design, angle, etc. (S101 in FIG. 3) selected in advance, and the setting information is stored for each play. The Based on the setting information, the photographing unit 2 controls the depression angle and the angle of view of the camera 21 and the illumination device group 23. Then, the CPU 70 of the image processing unit 7 can specify and acquire image data obtained by whole-body imaging based on the setting information.

  The CPU 70 estimates and determines the state of the leg of the person subject based on the person subject area extracted for the acquired image (step S22). Details of the leg state estimation determination process in step S22 will be described later.

  The CPU 70 determines whether or not the estimated leg state is a state where the leg is bent (step S23). Here, the state where the leg is bent is, for example, a state where the tip of one leg is lifted backward or laterally, and the leg portion from the knee is directed in a direction not perpendicular to the floor. The feet of the lifting leg may be on the floor. Further, it may include a state where both legs are crossed and the leg portion is not perpendicular to the floor, or a state where one leg is extended to the camera 21 side. Good.

  If it is determined in step S23 that the leg is not bent (S23: NO), it is estimated that the human subject is captured without bending the leg, so the CPU 70 particularly extends the region corresponding to the lower knee part. The style correction process is executed (step S24). In the style correction processing in step S24, for example, the acquired rectangular image is extended in the vertical direction as a whole. The range that does not include the floor in the image determined in advance by the angle of view and the depression angle, and the area below the position stored in the HDD 72 as the height at which the knee portion is statistically reflected is higher than the other areas It is performed by extending in the vertical direction at a rate.

  If it is determined in step S23 that the leg is bent (S23: YES), the human subject is shown with a bent leg or something that is not a leg (bag, skirt, etc.) It is estimated to be. When a part of a bent leg or a leg that is not a leg is particularly elongated as in step S24 described above, it appears to be extremely distorted. Therefore, if it is determined in step S23 that the leg is bent (S23: YES), the CPU 70 executes a style correction process that extends the region including the leg portion at a plurality of locations and expands the entire region. (Step S25). The decompression process in step S25 is, for example, a range that does not include the floor surface in the image that has been previously determined by the angle of view and the depression angle, and is below the position stored in the HDD 72 as the height at which the inseam is statistically reflected. This is done by extending the region in the vertical direction at an equal rate.

  The leg state estimation determination process in step S22 will be described in detail. FIG. 5 is a flowchart illustrating an example of a processing procedure for determining and estimating the leg state.

  The CPU 70 acquires a mask image (an image in which pixels corresponding to the person subject area are α) for the person subject area extracted from the acquired image (step S301). The CPU 70 executes the following processing after reducing the image size (vertical / horizontal) of the acquired mask image to 1/10, for example.

  The CPU 70 extracts a rectangular area including the leg portion from the mask image (step S302). The rectangular area extracted in step S302 is a range that does not include the floor surface in the image determined in advance by the angle of view and the depression angle, and from the position stored in the HDD 72 as the height at which the inseam is statistically reflected. The lower area.

  The CPU 70 specifies the outline of the person subject area in the extracted rectangular area (step S303). The contour may be specified by using the filter processing of the graphic board 73. The CPU 70 specifies representative points on the contour at predetermined intervals in order in a predetermined direction in the image (for example, from the upper left to the lower and right directions of the image) (step S304). The predetermined interval is, for example, an interval of 15 pixels when the original image size is about 3000 pixels × 5000 pixels and the mask image is used after being reduced to 1/10. The predetermined interval may be determined from the image size and the reduction ratio, or may be determined according to the size of the body such as the face, head, and shoulder width in the image.

  The CPU 70 calculates a tangent vector for the contour at each representative point (step S305). The tangent vector may be normalized in length (for example, a length corresponding to 10 pixels). For each calculated tangent vector, the CPU 70 calculates the length of the horizontal component in the image (step S306), and calculates the total length of the horizontal components of all representative points (step S307).

  CPU 70 determines whether or not the calculated total value is equal to or greater than a predetermined value stored in HDD 72 (step S308). If it is determined that the total value is equal to or greater than the predetermined value (S308: YES), the CPU 70 determines that the leg is bent (step S309), and the process returns to step S23 in FIG.

  When it is determined in step S308 that the total value is less than the predetermined value (S308: NO), the CPU 70 determines that the leg is not bent (step S310), and the process proceeds to step S23 in FIG. return.

  If the number of representative points specified in step S304 is equal to or greater than the predetermined number, objects other than the legs, for example, clothes such as bags and skirts, or parts of other bodies other than the legs of other people in the region where the legs are to be captured Therefore, the process of step S305 may not be performed. In this case, it may be determined that the leg is bent, and the process of particularly extending the lower knee portion (S24) may not be performed.

In the example shown in the flowchart of FIG. 5, the contour is specified based on the mask image, and it is determined whether the portion estimated to be a leg is bent from the specified contour. However, it goes without saying that the process of estimating and determining the state of the leg is not limited to the procedure shown in the flowchart of FIG.
Furthermore, for example, using a camera that captures images from different directions, the state of the leg portion is estimated three-dimensionally based on images from a plurality of directions, or an infrared LED and a sensor that detects reflected light from the infrared LED. It may be used to capture the state of the leg portion in three dimensions.

  Next, the decompression process of steps S24 and S25 in the process procedure shown in the flowchart of FIG. 4 and the leg state estimation determination process shown in the flowchart of FIG. 5 will be described with specific examples. FIG. 6 is an explanatory diagram illustrating an example of an image obtained by whole body imaging. The image shown in FIG. 6 is an image before performing the predetermined image processing in step S108 in FIG.

  In the example shown in FIG. 6, the person shown on the right side in the figure has a posture in which the body is directed to the right side with respect to the camera 21 and the left foot is flipped up to the rear side. Thereby, in FIG. 6, the leg region in the human subject region is shown sideways (outward). In order to extend the legs of two people, including the person shown on the right side, for example, when processing is performed to extend the area below the floor and below the knee, the legs are sideways. Therefore, there is a possibility that the calf part is processed to be extremely thick.

  7-10 is explanatory drawing which shows the process of the estimation judgment of the state of a leg. FIG. 7 shows a rectangular area extracted in step S302 from the mask image (person subject area) created for the image shown in FIG. A white portion in FIG. 7 is a region having a relatively high α value and corresponds to a human subject region. Of the two alternate long and short dash lines in FIG. 7, the alternate long and short dash line corresponds to the boundary on the background side of the floor surface in the image. The position of the boundary on the background side of the floor in the image is stored in advance in the HDD 72. Also, the alternate long and short dash line at the top corresponds to the height at which the inseam is statistically reflected. The height at which the crotch is shown in the image is also stored in the HDD 72 in advance.

  FIG. 8 shows representative points identified on the contour in the rectangular area extracted in step S302, FIG. 9 shows tangent vectors calculated on the representative points by bold lines, and FIG. 10 shows one of the tangent vectors. It is an enlarged view. As shown in FIG. 8, when representative points are specified at predetermined intervals in the process of step S304 by the CPU 70 of the image processing unit 7, the tangent vector on each representative point is contoured from the upper left in the image as shown in FIG. It is calculated by tracing. As shown in the enlarged view of FIG. 10, the tangent vector with respect to the contour is decomposed into a vertical component and a horizontal component in the image, and the length of each component when the magnitude of the tangent vector is set to 10, for example (scalar). Value) is calculated. In the case of the image of FIG. 6, the total value of the lengths of the horizontal components of the tangent vectors on the representative points shown in FIG. 9 is determined to be greater than or equal to a predetermined value because the legs are shown in the horizontal direction.

  In this way, the total length of the horizontal components of the contour tangent vector is calculated from a two-dimensional captured image without depth information, and compared with a predetermined value to determine whether the leg is bent or not. To do. The knowledge that the possibility of estimating and judging the state where the legs are bent sideways as a whole is increased by taking the total length of the horizontal components in the entire rectangular area extracted at each representative point. It is.

  FIG. 11 is an explanatory diagram showing a difference between two decompression processes in the style correction process. The upper rectangle in FIG. 11 shows a rectangular image before the style correction process, and further mesh lines that divide the rectangular image into 8 and 10 are shown. A in the lower two rectangles corresponds to the processing in step S24 in the flowchart of FIG. 4 and shows a deformation mode of the mesh when the lower knee range is extended longer than the other regions. B in the two rectangles corresponds to the process of step S25 in the flowchart of FIG. 4 and shows a deformation mode of the mesh when the leg portions are extended so as to be dispersed at a plurality of locations as a whole.

  As shown in FIG. 11, in the mesh deformation (A) corresponding to step S <b> 24, the region where the lower knee part of the leg is reflected is extended in the longitudinal direction extremely longer than the other regions. Specifically, if the leg portion region is extended 20% in the vertical direction, in the deformation (A) of FIG. 11, the portion that is particularly extended (the lower knee portion indicated by the arrow) is extended approximately 20%. As a result, an effect that looks like a leg length can be expected. In the area around the waist, the body is contracted in the vertical direction more than the other areas so that the body appears shorter. Thereby, the effect which looks like a leg length similarly can be anticipated. In order to correct the style, the entire width of the upper portion of the image is deformed to be smaller than the width of the lower portion. That is, the upper part of the person subject area is deformed so as to be tilted to the back side in the depth direction of the image. As a result, the human subject area is stretched in the vertical direction as a whole, and correction to a style in which the limbs are long and smooth is realized. Furthermore, in the deformation of the mesh corresponding to step S24, the foot (shoes) appears larger when shooting in the shooting space, so the area corresponding to the floor surface is deformed so as to contract in the vertical direction.

  And in the deformation | transformation (B) of the mesh corresponding to step S25, as FIG. 11 shows, compared with the deformation | transformation of the mesh corresponding to step S24, there is no area | region extended extremely longer than the other area | region. . Specifically, in the above-described mesh deformation (A), the lower knee portion was stretched by 20%, whereas in the deformation (B), the lower knee portion was 5% and each mesh portion corresponding to the remaining leg region was 2%. Extend by%. Even in the deformation corresponding to step S25, in the region around the waist, the body is contracted in the vertical direction more than the other region so that the trunk is reflected in a shorter direction, but the extent of the region extension in the leg portion is distributed evenly as a whole. ing. As a result, it is possible to expect the effect of appearing as a leg length while avoiding conspicuous conversely due to a part of the region being extremely stretched. Similarly, in the deformation corresponding to step S25, the entire width of the upper portion of the image is deformed to be smaller than the width of the lower portion. That is, the upper part of the person subject area is deformed so as to be tilted to the back side in the depth direction of the image. The human subject area is stretched in the vertical direction as a whole, and correction to a style in which the limbs are long and slender is realized.

  Note that in the deformation (B) of the mesh corresponding to step S25 in FIG. 11, the extent of the extension of the region in the leg portion is generally uniform. However, the present invention is not limited to this, and a range including a representative point where a tangent vector having a relatively long horizontal component length is calculated may be configured not to be expanded. In this case, you may make it provide a difference in the extent of expansion | extension so that it may extend in another range. Furthermore, depending on the length of the component in the horizontal direction, the leg part is selected by extending the leg part in multiple patterns, including an intermediate stage from the pattern in which the degree of extension is uniform throughout the pattern to emphasize the lower part of the knee. It is good also as a possible structure.

  FIG. 12 is an explanatory diagram showing the result of the style correction processing for the image shown in FIG. Note that FIG. 12 shows only the leg portion in the image of FIG. 6 in an enlarged manner. The upper part of FIG. 12 shows the result of the style correction when the mesh deformation of A of FIG. 11 is applied (S24), and the lower part is the style of the case of applying the mesh deformation of B of FIG. 11 (S25). The result of correction is shown. When the upper part and the lower part of FIG. 12 are compared, in the result of the style correction of the upper part, the left leg of the person shown on the right side is turned sideways, so that the calf part is particularly processed to be extremely thick. As a result of the lower style correction, it is clear that specific portions of the leg region are not processed to be extremely thick or extremely long and are elongated in the vertical direction as a whole.

  The above-described decompression process based on the mesh is performed on an image obtained by whole body photography. It is preferable to set whether or not to perform the expansion process based on the mesh according to the angle of view or the depression angle of the camera 21 or to adjust the deformation mode of the mesh according to the angle of view or the depression angle even when the expansion process is performed. As a result, it is possible to obtain a photographing result that naturally and in a good style.

  It should be understood that the embodiment disclosed above is illustrative in all respects and not restrictive. The scope of the present invention is defined by the terms of the claims, rather than the meanings described above, and is intended to include any modifications within the scope and meaning equivalent to the terms of the claims.

7 Image processing unit 70 CPU
71 Memory 72 HDD
73 graphics board 7P image processing program

Claims (7)

An acquisition unit that acquires image data from a photographing device that captures a human subject; and an extraction unit that extracts a human subject region from an image based on the acquired image data, and the range including the human subject region of the image is included in the person In an image processing apparatus that performs image processing for extending in the height direction of a subject,
Determining means for determining whether or not a leg of a human subject in the image is bent;
An image processing apparatus comprising: extension means for extending a leg portion area of the person subject in a different pattern depending on whether or not the determination means determines that the leg is bent. . The extension means includes
A first extension means for extending an area corresponding to a lower knee portion of the person subject area longer than other areas when the determination means determines that the leg is not bent;
2. The apparatus according to claim 1, further comprising: a second extending unit that extends the region of the leg portion of the human subject in a distributed manner when the leg is determined to be bent by the determining unit. The image processing apparatus according to 1. The determination means includes
Means for identifying an outline of the person subject area;
Direction specifying means for specifying a direction with respect to the height direction in an image of a contour line in a predetermined range including a leg portion of the specified contour, and
The image processing apparatus according to claim 1, wherein when the direction specified by the specifying unit is determined to be horizontal, it is determined that the leg is bent. The specifying means is:
Means for identifying a plurality of points at predetermined intervals on the contour line;
Means for calculating the sum of the horizontal sizes in the image of tangents of the contour lines at the plurality of points;
Means for determining whether the calculated total is larger than a predetermined value,
The image processing apparatus according to claim 3, wherein when it is determined that the means is larger, it is determined to be in a landscape orientation. The image processing apparatus according to claim 2, wherein the second elongating unit disperses the elongation at a place other than the bent leg portion. An image processing apparatus comprising means for acquiring image data from a photographing device that captures a person subject, and means for extracting a person subject region from an image based on the acquired image data. In an image processing method for performing image processing for extending in the height direction of a subject,
The image processing apparatus includes:
Determining whether the leg of the human subject in the image is bent,
An image processing method, comprising: extending a region of a leg portion of the person subject with a different pattern depending on whether or not the leg is determined to be bent. A computer connected to the image capturing device acquires image data from the image capturing device, extracts a human subject region from an image based on the image data, and sets a range including the human subject region in the image in the height direction of the human subject. In a computer program for executing image processing for decompression,
In the computer,
The step of determining whether or not the leg of the human subject in the image is bent, and whether or not the leg portion area of the human subject is determined according to whether or not the leg is determined to be bent A computer program for executing a step of expanding with different patterns.