JP2010061646A - Image processing apparatus, image output device, image processing method, and computer program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an image processing apparatus for effectively performing processing for allowing the faces of persons to appear smaller in size on a photographed image, even when a plurality of persons pose with motions, and to provide an image output device, an image processing method and a computer program. <P>SOLUTION: When the face to be detected from the image is processed, it is determined whether a processing contour line is crossed with another processing contour line, concerning the respective detected faces (S73). When the processing contour line is crossed with another processing contour line (S73: YES), the face is removed from processing objects (S75). Thus, distortions will not be generated by the processings in the neighborhood of the contour. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to an image processing apparatus that processes a human face in an image. In particular, even when a plurality of persons take a moving pose, an image processing apparatus capable of effectively performing an image processing process that makes each person's face appear as a small face on an image, and an image output apparatus using the image processing apparatus The present invention relates to an image processing method and a computer program.

  Due to advances in image processing technology, image processing for applying various processing to an image obtained by photographing with a camera has been immediately performed.

  Patent Document 1 relates to an automatic photography apparatus in which a photographic image obtained by photographing a user himself / herself can be easily printed out and a user can obtain a photographic image, and a part of the image is enlarged or reduced with respect to a captured image. The technique of the automatic photography apparatus which can be deformed and can obtain an interesting print output is disclosed.

  Further, in Patent Document 2, when outputting a photographic image such as an ID photo, a face region is extracted from a captured image obtained by photographing a person, a face feature point is further detected, and the detected face feature point position is used. Thus, there has been disclosed a technique of an image correction apparatus that can obtain a good-looking image by processing the contour of the face so that the face is slender, without relying on conventional photography techniques or know-how in a photo studio. .

JP 2000-267156 A JP 2004-318204 A

  With the technique disclosed in Patent Document 2, it is possible to extract the outline of a face from an image obtained by photographing a person and process the outline so that the face is slender. However, an image to be processed that is assumed in the technology disclosed in Patent Document 2 is an image such as a portrait photograph or an ID photograph that captures a person with the front facing the camera.

  Except for a specific case such as taking a portrait photograph or an identification photograph, the person who is the subject is not always photographed in an upright posture so as to be photographed parallel to the vertical direction of the photographing range of the camera. In particular, the user himself takes a photograph with a camera, accepts editing of the image to be combined with the photographed image, combines the edited image with the photographed image, prints the image on a predetermined sheet, and provides the image as a photo sticker In an output device, so-called “Purikura” (registered trademark), the subject is not limited to one person, and each person is often photographed in various poses.

  When a plurality of persons are photographed in an overlapping manner, if the configuration is such that the contour of each person is processed, there is a possibility that problems such as distortion of the contour may occur. For example, in the technique disclosed in Patent Document 2, the face area is reduced symmetrically so as to narrow the width from both sides of the face. When the face area of one person is reduced in spite of overlapping persons, there is a possibility that distortion occurs, for example, image information in the face area of the other person near the reduced outline is missing.

  In addition, there are cases where each person is photographed with his / her face tilted or with his / her face tilted. If the face area recognized in this case is processed to be reduced symmetrically on the premise that it is facing the front, there is a possibility that distortion occurs in the face area.

  In this way, when there is a possibility that there are a plurality of persons to be imaged and it is assumed that a pose with movement is assumed, in the configuration in which processing is similarly performed on the face area of any person, It is difficult to improve the appearance effectively because there is a possibility that problems such as distortion of the person's face may occur due to unnecessary processing on the parts that should not be performed.

  The present invention has been made in view of such circumstances, and even when a person takes a moving pose, image processing that can effectively perform a processing process that makes a face of a person in a photographed image appear as a small face. An object is to provide an apparatus, an image output apparatus, an image processing method, and a computer program.

  It is another object of the present invention to provide an image processing apparatus that can prevent unnatural contour distortion due to image processing and can effectively perform processing that makes a human face look like a small face. is there.

  The object of the present invention is to identify a position as a reference from the positions where the eyes, nose or mouth of the face are captured, and to perform image processing based on the reference, so that even when a person takes a moving pose, An object of the present invention is to provide an image processing apparatus capable of effectively performing a processing process of making a face look like a small face.

  It is an object of the present invention to provide an image processing apparatus that can be processed according to a person's face classification and that can effectively perform a processing process that makes each face look like a small face even when the person takes a moving pose. It is to provide.

  In addition, an object of the present invention is to prevent unnecessary processing on a portion that should not be processed, and even when a person takes a moving pose, an image that can effectively perform processing that makes each face look like a small face. It is to provide a processing apparatus.

  Further, an object of the present invention is to effectively perform a processing process shown on a small face so that it does not become unnatural depending on how each person is captured and the characteristics of the face even when the person takes a moving pose. An object of the present invention is to provide an image processing apparatus that can perform such processing.

  An image processing apparatus according to a first aspect of the present invention is an image processing apparatus comprising a detecting means for detecting a human face in an image and a processing means for processing the detected face, based on the face contour detected by the detecting means. Processing boundary setting means for setting a processing boundary line for specifying a boundary of the processing processing range, means for extracting a predetermined range including a face outline from the image as a face area, and processing boundary line set by the processing boundary setting means Determining means for determining whether the distance in the image between the processing boundary line of another face is equal to or less than a predetermined value, and the face of the face corresponding to the processing boundary line whose distance is determined to be equal to or less than the predetermined value And a means for excluding the region from the processing target.

  In the image processing apparatus according to the second invention, the processing boundary setting means includes means for extracting a face outline detected by the detecting means, and sets the outside of the extracted outline as a processing boundary. It is characterized by being.

  An image processing apparatus according to a third aspect of the present invention comprises means for specifying each area of the face eye, nose or mouth detected by the detecting means, and means for specifying the face center based on the position of each specified area. The processing means is characterized in that the image in the area within the processing boundary line is contracted at a predetermined ratio both vertically and horizontally toward the face center specified in the face area.

  The image processing apparatus according to the fourth aspect of the invention is the first straight line connecting the eye regions, the center of the nose region or the mouth region in the vertical direction with respect to the face. Means for specifying the second straight line passing through, means for specifying the length from the intersection of the first straight line and the second straight line to the two intersections where the first straight line intersects the contour of the face, and the first straight line and the second straight line. Means for specifying the length from the intersection of the straight line to the intersection where the second straight line intersects the contour of the face at the lower part of the face, and the type of the face of the person shown in the captured image based on the specified length And a means for determining a coefficient for the predetermined ratio for adjusting the vertical and horizontal shrinkage ratios according to the classified type.

  The image processing apparatus according to a fifth aspect is characterized in that the processing means contracts an image of a region within the face region and further within the processing boundary line except for both eye regions.

  According to a sixth aspect of the present invention, there is provided an image processing apparatus according to a comparison result obtained by a means for obtaining a size of a specified eye region with respect to a face region, a means for comparing the obtained size with a predetermined reference size. Means for calculating an enlargement ratio of the size of the eyes, and the processing means enlarges the image of the specified region of both eyes based on the calculated enlargement ratio.

  An image output apparatus according to a seventh aspect of the present invention is an image output device comprising: an imaging unit; a combining unit that combines a captured image obtained from the imaging unit with another image; and a unit that outputs the combined image to a predetermined medium. The apparatus further includes image processing means for detecting a human face in the captured image and processing the detected face, and the image processing means specifies a boundary of the processing range based on the detected face contour. Processing boundary setting means for setting a boundary line; means for extracting a predetermined range including a face contour from the image as a face area; and processing boundary lines set by the processing boundary setting means for processing boundary lines of other faces Means for judging whether or not the distance in the image between is less than or equal to a predetermined value, and means for excluding the face area of the face corresponding to the processing boundary line determined to be less than or equal to the predetermined value from the processing target Specially equipped with To.

  According to an eighth aspect of the present invention, there is provided an image processing method for detecting a human face in an image and processing the detected face, wherein a processing boundary line for identifying a boundary of a processing range based on the detected face contour A predetermined range including the face outline is extracted from the image as a face region, and the distance between the set processing boundary line and the processing boundary line of another face in the image is equal to or less than a predetermined value And the facial region of the face corresponding to the processing boundary line for which the distance is determined to be equal to or less than a predetermined value is excluded from the processing target.

  A computer program according to a ninth aspect of the invention is a computer program for causing a computer to detect a face of a person shown in an image and to process the detected face, and to specify a boundary of a processing range based on the detected face contour A step of setting a boundary line, a step of extracting a predetermined range including the outline of the face from the image as a face area, and a distance between the set processing boundary line and the processing boundary line of another face in the image is a predetermined value It is characterized in that a step of determining whether or not it is below and a step of excluding the face area of the face corresponding to the processing boundary line determined to be below a predetermined value from the processing target are executed.

  In the present invention, a human face is detected from an image, and a processing boundary line that specifies the boundary of the processing range is set based on the detected face contour. Further, a predetermined range including the face outline is extracted as a face area. For each processing boundary set for each extracted person's face, determine whether the distance between the processing boundary lines in the image is equal to or less than a predetermined value and the processing boundary lines are close to each other Is done. When another processing boundary line is close to one processing boundary line, the face area of the face corresponding to each processing boundary line is excluded from the processing target, and the processing process is not executed.

  In the present invention, when the processing boundary line of each face is set, the detected face contour line is extracted, the outside of the face is set as the processing boundary line from the contour line, and the image within the processing boundary line is set. Processing is performed. If an image in the contour is processed using the extracted facial contour as a boundary, there is a possibility that the contour is distorted near the boundary between the region where the processing is performed and the region where the processing is not performed. On the other hand, since the outer side of the contour is used as the processing boundary line, distortion in the face contour due to the processing is prevented.

  In the present invention, a region where the eyes, nose or mouth of the face is reflected is specified, and the face center is specified based on the position of the region of the eyes, nose or mouth. Processing is performed in which the area within the processing boundary line is shrunk at a predetermined ratio in the vertical and horizontal directions toward the face center. Accordingly, when the face is tilted, it can be processed so that a natural image is obtained even when the face is not facing the front.

  In the present invention, a first straight line connecting the regions of both eyes and a second straight line passing through the center of the nose or mouth region are specified. Face types are classified based on the width from the intersection of the first straight line and the second straight line to the lateral ends of the face, and the length from the intersection to the portion corresponding to the tip of the chin. A coefficient for adjusting the shrinkage rate is determined. This enables effective processing according to face type classification, such as a relatively large vertical shrinkage for face-length faces and a relatively large horizontal shrinkage for wide faces. Is done.

  In the present invention, both eye regions are excluded and contracted. By correcting the face area other than both eye areas to be shrunk, both eyes are processed to be relatively large.

  In the present invention, the image of the area of both eyes is such that both eyes are relatively large due to the enlargement ratio obtained in accordance with the comparison determination result between the size relative to the size of the face region and a predetermined reference size. To be processed.

  According to the present invention, when the face of a person is close to the face of another person in the image, the face area is excluded from the processing target and is not processed. Therefore, even if the person takes a pose with movement, for example, an image in which a plurality of people are in close contact, or an image in which images are overlapped, useless processing that results in an unnatural image is not performed. Processing can be performed effectively without causing distortion such as missing face image information.

  According to the present invention, since the distortion of the face outline is prevented, it is possible to prevent an unnatural distortion from occurring due to the process of making a small face, and a pose with movement such as a person appearing in an overlapping manner. Even when it is assumed that the image is taken, it is possible to effectively perform the process of making the face of a person look like a small face.

  According to the present invention, since the center of the region extracted as the face region is not the center of the face, the portion corresponding to the eyebrows is specified as the center of the face, and the processing is performed by contracting toward the center of the face. Even if the image has a moving pose, such as when the image is tilted or the image is taken without facing the front, it is possible to effectively perform the process of making a person's face appear as a small face.

  According to the present invention, the face type is classified on the basis of the face center, and processing according to the face type classification is performed, so even if the image is a pose with a person moving, the face can be seen as a small face. Processing can be performed effectively.

  According to the present invention, even in an image in which a person poses a movement, it is possible to prevent unnecessary processing from being performed on both eyes, and to effectively perform processing that makes each person's face appear as a small face. it can.

  According to the present invention, even in an image in which a person poses a movement, the enlargement ratio according to the characteristics of the size of the area of both eyes with respect to the size of the face area depends on the size of the face area of each person. As a result, the image of the area of both eyes is enlarged, and the process of making the face look small and close to the ideal face can be performed effectively. Since the enlargement ratio is adjusted according to the size of the face, the enlargement of the area of both eyes can be appropriately adjusted between the person in the back and the person in the foreground. Furthermore, since the enlargement ratio is adjusted according to the result of comparison with the reference size, processing that enlarges the area of both eyes according to the face with different characteristics and makes it appear as a small face rather than enlarging at a uniform ratio Can be performed effectively.

It is a perspective view which shows the external appearance of the image output device in this Embodiment. It is a perspective view which shows the external appearance of the image output device in this Embodiment. It is a block diagram which shows typically the structure of the image output device which concerns on this Embodiment. It is a block diagram which shows typically the structure of the image output device which concerns on this Embodiment. 4 is a flowchart illustrating a processing procedure in which the CPU according to the first embodiment executes photography, editing, and printing. 3 is a flowchart illustrating an example of an entire image processing procedure executed by the CPU of the image processing unit according to the first embodiment. 3 is a flowchart illustrating an example of analysis processing executed by a CPU of an image processing unit according to Embodiment 1. 5 is a flowchart illustrating an example of an overlapping contour removal process executed by a CPU of the image processing unit according to the first embodiment. 6 is a flowchart illustrating an example of small face processing executed by the CPU of the image processing unit in the first embodiment. 3 is a flowchart illustrating an example of a processed image creation process executed by the CPU of the image processing unit according to the first embodiment. It is explanatory drawing which shows the example of the content of the image which CPU acquired from the control part. It is explanatory drawing which shows the example of the content of the information obtained by performing face detection processing by CPU. It is explanatory drawing which shows the example of the face center specified by the analysis process by CPU. It is explanatory drawing which shows the example of the outline process line and work area which are set by the analysis process by CPU. It is explanatory drawing which shows the other example of the outline process line and work area which are set by the analysis process by CPU. It is explanatory drawing which shows the outline | summary of the small face process by CPU. 10 is a flowchart illustrating an example of analysis processing executed by the CPU of the image processing unit according to the second embodiment. 12 is a flowchart illustrating an example of face classification determination processing executed by the CPU of the image processing unit according to the second embodiment. It is explanatory drawing which shows the example of the outline and centerline which are identified with respect to the face detected in an image. 10 is a flowchart illustrating an example of small face processing executed by the CPU of the image processing unit according to the second embodiment. 10 is a flowchart illustrating an example of analysis processing executed by a CPU of an image processing unit according to Embodiment 3. 15 is a flowchart illustrating an example of an eye enlargement ratio calculation process executed by the CPU of the image processing unit according to the third embodiment. It is explanatory drawing which shows the example of the magnitude | size of the eye calculated from the face area | region detected in an image. 14 is a flowchart illustrating an example of small face processing executed by a CPU of an image processing unit according to Embodiment 3.

  Hereinafter, the present invention will be specifically described with reference to the drawings showing embodiments thereof.

  In the following description, an example in which the image processing apparatus according to the present invention is applied to an image output apparatus that takes a photograph of a user and outputs a photographic sticker based on the captured image will be described. In other words, in the embodiment described below, the image processing apparatus according to the present invention is included in the image output apparatus, and corrects an image taken to output a photo sticker.

(Embodiment 1)
1 and 2 are perspective views showing an appearance of the image output apparatus according to the present embodiment. The image output apparatus is a box-type apparatus having a substantially rectangular parallelepiped shape as a whole, and is provided with a partition so as to be divided in substantially half in the vertical direction. The image output apparatus is divided into an imaging space R1 and an editing space R2 by this partition. FIG. 1 shows an external perspective view mainly viewed from the photographing space R1 side of the image output apparatus, and FIG. 2 shows an external perspective view of the image output apparatus mainly viewed from the editing space R2 side.

  As shown in the perspective view of FIG. 1, the imaging space R1 is a space surrounded by a frame that forms a substantially rectangular parallelepiped. The shooting space R1 is a space for accepting the input of a fee from the user and for taking a picture of the user. The photographing space R1 is provided with a receiving unit 1 that accepts a predetermined fee on the outside, and devices such as a camera 21 and a display 22 for photographing a user on an inner wall surface corresponding to a partition with the editing space R2. An imaging unit 2 including is provided.

  The reception unit 1 includes a coin insertion slot 10 that accepts coin insertion, a coin selector 11 that determines the authenticity of coins inserted from the coin insertion slot 10, and various types of information such as the total amount of coins inserted or the amount remaining. And a display panel 12 for displaying. The coin selector 11 determines the authenticity of the coin inserted from the coin insertion slot 10 and sends a determination signal. The coin selector 11 discharges a coin to a coin return port (not shown) when the coin is determined to be false or while it is set not to accept a coin. The display panel 12 is composed of a liquid crystal panel, a segment LED, etc., and receives a control signal to prompt the insertion of coins, a charge to be inserted, the total amount to be inserted, the remaining amount, or another user is shooting and accepts the insertion of a coin A message indicating that it has not been displayed.

  The display panel 12 and the coin selector 11 included in the receiving unit 1 are connected to a control mechanism housed in a first housing described later, and a determination signal, a control signal, and the like are transmitted to the control mechanism. Send and receive.

  The user can receive a service by the image output device by inserting coins corresponding to a predetermined fee from the coin insertion slot 10 of the reception unit 1.

  In the imaging space R1, that is, inside the imaging booth, each device constituting the imaging unit 2 is installed. A first housing 20 containing a control mechanism for controlling each device is installed at the lower part of the wall surface that partitions the space with the editing space R2 inside the photographing booth. A camera 21 is installed above the first housing 20 and at a height that is approximately the height of the standing user. A touch panel display 22 that displays a preview screen based on a video signal output from the camera 21 and accepts a user operation is set below the camera 21. Further, a strobe lamp 23 for irradiating the subject with light is provided so as to surround the camera 21 and the display 22. Speakers 24 and 24 for outputting music and guidance voice are installed on the upper portion of the first housing 20.

  The camera 21, the display 22, the strobe lamp 23, and the speakers 24 and 24 constituting the photographing unit 2 are respectively connected to a control mechanism installed in the first housing 20.

The camera 21 is a digital camera that includes an imaging element such as a lens group, a diaphragm, a shutter, and a CCD (Charge Coupled Device) in order to photograph a user as a subject. The camera 21 takes a picture by acquiring and storing a still image from the imaging device by operating a shutter by a timer function. While the camera 21 is activated, the camera 21 outputs a digitized video signal output from the image sensor to the control mechanism. The control mechanism outputs a captured image for displaying a preview screen based on the video signal.

  The display 22 has a liquid crystal panel for displaying an operation screen for guiding operations during shooting and a preview screen, and accepts an image output from the control mechanism based on a video signal output from the camera 21 in real time. To display. Further, the display 22 is a touch panel type, and a signal including information on a position on the screen at a location touched by the user is output to the control mechanism.

  The strobe lamp 23 is a lighting device in the photographing booth and functions as a flash when taking a picture. In the present embodiment, the strobe lamp 23 is divided into seven as shown in the figure, and the timing of each light emission is taken by the camera 21 via the control base included in the control mechanism (the operation of the shutter). ) Is controlled in synchronization with

  The outer periphery of the photography booth is covered with a light-shielding sheet material (not shown). This sheet material blocks external light, prevents peeping from the outside, and realizes a private space where the user can freely pose in the shooting booth.

  In addition, the sheet material viewed from the inside of the photographing booth has uniform color tone and brightness. When viewed from the camera 21, the rear side of the user has the same color tone and lightness of the sheet material, so that the chroma key process for making the area other than the area in which the user is visible transparent becomes possible, as will be described later.

  The user enters the shooting booth, operates the operation screen according to the guidance displayed on the display 22 and the guidance voice output from the speakers 24, 24, determines the pose by receiving illumination from the strobe lamp 23, and uses the camera 21. A photo shoot can be performed. First, the user can select a foreground image and / or a background image that decorates a photograph taken by the user from an operation screen displayed on the touch panel display 22. When the user selects the foreground image and / or the background image, the display 22 displays an image output in real time from the camera 21 via the control mechanism, that is, a captured image output from the camera 21, and the selected foreground. A print result preview screen based on the image and / or the background image is displayed. The user can determine a pose while confirming the preview screen, and execute a photo shoot.

  When a photograph is taken, the control mechanism acquires a still image stored by the camera 21 and copies it as an image for later accepting an editing operation by the user and an image for later printing out.

  Next, each component installed in the editing space R2 shown in the perspective view of FIG. 2 will be described. The editing space R2 is a space behind the wall where the camera 21, display 22, strobe / lamp 23, speakers 24 and 24, and the first housing 20 are installed, and the user can take photographs. This is a space where an image can be edited after shooting. The editing space R2 shown in FIG. 2 is partitioned from other spaces by a curtain-like sheet material (not shown) attached to a bar extending from the upper surface of the photographing space R1. The editing space R <b> 2 is provided with a substantially rectangular parallelepiped second housing 30 having the following components inside and outside.

  One side surface of the second housing 30 is in contact with a wall surface that is a partition with the imaging space R1, and a photographed image and other images are edited on the opposing surfaces of the other three side surfaces, respectively. A first editing unit 4a and a second editing unit 4b having an interface for the above are provided. On the other side, a printing unit 5 is provided for printing out the edited image on a predetermined photo sticker sheet. In addition, a control mechanism that controls each component of the first editing unit 4 a, the second editing unit 4 b, and the printing unit 5 is installed in the second housing 30.

  Note that the surfaces of the first casing 20 and the second casing 30 that are in contact with each other communicate with each other through an opening (not shown) of each, and are installed inside the first casing 20. The control mechanism of the photographing unit 2 and the control mechanism of the first editing unit 4a, the second editing unit 4b, and the printing unit 5 installed in the second housing 30 are connected by a plurality of cables. Has been.

  The first editing unit 4a includes a tablet display 40a that displays an operation screen for editing a photographed image and other images, two touch pens 41a and 42a, and a speaker 43a. Note that the second editing unit 4b described later also includes a tablet display 40b, two touch pens 41b and 42b, and a speaker 43b. The tablet displays 40a and 40b, the two touch pens 41a, 42a, 41b, and 42b, and the speakers 43a and 43b of the first editing unit 4a and the second editing unit 4b, respectively, face the opposing surfaces of the second housing 30. Is installed. The tablet displays 40 a and 40 b, the touch pens 41 a, 42 a, 41 b and 42 b and the speakers 43 a and 43 b are connected to a control mechanism inside the second housing 30.

  The printing unit 5 displays a printer 50 that is installed inside the second housing 30 and prints an image on a predetermined sheet, a seal discharge port 51 through which the printed sheet is discharged from the printer, and various types of information. Display panel 52. The display panel 52 is connected to a control mechanism installed inside the second housing 30, and the printer 50 is also connected to the control mechanism inside the second housing 30.

  The user can perform operations such as selecting an image to be printed by the first editing unit 4a or the second editing unit 4b and graffiti on the selected image after completing the shooting by the shooting unit 2 in the shooting booth. On the tablet displays 40a and 40b of the first editing unit 4a and the second editing unit 4b, both the image acquired by executing the photography and the selected foreground image and / or background image are displayed. The The tablet displays 40a and 40b are configured to accept operations by the touch pens 41a, 42a, 41b, and 42b installed together, and the user selects an image to be printed by the touch pens 41a, 42a, 41b, and 42b, and selects the selected image. Operations such as insertion of characters and stamp-like patterns.

  The control mechanism performs a correction process by an image processing apparatus described later on the photographed image. When the editing by the user is completed, the selected foreground image and / or background image, and characters, designs, and the like based on the editing result are combined with the print output image after the correction process. The combined image is printed on a predetermined sheet by the printer 50 installed in the second housing 30. The printed predetermined paper is discharged to the seal discharge port 51 as a photo sticker. The user can obtain the completed photo sticker from the sticker discharge port 51, thereby realizing a photo sticker photographing providing service.

  3 and 4 are block diagrams schematically showing the configuration of the image output apparatus according to the present embodiment. 3 mainly shows configurations of the control unit 6, the image processing unit 7, the power supply unit 8, the communication unit 9, and the receiving unit 1 described below. FIG. 4 shows the photographing unit 2, the first editing unit 4a, and the first unit. 2 shows the configuration of the editing unit 4b and the printing unit 5. As shown in FIGS. 3 and 4, the image output apparatus includes a control unit 6 that controls each component, an image processing unit 7 that corrects a captured image, a power supply unit 8 that supplies power from an AC power source, A communication unit 9 capable of communicating with an external network N such as the Internet, a reception unit 1 including the above-described coin insertion slot 10, coin selector 11 and display panel 12, a camera 21, a display 22, a strobe lamp 23, and a speaker 24. , 24, a first editing unit 4a including a tablet display 40a, two touch pens 41a and 42a and a speaker 43a, and a second editing unit including a tablet display 40b, two touch pens 41b and 42b and a speaker 43b. 4b, a printer 50, a printing unit 5 including a seal discharge port 51 and a display panel 52, and a bus connecting the units 00 and by the constructed. The control unit 6, the image processing unit 7, the power supply unit 8, the communication unit 9, and the bus 100 correspond to control mechanisms installed inside the first casing 20 and the second casing 30.

  As described above, the hardware constituting each of the receiving unit 1, the photographing unit 2, the first editing unit 4a, the second editing unit 4b, and the printing unit 5 is photographed so that it can be operated by the user of the image output apparatus. It is installed in the space R1 and the editing space R2, and is connected to the bus 100 in the control mechanism installed in the first housing 20 and the second housing 30. On the other hand, the control unit 6, the image processing unit 7, the power supply unit 8, and the communication unit 9 are included in a control mechanism installed inside the first casing 20 and the second casing 30. Are connected to the bus 100.

The control mechanism includes a plurality of personal computers (hereinafter referred to as PCs) installed in the first housing 20 and the second housing 30, the above-described receiving unit 1, the photographing unit 2, and the like.
The first editing unit 4a, the second editing unit 4b, and the control unit for controlling each hardware constituting the printing unit 5 from the PC. Note that the functions of each of the plurality of PCs are divided depending on the control target. For example, the PC that controls the reception unit 1 of the imaging space R1, the PC that controls the imaging unit 2, the PC that controls the first editing unit 4a and the second editing unit 4b, and the captured image are subjected to image processing. The PC that is output by the printing unit 5 is configured so that the functions are divided and operated. In the present embodiment, in order to mainly describe image processing, a first PC that virtually controls the receiving unit 1, the photographing unit 2, the first editing unit 4a, and the second editing unit 4b, and a printing unit that performs image processing. It will be described below that each component is controlled by the operation with the second PC that outputs at 5.

The control unit 6 includes a CPU 60 provided in the first PC that controls hardware of each component, a memory 61 such as a DRAM (Dynamic Random Access Memory) that temporarily stores information generated by the operation of the CPU 60, HDD (Hard Disk Drive) 6 for storing various data
2 and a capture / chroma keyboard 63 for capturing a still image as a photographed image from the video signal output from the camera 21 of the photographing unit 2 and executing chroma key processing.

  The CPU 60 reads and executes a later-described control program 6P stored in the HDD 62, thereby accepting a judgment signal from the coin selector 11 of the accepting unit 1, and sending a control signal to the display panel 12 in accordance with the accepted judgment signal. Send out and realize acceptance of a predetermined charge. Further, the CPU 60 reads out and executes the control program 6P, thereby performing control to synchronize the photographing with the camera 21 of the photographing unit 2 and the operation with the strobe lamp 23, and the print photographed by the camera 21 and stored. A process of acquiring an image for output; a process of acquiring a video signal output from the camera 21 as a captured image for a preview screen and an editing screen via the capture / chroma keyboard 63; a process of displaying on the display 22; Various processes such as a process of accepting editing by the first editing unit 4a and the second editing unit 4b are executed.

  The HDD 62 stores a control program 6P for the CPU 60 to control the hardware of each component. When the CPU 60 reads the control program 6P from the HDD 62 to the memory 61 and executes it, control of each hardware and various software functions are realized.

  In addition to the control program 6P, the HDD 62 includes a foreground / background image to be combined with a photographed printout image, and a foreground / background image to be displayed on the display 22 of the photographing unit 2 together with the photographed image. Is remembered. The HDD 62 further includes images used for operation screens displayed on the display 22 of the photographing unit 2, images used for editing screens displayed on the tablet displays 40a and 40b of the first and second editing units 4a and 4b, Data such as a text file and voice data such as guidance voices output to the speakers 24, 24, 43a, and 43b are stored. The HDD 62 also includes a composite image obtained by executing a composite process with a print output image, foreground image, and / or background image acquired from the camera 21 by the CPU 60 when the user has taken a picture. An edited image obtained by the editing process is stored.

  The capture / chroma keyboard 63 acquires, for example, 30 still images per second from the received video signal and outputs them, and recognizes pixels of a predetermined color tone and brightness from each captured still image, and has a certain range. Hardware having a chroma key function for transparentizing and outputting an alpha value (transparency) for pixels having the same color and lightness. The capture / chroma keyboard 63 is connected to receive a video signal output from the camera 21. The still image after the capture and chroma key processing is written in a memory (not shown) built in the capture / chroma keyboard 63. The CPU 60 reads a still image from the memory as a photographed image for preview, performs a chroma key process, and executes a process for displaying on the preview screen of the display 22.

  In addition, the capture / chroma keyboard 63 can set the number of still images acquired every second by the driver software, and a predetermined color tone and lightness to be made transparent, for example, RGB values. The CPU 60 instructs to make the color tone and lightness of the area in which the sheet material located behind the user in the image photographed by the camera 21 is shown transparent. As a result, the CPU 60 can obtain a person image of only the person who has been subjected to the chroma key processing with the capture / chroma keyboard 63, and can display a preview screen superimposed on the foreground image and / or the background image.

  The image processing unit 7 stores a CPU 70 provided in the second PC for controlling the hardware of each component, a memory 71 such as a DRAM for temporarily storing information generated by the operation of the CPU 70, and various data. An HDD 72 and a graphic board 73 that receives an image and a drawing command and performs a drawing process are included. The image processing unit 7 alone can operate as an image processing apparatus that performs a later-described processing process on a target image.

  The CPU 70 reads out and executes an image processing program 7P, which will be described later, stored in the HDD 72, thereby realizing image processing for processing an image photographed by the camera 21 by photographing based on the control of the control unit 6. Specifically, under the control of the control unit 6, an image taken by the user and photographed by the camera 21 is delivered to the image processing unit 7. The CPU 70 obtains a photographed image from the control unit 6, writes it in the memory 71, and executes image processing so as to improve the appearance such as making the user's face in the image a small face. . Details of the image processing will be described later.

  The HDD 72 stores an image processing program 7P for the CPU 70 to perform image processing such as processing on the acquired image. The image processing program 7P may be a copy of the image processing program recorded on a recording medium such as a CD-ROM or DVD (not shown) in the HDD 72, or may be acquired via the communication unit 9.

  The graphic board 73 is hardware for drawing an image in a built-in memory (not shown) based on the received image and drawing command. The CPU 70 uses the graphic board 73 to perform image processing on the image acquired from the control unit 6 and temporarily stored in the memory 71. Specifically, the CPU 70 gives an image and a drawing command accompanying the image processing to the graphic board 73 to perform the drawing processing, and acquires the image obtained by the drawing processing in the memory 71.

  The power supply unit 8 realizes power supply from the AC power supply to each component as well as the first PC and the second PC. The image output apparatus is provided with a power switch (not shown) for turning on the power, and is connected to the power supply unit 8. The power supply unit 8 detects the on / off of the power switch, and switches the operation / operation of the image output apparatus. The power supply unit 8 includes a UPS (Uninterruptible Power Supply) unit 80. Even when the power from the AC power supply is suddenly cut off by the UPS unit 80, damage to each component can be reduced. When the power switch is switched off, the UPS unit 80 outputs a power failure signal to the CPU 60, and the CPU 60 performs predetermined termination processing when the power failure signal is detected, and turns off the power of the entire image output apparatus.

  The communication unit 9 includes a PHS unit 90 and is connected to the control unit 6 (CPU 60) via the internal bus 100. The PHS unit 90 is connected to an external network N such as the Internet to realize data transmission / reception. The CPU 60 can transmit and receive data by communicating with other devices on the external network N via the PHS unit 90 of the communication unit 9. In the image output apparatus according to the present embodiment, the CPU 90 is connected to the external network N by connecting the image after shooting and editing together with identification information such as the user name and mail address or the URL of the transmission destination. It has a function to transmit to. With this function, it is possible to realize a service that allows a user to browse an image that the user has taken and edited from the server device using a mobile phone or a home PC, and a service that can be downloaded to the mobile phone or the PC.

  An overall process executed by the image output apparatus based on the processing of the CPU 60 when the user receives a photo print providing service in the image output apparatus configured as described above will be described. FIG. 5 is a flowchart illustrating a processing procedure in which the CPU 60 according to the first embodiment executes photography, editing, and printing.

  The CPU 60 determines whether or not a coin satisfying a predetermined fee has been inserted from the coin insertion slot 10 (step S1). If the CPU 60 determines that no coin satisfying the predetermined fee has been inserted (S1: NO), the CPU 60 returns the process to step S1 and waits until it is determined that a coin satisfying the predetermined fee has been inserted. During this time, the CPU 60 outputs a control signal to the display panel 12 so as to display the total amount of coins already inserted on the display panel 12.

  When the CPU 60 determines in step S1 that coins satisfying a predetermined fee have been inserted (S1: YES), the CPU 60 displays a guidance information screen on the display panel 12, and further displays an operation screen on the display 22 of the photographing unit 2, Further, a guidance voice is output from the speaker (step S2), and the user is prompted to enter the shooting booth and execute shooting. Thereby, the provision of the photo print service is started.

  The CPU 60 determines whether the foreground image and / or the background image to be combined is selected by operating the operation screen on the display 22 and the photographing is executed and completed (step S3). Specifically, when a foreground image and / or a background image is selected, counting for operating the shutter is started, and a preview screen is displayed on the display 22. Then, the CPU 60 determines that the count has been incremented, operates the shutter, and executes photography. The shooting is completed depending on whether the user operates a button or the like indicating that the image is determined by looking at the preview screen, or whether a predetermined number of times of shooting such as six times is completed. If the CPU 60 determines that shooting has not been completed (S3: NO), it returns the process to step S3 and waits until shooting is completed.

  If the CPU 60 determines that shooting has been completed (S3: YES), the CPU 60 acquires a photographed image from the camera 21 by operating the shutter, and associates it with the selected foreground image and / or background image in the memory 61. Save (step S4). The CPU 60 displays the guidance information screen on the display 22, displays the operation screen on the tablet displays 40a and 40b of the first editing unit 4a or the second editing unit 4b, and outputs guidance voices through the speakers 24 and 24 ( Step S5), prompting the user to move to the editing space R2 and perform an editing operation.

  Next, the CPU 60 transfers the photographed image stored in the memory 61 to the image processing unit 7 to execute image processing (step S6). The CPU 60 accepts editing on the operation screen of the tablet displays 40a and 40b (step S7). At this time, the image of the user displayed on the operation screen may be an image after image processing. In this case, the CPU 60 performs chroma key processing on the captured image that has been processed to make a small face after image processing, using the capture / chroma keyboard 63 to create an image of only the person, and displays the foreground image and / or background image on the operation screen. And overlaid on the display.

  The CPU 60 determines whether the editing is completed due to the operation for instructing the completion of editing and whether the image processing by the image processing unit 7 is completed (step S8). If the CPU 60 determines that either editing or image processing by the image processing unit 7 has not been completed (S8: NO), the process returns to step S8 and waits until both are completed.

  When the CPU 60 determines that both editing and image processing have been completed (S8: YES), the processed image that has been processed by the image processing by the image processing unit 7, information such as characters or images of the editing result, A synthesis process is performed on the background image and / or the foreground image selected and associated and stored in the memory 61 to generate a synthesized image (step S9). The CPU 60 displays the generated composite image on the operation screen and outputs it to the printer 50 of the printing unit 5 to instruct printing output (step S10). At this time, the CPU 60 may accept a print layout selection and a print instruction on the operation screen.

  The CPU 60 determines whether or not printing has been completed (step S11). If it is determined that printing has not been completed (S11: NO), the process returns to step S11 and waits until it is determined that printing has been completed. To do. When the CPU 60 determines that the printing in the printer 50 is completed (S11: YES), the CPU 60 displays a guidance display indicating that the printing is completed on the display panel 52 of the printing unit 5, the first editing unit 4a, or the second editing unit 4b. The information is displayed on the tablet displays 40a and 40b, guidance voices are output from the speakers 43a and 43b (step S12), and the process is terminated.

  At this time, since the printed photo sticker is discharged to the sticker discharge port 51, the user can obtain the photo sticker in which the user is photographed. Thus, the provision of a photo print service is realized.

  Next, details of the image processing executed by the image processing unit 7 based on an instruction to execute image processing in step S6 in the flowchart shown in FIG. 5 will be described. Note that the image acquired from the camera 21 and delivered from the control unit 6 is digital image data including luminance values and color difference values of a plurality of pixels.

  FIG. 6 is a flowchart illustrating an example of the entire image processing procedure executed by the CPU 70 of the image processing unit 7 according to the first embodiment. The processing described below is realized by the CPU 70 reading and executing the image processing program 7P from the HDD 72.

  The CPU 70 acquires a photographed image delivered from the control unit 6 (step S101), detects a human face from the acquired image (step S102), and specifies a region where each organ in the detected face is captured. (Step S103). Further, the CPU 70 extracts the contour of the detected human face (step S104).

  The face detection, identification of each organ region, and contour extraction from step S102 to step S104 are performed using various existing techniques such as skin color detection and template matching. Specifically, the CPU 70 obtains coordinate information in the image of the region (face region) where the face in the image is captured by the face detection processing in step S102. Specifically, the face area is a circumscribed rectangle of the eyebrow and mouth areas. The face area may be oval. At this time, the tilt of the face is also detected, and when the face of a person in the image is tilted, the horizontal and vertical directions of the face area obtained by face detection are relative to the horizontal and vertical directions of the image. Leaning. At this time, the number of detected faces can be obtained and used in later processing. Further, in the region specifying process of each organ in step S103, the contour of each organ, for example, the eyes, nose, or mouth, is specified by template matching, and several coordinate information on the contour of each organ or each organ is covered. Coordinate information in an elliptical image is obtained. Also in the face outline extraction process in step S104, the face outline is extracted by skin color detection or the like, and coordinate information in several images on the extracted face outline is obtained.

  Next, the CPU 70 performs processing for analyzing the contour of the face obtained by the processing from step S102 to step S104 and coordinate information regarding each organ region (step S105). The details of the analysis process will be described later. Here, the CPU 70 specifies various information for correcting the detected face and sets a processing work area.

  The CPU 70 determines whether or not a plurality of faces are detected from the acquired image based on the result of the face detection process in step S102 (step S106). If the CPU 70 determines that a plurality of faces are not detected from the acquired image (S106: NO), the process proceeds to step S108 described later.

  When the CPU 70 determines that a plurality of faces have been detected from the acquired image (S106: YES), the CPU 70 determines whether or not other contours overlap for each of the contours of each face, and corresponds to the overlapping contours. A process of excluding the face from the processing target is performed (step S107). Details of the overlapping contour removal processing in step S107 will also be described later.

  Next, the CPU 70 creates a grid in the work area set in the analysis processing in step S105 in preparation for performing small face processing for processing the detected face to be a small face (step S108). In creating the grid, for example, the CPU 70 divides the work area by a predetermined number such as 50 vertically and horizontally. At this time, mesh division may be performed, or division according to the golden ratio of the face may be performed.

  The CPU 70 executes small face processing (step S109). Details of the small face processing will be described later.

  The CPU 70 uses the image obtained as a result of executing the small face processing in step S109, executes the processed image creation processing (step S110), generates an image that can be used for the composite image generation processing by the CPU 60 later, The image processing in the image processing unit 7 ends. Details of the processed image creation processing in step S110 will be described later.

  Note that the order of processing from step S102 to step S104 is not limited to the processing procedure shown in the flowchart of FIG. In addition, it is not essential to determine whether or not there are a plurality of faces in step S106. Therefore, the process may proceed to step S107 regardless of whether or not the number of faces is plural. In the process in step S107, the extracted face areas are selected one by one, and the process is performed regardless of whether the number of faces is plural or singular.

  FIG. 7 is a flowchart illustrating an example of analysis processing executed by the CPU 70 of the image processing unit 7 according to the first embodiment. The processing procedure shown in the flowchart of FIG. 7 is the details of the processing procedure of step S105 shown in the flowchart of FIG.

  The CPU 70 selects one face area from the face areas obtained by the face detection (step S51), and specifies the face center based on the position of the eye, nose or mouth area specified in the selected face area. (Step S52). As the face center specifying process in step S52, the CPU 70 may specify the center position of the nose region specified in step S103 as the face center, or the center position of both eye regions specified in step S103. The center of gravity of the triangle connecting the center position of the nose region may be specified as the face center. In the processing by the image processing unit 7 in the present embodiment, the center of the face is specified from the position of the eyes, nose, or mouth, instead of using the center of the rectangle that is the face region obtained by face detection as the center of the face. The user who is provided with the photo print service as in the present embodiment may be in a situation such as posing in an oblique direction without facing the camera 21, and the center of the face area does not necessarily coincide with the center of the face. This is not always the case.

  Based on the face contour, the CPU 70 sets a processing contour line whose contour has been expanded outward as a face area for use as a processing processing boundary line (step S53). Specifically, the CPU 70 enlarges the contour of the face extracted in step S104 around the face center specified in step S52. That is, the CPU 70 multiplies the distance from the center of the face to each point on the face contour by a predetermined enlargement factor, and sets a line connecting points at a distance obtained by multiplying the enlargement factor as a processing contour line.

  Next, the CPU 70 moves to the horizontal axis direction and vertical axis direction of the face area obtained by the face detection, that is, parallel to the horizontal direction and vertical direction of the face, and passes through the center line of the face, that is, the line passing through the center of the nose or mouth. A processing vector to be headed is specified (step S54). The processing vector specified in step S54 is used as a vector indicating the direction and size of image shrinkage in small face processing described later. Here, by specifying a processing vector for each of the horizontal direction and the vertical direction of the face, small face processing described later can be performed at a separate ratio for each of the horizontal direction and the vertical direction of the face.

  The CPU 70 determines a processing coefficient that is a ratio for shrinking the image (step S55). In the small face processing described later, the image is contracted along the horizontal and vertical processing vectors specified in step S54. The processing coefficient is the image contraction rate (small face rate) at this time. This is for adding changes according to the size of the face. Specifically, the CPU 70 determines the processing coefficient based on the detected face width, that is, the width of the circumscribed rectangle of the processing outline. The processing coefficient is set in advance so as to change in accordance with the size of the face area, that is, the ratio to the image, and when the size of the face area is relatively large, that is, when the face appears in front, When the size is relatively small, that is, when the face is in the back, the contraction ratio is increased when the size of the face is relatively large. As a result, it is possible to prevent a situation in which the face of a person in the back is further processed unnecessarily even though it is small.

  The CPU 70 sets a predetermined range including the processing contour expanded in step S53 as a work area (step S56). Here, a circumscribed rectangle of the processing outline (the vertical and horizontal directions of the image are set to the vertical and horizontal directions) enlarged at a predetermined magnification is set as the work area. By setting the work area and limiting the target area for the small face processing described later, the processing speed can be increased.

  The CPU 70 determines whether or not the processing has been performed for all the face regions obtained by the face detection (step S57), and when it is determined that the processing has not been performed for all the face regions (S57: NO), the processing is performed in step S51. Return to. If the CPU 70 determines that the processing has been performed for all the detected faces (S57: YES), it ends the analysis processing and returns the processing to step S106 in the flowchart of FIG.

  FIG. 8 is a flowchart illustrating an example of the overlapping contour removal process executed by the CPU 70 of the image processing unit 7 according to the first embodiment. The processing procedure shown in the flowchart of FIG. 8 is the details of the processing procedure of step S107 shown in the flowchart of FIG.

  The CPU 70 selects one face area from the face areas obtained by the face detection (step S71), and acquires the processing contour set in the selected face area (step S72). The CPU 70 determines whether or not the acquired processing contour line intersects with the processing contour line associated with another face area (step S73). The determination process in step S73 may determine whether or not the distance between the process outlines is not more than a predetermined value as well as whether or not the process outlines intersect each other. When the number of detected faces is singular, there is no other face area, and therefore the CPU 70 determines that it does not intersect with other processing contour lines in the determination processing in step S73.

  When the CPU 70 determines that the acquired processing contour does not intersect with another processing contour (S73: NO), the CPU 70 determines whether or not processing has been performed for all the face regions obtained by face detection (step S74). .

  When the CPU 70 determines that the acquired processing contour line intersects with another processing contour line (S73: YES), the face area in which the acquired processing contour line is set is the target of small face processing, that is, processing processing described later. (Step S75), and the process proceeds to step S74. Specifically, the CPU 70 associates each face area with a flag (OFF by default) indicating whether or not it is out of the processing target, and turns on the flag in step S75. The CPU 70 does not perform processing on a face area in which the flag is turned on in small face processing described later.

  If the CPU 70 determines that the process has not been performed for all the face areas in step S74 (S74: NO), the process returns to step S71, and if it is determined that the process has been performed for all the face areas (S74: YES). The analysis process is terminated, and the process returns to step S108 in the flowchart of FIG.

  When the faces overlap, such as when a plurality of users appear back and forth toward the camera 21, or when the faces are close to each other, such as when looking close together, according to the processing procedure shown in the flowchart of FIG. In addition, a situation in which distortion is generated in the contour due to unnecessary processing on the face region is prevented.

  FIG. 9 is a flowchart illustrating an example of small face processing executed by the CPU 70 of the image processing unit 7 according to the first embodiment. The processing procedure shown in the flowchart of FIG. 9 is the details of the processing procedure of step S109 shown in the flowchart of FIG. Here, when the following processing is performed, a work area is set in the face area of the detected face, and a grid is created in the work area.

  The CPU 70 selects one face area from the face areas obtained by the face detection (step S91), and does not perform the later-described processing (S96, S97) for the work area set in the selected face area. Is created (step S92). Specifically, the CPU 70 sets a masking target area (mask) in the set work area and outside the processing outline, above the eye area, and above the straight line connecting the centers of both eyes. . The mask value for the pixels in the mask is set to “0”, and the mask value for the pixels in the other work areas is set to “1”. Next, the CPU 70 performs a process of blurring the boundary with the inner side of the processing outline in the mask created in step S92 (step S93). By performing the blurring process, the boundary between the area where the mask value is “1” and the area where the mask value is “0” is blurred, and the result of the processing (S96, S97) described later becomes a natural image.

  The CPU 70 determines whether or not the mask processing in steps S92 and S93 has been performed for the entire face area (step S94), and if it is determined that the process has not been performed for the entire face area (S94: NO), the process is stepped. Return to S91. If the CPU 70 determines that the process has been performed for the entire face area (S94: YES), the process proceeds to the next step.

  The CPU 70 again selects one face area from the face areas obtained by the face detection (step S95), and sets a small face ratio to the face corresponding to the selected face area (step S96). The small face rate indicates the strength of the effect of reducing the face, and specifically the shrinkage rate of the image in the work area. An arbitrary value such as 0.9 or 0.8 can be set for the small face ratio. At this time, the small face ratio in the horizontal direction and the small face ratio in the vertical direction are set so that different values can be set for the small face ratio in the horizontal direction and the vertical direction of the face.

The CPU 70 moves each pixel in the work area on which the mask is applied at the small face rate set in step S96 toward the face center for each grid (step S97), thereby realizing small face processing. The drawing for moving the grid is performed by using the graphic board 73 when the CPU 70 gives a drawing command. The amount of movement for each grid varies depending on the distance from the face center. Specifically, the pixels in each grid are moved toward the face center based on the movement amount obtained by the following equation (1).
Horizontal movement amount = distance from the face center × horizontal small face rate × mask value Vertical movement amount = distance from the face center × vertical small face rate × mask value (1)

  As shown in Expression (1), the pixels of the grid whose mask value is set to “0” are not moved. Further, by the blurring process in step S93, the amount of movement of the grid near the boundary with the mask in the work area is adjusted so as to be closer to the boundary, and the amount of movement at the boundary with the mask is zero. Thereby, it is avoided that the image in the work area outside the masked area moves to the center of the face as a whole, and pixels near the boundary with the mask are lost.

  The CPU 70 determines whether or not the predetermined ratio determination and small face processing of steps S96 and S97 have been performed for the entire face area (step S98), and if it is determined that the processing has not been performed for the entire face area (S98: NO), the process returns to step S95. When the CPU 70 determines that the process has been performed for the entire face area (S98: YES), the small face process is terminated, and the process returns to step S110 in the flowchart of FIG.

  In the process shown in the flowchart of FIG. 9, the process is divided into two stages, steps S91 to S94 and steps S95 to S98. After the processes of steps S92 and S93 are performed on the entire face area, the entire face area is again displayed. In contrast, the processing of steps S96 and S97 is performed. This is because the drawing process using the graphic board 73 is performed in step S97, so that the drawing process can be performed efficiently at a high speed by collecting the time occupied by the graphic board 73. Therefore, it is not always necessary to perform the processing in two stages as in the processing procedure shown in the flowchart of FIG.

  Next, a process for creating a processed image reflecting the result of the small face process will be described. FIG. 10 is a flowchart illustrating an example of the processed image creation process executed by the CPU 70 of the image processing unit 7 according to the first embodiment. The processing procedure shown in the flowchart of FIG. 10 is the details of the processing procedure of step S110 shown in the flowchart of FIG.

  When the small face processing in step S109 in the flowchart of FIG. 6 is completed, the CPU 70 draws the original image acquired from the control unit 6 stored in the memory 71 using the graphic board 73 (step S1001). . The CPU 70 again selects one face area from the face areas obtained by the detection (step S1002), and the image in the work area after the small face processing for the selected face area is placed on the image drawn in step S51. Drawing is performed so as to overlap (step S1003). At this time, only the part in the work area is drawn so as to be superimposed on the original image.

  Next, the CPU 70 determines whether or not the drawing process has been performed for the entire face area (step S1004). If it is determined that the drawing process has not been performed for the entire face area (S1004: NO), the process returns to step S1002. . If the CPU 70 determines that the drawing process has been performed for the entire face area (S1004: YES), the processed image creation process ends, the process returns to the flowchart of FIG. 6, and the image process ends.

  According to the processing procedures shown in the flowcharts of FIGS. 6 to 10, even when a person takes a pose with movement, the process of making each face a small face is effectively performed.

  Next, each processing procedure executed by the CPU 70 of the image processing unit 7 shown in the flowcharts of FIGS. 6 to 10 will be described with reference to the drawings showing specific examples.

  FIG. 11 is an explanatory diagram illustrating an example of the content of an image acquired by the CPU 70 from the control unit 6. In the example of content shown in the explanatory diagram of FIG. 11, it can be seen that three people are shown in the image. As shown in FIG. 11, the person shown on the right side and the person shown in the center do not face in front, and the cheek area is not symmetrical with respect to the center line of the face passing through the nose. In this case, when the contour is shifted inward with respect to the left and right contrast with respect to the center line of the skin color region, the face is distorted and unnatural. Also, as shown in FIG. 11, the person shown on the left side and the person shown in the center are posing with their faces tilted. Therefore, if the skin color region is processed around the vertical direction of the image, the face is distorted.

  FIG. 12 is an explanatory diagram illustrating an example of the content of information obtained by the face detection process performed by the CPU 70. It is shown that three rectangles are obtained as detected face areas for the image shown in FIG. 11 by the face detection process. For the person shown on the left side and the person shown in the center, it is shown that the horizontal axis direction and the vertical axis direction of the rectangular face area are inclined with respect to the horizontal direction and the vertical direction of the image. Further, the points on the contours of the organs of the face for each person are specified as the organ regions. Each point shown in FIG. 12 is shown without distinction, but for each organ such as a point on the right eye contour, a point on the left eye contour, a point on the nose contour, etc. by face detection processing. It is detected by distinction. Similarly, points on the face outline are identified.

  FIG. 13 is an explanatory diagram illustrating an example of the face center specified by the analysis processing by the CPU 70. FIG. 13 shows a vertical center line and a horizontal center line of a face passing through the specified face center for each detected face. In the subsequent processing, processing for making the face a small face is performed with the face center shown in FIG. 13 as a reference. Since the small face processing is performed toward the center of the face, an image in which a plurality of persons pose with a motion, such as whether the face is tilted as shown in FIG. 13 or the face is not faced. Even if it exists, it does not become an unnatural image and can be processed effectively.

  FIG. 14 is an explanatory diagram illustrating an example of contour processing lines and work areas set by analysis processing performed by the CPU 70. In FIG. 14, for each person's face, in addition to the two center lines for specifying the face center shown in FIG. 13, the outline of the face specified by the face detection process is indicated by a broken line. Further, 74a, 74b, and 74c in FIG. 14 are process outlines specified by the analysis process, and 75a, 75b, and 75c indicate the set work areas.

  As shown in FIG. 14, the processing contour lines 74a, 74b, and 74c for each person's face are set by enlarging the actual contour indicated by the broken line, and are located outside the actual contour. . By actually setting the lines outside the face contour as the processing contour lines 74a, 74b, and 74c, distortion in the actual contour can be prevented. It is possible to prevent distortion from occurring.

  As shown in FIG. 14, the work areas 75a, 75b, and 75c are set by enlarging the circumscribed rectangles of the processing contour lines 74a, 74b, and 74c at a predetermined magnification. Thereafter, the small face processing is performed at high speed only for the images in the work areas 75a, 75b, and 75c.

  Next, the overlapping contour excluding process shown in the flowchart of FIG. 8 will be described with a specific example. In the case of performing the overlapping contour excluding process for the example shown in FIG. 14, first, the CPU 70 selects the face area of the central person in step S71, for example (S71), and acquires the processing contour line 74b (S72). Then, the CPU 70 determines whether or not the process outline 74b intersects with another process outline 74a or the process outline 74c (S73). In this case, since the CPU 70 determines that it does not intersect with any of the processing contour lines 74a and 74c (S73: NO), no face area is excluded from the processing target.

  FIG. 15 is an explanatory diagram illustrating another example of the contour processing line and the work area set by the analysis processing performed by the CPU 70. The example shown in FIG. 15 is an example in which the faces of two persons in the image are detected. In the example shown in FIG. 15, two persons are shown overlapping each other. And the outline shown with a broken line is extracted with respect to each face area | region. Further, similarly to the example shown in FIG. 14, processing contour lines 74d and 74e and work areas 75d and 75e are set based on the extracted contour.

  In contrast to the example shown in FIG. 15, when the CPU 70 performs the overlapping contour exclusion process, both of the two faces are excluded from the processing target. This is because the processing contour lines 74d and 74e intersect as shown in FIG. As described above, when a plurality of human faces in the image are close to each other so as to overlap each other, there is a possibility that image distortion occurs mutually when the small face processing described later is performed and each pixel is moved. There is. The overlapping contour exclusion process of the image processing unit 7 can prevent a face that may cause distortion from being processed, so that a plurality of persons pose a moving pose and a plurality of persons are in close contact with each other. Even if an image is captured in an overlapping manner, or an image is captured in an overlapping manner, it does not become an unnatural image due to processing.

  In the example of FIG. 14, since the processing contour lines 74a, 74b, and 74c do not intersect, no face is excluded from processing, and in the example of FIG. 15, the processing contour lines 74d and 74e intersect. As a result, both faces were not processed. In the overlapping contour exclusion process, not only when completely intersecting, but when the distance between processing contour lines is less than or equal to a predetermined value, it is determined that the face is close and should not be processed Also good. Accordingly, even in the example of FIG. 14, for example, the distance between the processing contour lines 74b and 74c is equal to or less than a predetermined value, so that the face of the central person and the face of the right person may be excluded from the processing target. Moreover, it is good also as a structure which does not become a process target when not only whether the process outlines cross | intersect but between work areas overlaps.

  Note that the work area 75d and the work area 75e shown in FIG. 15 overlap. The overlapping area may be used as a mask so as to be excluded from the work area.

  Next, the small face processing shown in the flowchart of FIG. 9 will be described with a specific example. FIG. 16 is an explanatory diagram showing an outline of the small face processing by the CPU 70. In FIG. 16, the masks created for the work areas 75a, 75b, and 75c set for each face area are shown in black. The mask shown in FIG. 16 is an area inside the rectangle of the work areas 75a, 75b, and 75c and outside the processing outlines 74a, 74b, and 74c. Furthermore, the area above the image from the straight line connecting the areas of both eyes and the center of both eyes is also set as the mask area. By masking the area and performing small face processing, the boundary between the area where the pixel is moved and the area where the pixel is not moved becomes natural. Note that the mask shown in FIG. 16 shows a state before the blurring process is performed. Further, by using both eye regions as a mask, it is possible to avoid both eyes from being processed and to reduce the eyes, and it is possible to effectively perform the process of making each person's face appear as a small face.

  The work areas 75a, 75b, and 75c can be made smaller by shrinking the work areas as they are toward the center of the face, but in this case, pixel information is missing per rectangular boundary line of the work areas 75a, 75b, and 75c. There is a possibility of unnatural images such as distortion at the boundary.

  Small face processing is performed by moving the pixels in the work areas 75a, 75b, and 75c other than the mask shown in black in FIG. 16 toward the face center along the directions indicated by the arrows for each grid. .

  Then, the work areas 75a, 75b, and 75c after the small face processing are drawn on the original images as shown in the explanatory diagram of FIG. 11 (FIG. 10, S1003). A processed image is created so that the face of the person in the image is a small face.

(Embodiment 2)
In the second embodiment, in addition to the image processing shown in the first embodiment, the face type of the person shown in the image is classified, and the small face ratio in the vertical and horizontal directions of the face is adjusted according to the classification of the face type. Perform processing.

  The configuration of the image output apparatus according to the second embodiment is the same in hardware configuration and software processing procedure except for the details of analysis processing and the details of small face processing. In the following description, components that are the same as those in the first embodiment are denoted by the same reference numerals, and different processing procedures are described.

  FIG. 17 is a flowchart illustrating an example of analysis processing executed by the CPU 70 of the image processing unit 7 according to the second embodiment. Of the processing procedures described in the flowchart of FIG. 17, the same processing steps as those in the flowchart of FIG. 7 described in the first embodiment are denoted by the same step numbers, and detailed description thereof is omitted. .

  The CPU 70 selects the face area of the detected face (S51), specifies the center of the face (S52), sets a processing contour line that further expands the contour of the face (S53), and specifies a processing vector If the processing coefficient is determined (S55), the face classification is determined based on the information (step S58). The CPU 70 stores the face classification discrimination result in association with each face, and uses it when setting the small face ratio in subsequent small face processing.

  Details of the face classification discrimination processing will be described. FIG. 18 is a flowchart illustrating an example of face classification determination processing executed by the CPU 70 of the image processing unit 7 according to the second embodiment.

  Based on the identified face center, the CPU 70 calculates the length a1 to the left intersection of the intersections of the lateral center line of the face passing through the face center and the extracted actual face contour. Specify (step S801). Then, the CPU 70 specifies the length a2 from the center of the face to the right-side intersection of the intersections between the horizontal center line of the face passing through the center of the face and the actual face contour (step S802). Further, the CPU 70 specifies the length a3 from the face center to the intersection of the vertical center line of the face passing through the face center and the actual lower contour of the face (step S803).

Next, the CPU 70 uses the lengths a1, a2, and a3 specified in steps S801 to S803, and the ratio k of the length of the face in the vertical direction to the width of the face, that is, the length in the horizontal direction is equal to a predetermined value. Whether or not (step S804). Where the ratio k is
k = a3 / (a1 + a2)
It is represented by

  If the CPU 70 determines in step S804 that the ratio k is equal to the predetermined value (S804: YES), the CPU 70 classifies the face type into the pattern 1 that is a standard face type (step S805), and ends the classification process. Then, the process returns to step S56 in the flowchart of FIG.

  When determining that the ratio k is not equal to the predetermined value in step S804 (S804: NO), the CPU 70 determines whether the ratio k is larger than the predetermined value (step S806). When the CPU 70 determines that the ratio k is larger than the predetermined value (S806: YES), the face type is classified into the pattern 2 which is a face length type because the length in the vertical direction is long (step S807), and the classification process is performed. Then, the process returns to step S56 in the flowchart of FIG.

  If the CPU 70 determines in step S806 that the ratio k is smaller than the predetermined value (S806: NO), it classifies the face type into the pattern 3 that is a round face type because the horizontal length is long (step S808). ), The classification process is terminated, and the process returns to step S56 in the flowchart of FIG.

  At this time, 0.75 is used as the predetermined value because the ideal aspect ratio of the face is 4: 3. Other values may be used. In step S804, since it is unlikely that the ratio k is determined to be completely equal to 0.75, it is determined that the ratio k is equal to 0.75 within a predetermined range such as plus or minus 0.01.

  The process shown in the flowchart of FIG. 18 will be described with a specific example. FIG. 19 is an explanatory diagram illustrating an example of a contour and a center line specified for a face detected in an image. An example in which face type classification processing is performed on the face of the right person among the three persons in the image shown in the explanatory diagram of FIG. As shown in FIG. 19, the length a1 from the center of the face to the left intersection of the horizontal center line and the face outline is specified, and the length a2 to the right intersection is further specified. A length a3 to the intersection of the direction center line and the lower contour of the face is specified. The face type of the right person is classified by the ratio k calculated based on these lengths a1, a2, and a3.

  The process used when setting the small face ratio in the small face process for the classified face type will be described. FIG. 20 is a flowchart illustrating an example of the small face processing executed by the CPU 70 of the image processing unit 7 according to the second embodiment. Of the processing procedures described in the flowchart of FIG. 20, the processing steps common to the processing procedure of the flowchart of FIG. 9 shown in the first embodiment are denoted by the same step numbers and detailed description thereof is omitted. .

The CPU 70 creates a mask for the entire face area (S92), and performs the process of blurring the mask (S93) (S94: YES), selects one face area again (S95), and determines the selected face. Then, the small face ratio is set based on the face type classification stored in association (step S99). At this time, the CPU 70 sets the small face ratio in the horizontal direction and the small face ratio in the vertical direction based on the face type classification. For example, when the face is determined to be pattern 1 (standard type) by the analysis process, the CPU 70 uses the initially set small face ratio as it is in the horizontal direction and the vertical direction of the face. On the other hand, when the face is determined to be pattern 2 (surface length type) by the analysis process, the CPU 70 makes the small face ratio in the vertical direction of the face stronger than the small face ratio in the horizontal direction of the face. Set to. When the face is determined to be a face length type, the CPU 70 may set the vertical small face ratio based on the following equation (2) so that the aspect ratio of the face approaches an ideal 0.75. At this time, the initially set small face ratio is used as the small face ratio in the horizontal direction.
Small face ratio in the vertical direction = small face ratio in the vertical direction (initial setting) × [0.75 / {a3 / (a1 + a2)}] (2)

Further, when the face is determined to be pattern 3 (round face type) by the analysis process, the CPU 70 sets the small face ratio in the horizontal direction of the face to be stronger than the small face ratio in the vertical direction of the face. To do. When the face is determined to be a round face type, the CPU 70 may set the small face ratio in the horizontal direction based on the following equation (3) so that the aspect ratio of the face approaches the ideal 0.75. At this time, the initially set small face ratio is used as the small face ratio in the vertical direction.
Small face ratio in the horizontal direction = small face ratio in the horizontal direction (initial setting) × [{a3 / (a1 + a2)} / 0.75] (3)

  Next, the CPU 70 moves each pixel in the work area on which the mask is applied toward the face center for each grid at the small face ratio set based on the face classification as described above for the entire face area. (S97) The small face processing is realized and the small face processing is terminated.

  As described above, since the processing is performed according to the face type classification of each face of the person in the image, even if the image is a pose in which the person moves, the face of the person in the image is shown as a small face. Processing can be performed effectively.

(Embodiment 3)
In the third embodiment, in addition to the image processing shown in the first or second embodiment, the area of both eyes in the face of the person appearing in the image is not enlarged so that both eyes become large and unnaturally large. Processing is performed.

  The configuration of the image output apparatus according to the third embodiment is the same in hardware configuration and software processing procedure except for the details of analysis processing and the details of small face processing. In the following description, components that are the same as those in the first or second embodiment will be denoted by the same reference numerals, detailed description thereof will be omitted, and different processing procedures will be described.

  FIG. 21 is a flowchart illustrating an example of analysis processing executed by the CPU 70 of the image processing unit 7 according to the third embodiment. Of the processing procedures described in the flowchart of FIG. 21, the same processing steps as those in the flowchart of FIG. 7 or 17 shown in the first or second embodiment are denoted by the same step numbers and detailed. The detailed explanation is omitted.

  The CPU 70 selects the face area of the detected face (S51), specifies the center of the face (S52), sets a processing contour line that further expands the contour of the face (S53), and specifies a processing vector (S54), the processing coefficient is determined (S55), the face classification is determined (S58), and then the eye enlargement ratio is calculated (step S59). The CPU 70 stores the enlargement ratio calculated for each face in association with each face, and uses it when enlarging the eye area in the subsequent small face processing. Thereafter, the CPU 70 sets a work area (S56), and repeats the process until it is determined that all face areas have been processed (S57: YES).

  Details of the eye enlargement rate calculation process will be described. FIG. 22 is a flowchart illustrating an example of an eye enlargement rate calculation process executed by the CPU 70 of the image processing unit 7 according to the third embodiment. The processing procedure shown in the flowchart of FIG. 22 is the details of the processing procedure of step S59 shown in the flowchart of FIG.

  The CPU 70 obtains the size (number of pixels) of the face area extracted in step S102 during the image processing shown in FIG. 6 (step S9001). Specifically, for example, the CPU 70 obtains the area of the rectangle (see FIG. 12) obtained by the face detection process as the area of the face area. Moreover, it is good also as an area below the horizontal center line of the face passing through the face center specified in S52 of the analysis process in the face area, and extracting the skin color part, the area of the extracted area is You may obtain | require as an area of a face area | region.

  The CPU 70 obtains a reference eye size for the face region size obtained in step S9001 (step S9002). Specifically, the CPU 70 obtains the reference size as the area based on a predetermined ratio as a reference with respect to the area obtained in step S9001.

  The CPU 70 obtains the size (number of pixels) of the eye area specified in step S103 during the image processing shown in FIG. 6 (step S9003). Specifically, the CPU 70 determines, based on the coordinate information of several points on the identified eye contour, or the coordinate information in the elliptical image covering the eye region, the region included in the contour indicated by the coordinate information. Find the area.

  The CPU 70 compares the reference size obtained in step S9002 with the eye region size obtained in step S9003 (step S9004). The CPU 70 calculates the magnification rate of the eyes according to a predetermined mathematical expression according to the comparison result in step S9004 (step S9005). The calculation method in step S9005 is, for example, as follows.

  Assume that the reference size obtained in step S9002 is A, and the size of the eye area of the subject person obtained in step S9003 is B. Here, assuming that the reference eye enlargement rate is E (for example, 1.1 = 110%) and the desired eye enlargement rate is e, the CPU 70 calculates the enlargement rate according to the following equation as a predetermined equation.

  However, in the calculation process in step S9005, the CPU 70 determines whether the calculated enlargement ratio e is 1 (100%) or more and whether the enlargement ratio e falls within a natural range, and uses the coefficient α. Make adjustments. The mathematical formula used is not limited to the formula (4). You may obtain | require using the following formula | equation (5).

  Regardless of which of formula (4) and formula (5) is used, the enlargement ratio is set so that the eyes are larger than the reference size with respect to the size of the face area so that the eyes do not become too large. Is calculated to be small under a condition not less than 1, and for the face whose eyes are smaller than the reference size, the enlargement ratio becomes a relatively large value, and the eye area can be effectively enlarged.

  Thus, the CPU 70 ends the enlargement ratio calculation process, and returns the process to step S56 in the flowchart of FIG.

  The process shown in the flowchart of FIG. 22 will be described with a specific example. FIG. 23 is an explanatory diagram showing an example of the eye size calculated from the face area detected in the image. An example will be described in which the size of the face of the right person among the three persons in the image shown in the explanatory diagram of FIG. 23 is obtained. In FIG. 23, a region connecting points on the outline of the eye obtained by the face detection processing result shown in FIG. 12 of the first embodiment is shown by hatching. In step S9003 shown in the flowchart of FIG. 22, the CPU 70 obtains the area b of the eye region as shown by hatching in FIG.

  In the above description of the flowchart, the size of the face region and the size of the eye region are obtained as areas. However, the present invention is not limited to this, and the size of the face area may be a rectangular horizontal width or vertical width obtained by the detection process. In this case, as shown in FIG. 23, the horizontal width b1 or the vertical width b2 on the eye contour in which the size of the eye region is specified is obtained as the size of the eye region. Then, the CPU 70 may use the horizontal widths or the vertical widths to obtain the enlargement ratio according to the above equation (4). Further, the CPU 70 may multiply the horizontal width b1 by the vertical width b2 and treat it as the area B, and obtain the enlargement ratio by the above-described equation (4).

  Further, in the description of the flowchart described above, the calculation of the reference eye size and the calculation of the size of the eye area specified from the face area of the subject person are obtained as the area (for example, the number of pixels). Any of the required eye sizes may be a ratio (percentage) to the size of the face area calculated in step S9001. At this time, since a reference ratio is given in advance for the size of the reference eye, step S9002 for calculating again is unnecessary.

  Next, small face processing using the obtained eye enlargement ratio e will be described. FIG. 24 is a flowchart illustrating an example of small face processing executed by the CPU 70 of the image processing unit 7 according to the third embodiment. Of the processing procedures described in the flowchart of FIG. 24, the same processing steps as those in the flowchart of FIG. 9 or 20 shown in the first or second embodiment are denoted by the same step numbers. Detailed description is omitted.

  The CPU 70 creates a mask for the entire face area and performs blurring processing (S91 to S94), and performs a grid movement of the face area using the small face ratio set based on the face type classification for the entire face area (S95). ~ S99). The CPU 70 again selects one face area (step S910), and extracts an eye area from the selected face area (step S911). The eye region to be extracted may be a region surrounded by a point on the specified eye contour as shown in FIG. 23, or may be an ellipse including a point on the specified eye contour. Then, the CPU 70 enlarges the extracted eye region based on the eye enlargement rate e calculated for the eye region of the selected face (step S912).

  The CPU 70 determines whether or not the eye area enlargement process of steps S911 and S912 has been performed for the entire face area (step S913), and when it is determined that the process has not been performed for the entire face area (S913: NO) ), The process returns to step S910. When the CPU 70 determines that the process has been performed on the entire face area (S913: YES), the small face process is terminated.

  In the subsequent processed image creation process, the CPU 70 draws an image of the enlarged eye area and an image in the work area after the small face process on the original image. At this time, the image in the work area where the mask process is performed may be superimposed on the top, or the boundary of the eye area is blurred, and the image of the eye area is superimposed on the top. You may do it.

  In this way, using the reference eye size for each face region, the eye size can be appropriately enlarged according to the characteristics of the individual face of the person captured as the subject. In other words, since the enlargement ratio is adjusted individually for the face area in the image with the size of each face size, enlargement of the area of both eyes with the person in the back and the person in the foreground is performed. It can be adjusted appropriately. When there are many cases where multiple people pose a moving pose like a photo booth and overlap each other in front and back, if the same enlargement factor is used to enlarge the eye area, it is behind (in the back) There is a possibility that problems such as unnaturally large enlargement of the human eyes appearing in the image. However, since the image processing unit 7 according to the third embodiment can perform appropriate enlargement individually as described above, an image that satisfies the user can be obtained.

  In addition, since the magnification rate of the eyes is adjusted individually according to the size of the eyes of the person's face as a subject, rather than enlarging at a uniform rate, the areas of both eyes are matched according to faces with different characteristics. It is possible to effectively perform processing that enlarges and shows a small face. In other words, avoid the situation where the eyes of a person with a large face in the image become too large, or the expansion of the eyes of a person with a face with a small eye is insufficient. Even when a certain pose is taken, an image that satisfies each user can be obtained.

  In the first to third embodiments, drawing is realized in hardware using the graphic board 73 in the small face processing of the image processing unit 7. The image processing unit 7 may have a configuration in which the CPU 70 performs the drawing process by software without using the graphic board 73.

  In the first to third embodiments, the image processing unit 7 is a component included in the image output apparatus. The present invention is not limited to this, and the image processing unit 7 may operate independently as one image processing apparatus that processes a person's face with respect to an image read as a processing target. Further, the target of image processing by the image processing unit 7 can be widely applied not only to images in the photo booth but also to image processing for an image of a person.

  Of course, as described above, the scope of the present invention is not limited to this embodiment.

2 photographing unit 21 camera 5 printing unit 50 printer 6 control unit 60 CPU
7 Image processing unit 70 CPU
72 HDD
7P image processing program

Claims (9)

In an image processing apparatus comprising a detecting means for detecting a human face in an image and a processing means for processing the detected face,
Processing boundary setting means for setting a processing boundary line that identifies a boundary of the processing range based on the face contour detected by the detection means;
Means for extracting a predetermined range including the outline of the face from the image as a face area;
A determination unit that determines whether or not a distance in the image between the processing boundary line set by the processing boundary setting unit and the processing boundary line of another face is equal to or less than a predetermined value;
An image processing apparatus comprising: means for excluding a face area of a face corresponding to a processing boundary line whose distance is determined to be equal to or less than a predetermined value from a processing target. The processing boundary setting means includes
Means for extracting a face outline detected by the detection means;
The image processing apparatus according to claim 1, wherein an outside of the extracted contour line is set as a processing boundary line. Means for specifying each area of the face eye, nose or mouth detected by the detection means;
Means for identifying the center of the face based on the position of each identified area,
The processing means includes
The image processing apparatus according to claim 1, wherein an image in an area within the processing boundary line is contracted at a predetermined ratio in each of the vertical and horizontal directions toward the face center specified in the face area. . Means for identifying a first straight line connecting the areas of both eyes of each of the identified eye, nose or mouth areas, and a second straight line passing through the center of the nose area or mouth area in a longitudinal direction with respect to the face;
Means for specifying the length from the intersection of the first straight line and the second straight line to the two intersections where the first straight line intersects the face contour;
Means for specifying the length from the intersection of the first straight line and the second straight line to the intersection at which the second straight line intersects the face contour at the bottom of the face;
Means for classifying the face type of the person shown in the captured image based on each identified length;
4. The image processing apparatus according to claim 3, further comprising: a unit that determines a coefficient for the predetermined ratio for adjusting the vertical and horizontal contraction rates according to the classified types. 5. 5. The image processing apparatus according to claim 3, wherein the processing unit is configured to shrink an image of a region within the face region and further within the processing boundary line except for a region of both eyes. . Means for determining the size of the identified eye area relative to the face area;
Means for comparing the determined size with a predetermined reference size;
Means for calculating an enlargement ratio of the eye size according to the comparison result,
The image processing apparatus according to any one of claims 3 to 5, wherein the processing unit is configured to enlarge an image of the specified region of both eyes based on the calculated enlargement ratio. An image output apparatus comprising: an imaging unit; a combining unit that combines a captured image obtained from the imaging unit with another image; and a unit that outputs the combined image to a predetermined medium.
Image processing means for detecting a human face in the captured image and processing the detected face;
The image processing means includes:
Processing boundary setting means for setting a processing boundary line for specifying a boundary of the processing range based on the detected face contour;
Means for extracting a predetermined range including the outline of the face from the image as a face area;
A determination unit that determines whether or not a distance in the image between the processing boundary line set by the processing boundary setting unit and the processing boundary line of another face is equal to or less than a predetermined value;
An image output apparatus comprising: means for excluding a face area of a face corresponding to a processing boundary line whose distance is determined to be equal to or less than a predetermined value from a processing target. In an image processing method for detecting a human face in an image and processing the detected face,
Set the processing boundary line that identifies the boundary of the processing range based on the detected face contour,
A predetermined range including the face outline is extracted from the image as a face area,
For the set processing boundary line, determine whether the distance in the image between the processing boundary lines of other faces is equal to or less than a predetermined value,
An image processing method comprising: excluding a face area of a face corresponding to a processing boundary line whose distance is determined to be equal to or less than a predetermined value from a processing target. In a computer program that causes a computer to detect the face of a person in an image and process the detected face,
Setting a processing boundary line that identifies the boundary of the processing range based on the detected face contour;
Extracting a predetermined range including a face outline from the image as a face area;
Determining whether the distance in the image between the processing boundary line of the other face and the processing boundary line of the other face is equal to or less than a predetermined value; and
A computer program for executing a step of excluding a face area of a face corresponding to a processing boundary determined to be equal to or less than a predetermined value from a processing target.

Images