JP2009175745A - Face image photographing device and face image photographing method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a face image photographing method and apparatus for outputting excellent images again without calling back and photographing persons to be photographed again even if performing inferior image photographing. <P>SOLUTION: The face image photographing method for photographing at least face images of the persons for the purpose of preparing identification photographs stuck on identifications, various licenses or the like includes: a step of being taken in as continuous images of a plurality of frames by photographing at least face images of the persons; a step of storing the continuous images of the plurality of the taken-in frames as backup images; a step of preserving and outputting the images of prescribed frames in the continuous images of the plurality of the stored frames to a preservation device of a filing device or the other preservation medium; a step of selecting the images of the prescribed frames suitable for output of the face images from the continuous images of the plurality of the stored frame; and a step of outputting the selected images. <P>COPYRIGHT: (C)2009,JPO&INPIT

Description

  The present invention relates to a face image photographing apparatus and a face image photographing method for photographing at least a face image of a person to be photographed, for example, for the purpose of creating a proof photograph to be affixed to an identification card or various licenses.

In general, in this type of face image photographing apparatus, the photographer (shooting operator) provides guidance on the shooting situation, and if the photographer cannot guide, it is necessary to determine the subject.
However, there is a problem that the load of the photographer (photographing operator) is increased due to the conventional guidance of the photographing situation. In particular, when the photographing applicant has a photographing queue, it is necessary to efficiently perform the photographing flow of the photographing applicant and speed up the photographing cycle.

  At that time, the person to be photographed does not know the photographing situation and stays in front of the camera without knowing that the photographing has been completed, thereby hindering the photographing flow of the photographing applicant. In addition, during shooting, the person to be photographed moves by misjudging that the shooting has been completed, and therefore, the shooting becomes defective, which causes re-shooting and hinders the flow of shooting by the shooting applicant.

  The present invention provides a face image capturing method and a face image capturing apparatus capable of re-outputting a non-defective image without recalling the subject again and capturing the image again even if defective image capturing is performed. Objective.

  The face image photographing device according to the present invention captures at least a face image of a subject and captures it as a continuous image of a plurality of frames, and backs up a continuous image of the plurality of frames captured by the image capturing means. Image storage means for storing as an image, and whether or not the pupil state is suitable for print output of a face image by recognizing a state of a pupil from a predetermined frame image in a plurality of frames of continuous images stored in the image storage means And a first print output that prints out the image of the predetermined frame when it is determined by the first determination means that the state of the pupil is suitable for the print output of the face image. And the first determination unit determine that the pupil state is not suitable for the print output of the face image, the predetermined frame in the continuous image of a plurality of frames stored in the image storage unit. A retry control means for performing the determination process of the determination means again using the image of the next frame of the image of the image, and the number of times of the retry process by the retry control means is counted, and the count value is set in advance. A second determination unit that determines whether or not the number of retry processings has reached a specified value, and an error processing unit that performs error processing when the second determination unit determines that the number of retry processes has reached a specified value; An image selecting means for selecting an image of a predetermined frame suitable for outputting a face image from a plurality of consecutive frames stored in the image storage means when the image printed out by one print output means is not appropriate; And a second print output means for printing out the image selected by the image selection means.

  According to the face image photographing method of the present invention, at least a face image of a subject is photographed and captured as a continuous image of a plurality of frames, a step of storing the captured images of the plurality of frames as a backup image, A step of saving and outputting an image of a predetermined frame in the continuous image of the plurality of frames to a storage device such as a filing device or other storage medium, and if the output image is not appropriate, from the stored continuous image of the plurality of frames The method includes a step of selecting an image of a predetermined frame suitable for outputting a face image, and a step of outputting the selected image.

  According to the present invention, it is possible to provide a face image photographing method and a face image photographing apparatus capable of re-outputting a non-defective image without recalling the photographed person again and photographing again even if defective image photographing is performed. .

The figure which shows schematically the external appearance structure of the face image imaging device which concerns on this embodiment. The figure which shows the structure of a guidance display part roughly. The block diagram which shows the whole structure of a face image imaging device. The figure for demonstrating the physical flow of imaging | photography between a face image imaging device, a to-be-photographed person, and the imaging | photography applicant who waits for imaging | photography. The flowchart for demonstrating the processing operation in 1st Embodiment. The flowchart for demonstrating the processing operation in 1st Embodiment. The figure which shows the example of a display of the guidance display part in 1st Embodiment. The figure which shows the example of a display of the guidance display part in 1st Embodiment. The figure which shows the example of a display of the guidance display part in 1st Embodiment. The figure which shows a mode that the outline region of a face is detected from the input image from a camera in 2nd Embodiment, and the magnitude | size of a face is measured from this outline region of a face. The flowchart for demonstrating the processing operation in 2nd Embodiment. The flowchart for demonstrating the processing operation in 2nd Embodiment. The flowchart for demonstrating the processing operation in 2nd Embodiment. The flowchart for demonstrating the prescription | regulation range of the outline region of the face in 2nd Embodiment. The flowchart for demonstrating the top height position detection process in 2nd Embodiment. The flowchart for demonstrating the detection process of the center position of the face of the left-right direction in 2nd Embodiment. The figure which shows the image data example of the input image in 2nd Embodiment. The figure which shows the search range with respect to the input image in 2nd Embodiment. The figure which shows the line where the to-be-photographed person 1 does not exist, and the line which exists in the search area | region with respect to the input image in 2nd Embodiment. The figure which shows the height of the vertex with respect to the input image in 2nd Embodiment, the left head side position head left, and the center position center of a face. The figure which shows the outline area | region of the face with respect to the input image in 2nd Embodiment. The figure for demonstrating the prescription | regulation range of the outline region of the face with respect to the input image in 2nd Embodiment. The figure for demonstrating the detection process of the top height position with respect to the input image in 2nd Embodiment. The figure for demonstrating the detection process of the face left end vertical line: head left with respect to the input image in 2nd Embodiment. The figure for demonstrating the detection process of the face right end vertical line: head right with respect to the input image in 2nd Embodiment. The figure for demonstrating the detection process of the center position center of the face with respect to the input image in 2nd Embodiment. The figure which shows the area | region where both pupils will necessarily exist based on the outline reference position of the face with respect to the input image in 2nd Embodiment. The figure which showed the area | region and coordinate used at the time of each process during the imaging | photography process in the input image from the camera in 3rd Embodiment. The figure which showed the area | region and coordinate used at the time of each process during the imaging | photography process in the input image from the camera in 3rd Embodiment. The flowchart for demonstrating the processing operation in 3rd Embodiment. 10 is a flowchart for explaining a process for determining a shootable state according to the third embodiment. An input image from the camera according to the fourth embodiment, and an approximate position area of the pupil (eye) that is a relative position from the contour position of the face in the input image are specified, and the position area of both pupils from this approximate position area FIG. The flowchart for demonstrating the processing operation in 4th Embodiment. In the fifth embodiment, the approximate position area of the pupil (eyes), which is the relative position from the input image from the camera and the contour position of the face in the input image, is specified, and both pupils are determined from this approximate position area. The figure which shows a mode that a position area | region is extracted and an image cut-out area | region is determined by the relative position from the center position (coordinates) of the position area | region of both pupils. The flowchart for demonstrating the processing operation in 5th Embodiment. The flowchart for demonstrating the processing operation in 5th Embodiment. The flowchart for demonstrating the re-output process in 5th Embodiment. In the sixth embodiment, based on the input image from the camera and the coordinates between the eyebrows as the center of the face in the input image, the region where the pupil will be present is extracted, and the image of the output photo size The figure which shows a mode that an area | region is extracted. The flowchart for demonstrating the processing operation in 6th Embodiment. The flowchart for demonstrating the determination process in 6th Embodiment. 10 is a flowchart for explaining image output processing according to the sixth embodiment. The figure which shows the input image from the camera in 7th Embodiment. The flowchart for demonstrating the processing operation in 7th Embodiment.

Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.
FIG. 1 schematically shows an external configuration of a face image photographing system according to an embodiment of the present invention. This face image photographing system includes an illuminator 2 that illuminates a subject 1, a CCD type video camera 3 as photographing means for photographing at least a face image of the subject 1, a photographing state and a photographer (operator) 4. A display (monitor display) 5 for displaying the operation information of the camera, and a control for processing an input image from the camera 3 according to an operation instruction from the photographer 4 and outputting to the display 5 or a built-in storage unit A face image photographing device having a table (control unit) 6, a guidance display unit 7 as guidance means for guiding and displaying photographing conditions for the photographed person 1, and a chair 8 on which the photographed person 1 is seated when photographing. From a background plate 9 serving as a background of the person to be photographed 1 at the time of photographing, and a speaker 10 or the like as guidance means provided on the back side of the background board 9 and guiding the photographing state and guidance to the photographed person 1 by voice. It is configured.

  The video camera 3 outputs 640 pixels × 480 lines as a captured image of one screen. Each pixel includes R (red), G (green), and B (blue) color sensors, and outputs data (density value) for each color for each pixel.

  Since the background plate 9 is composed of blue, the background when captured by the video camera 3 is blue, and blue is the background color component. The distance of the subject 1 from the imaging unit of the video camera 3 is substantially constant. That is, the video camera 3 is shooting at a fixed shooting distance.

  As shown in FIG. 2, the guidance display unit 7 displays a confirmation-in-progress display unit (second display unit) 11 that performs a display prompting the subject person 1 to wait for photographing, and displays the end of photographing for the subject person 1. An end display unit (third display unit) 12 and an exit display unit (fourth display unit) 13 that performs a display prompting the photographed person 1 to travel in the direction after the end of shooting are configured.

  In this case, the in-confirmation display unit 11 is provided at a substantially central portion in the width direction of the apparatus at the lower part of the camera 3 on the front surface of the apparatus, and the end display unit 12 is provided on the right side with respect to the drawing of the in-confirmation display unit 11. The exit display unit 13 is provided on the right side with respect to the drawing of the end display unit 12.

  Further, as shown in FIG. 2, as one of the guidance display means for guiding and displaying the photographing state for the photographed person 1, the position where the photographed person 1 looks at the camera 3 (position where the viewpoint is adjusted to the camera 3) is displayed. A camera position display unit (first display unit) 14 is provided around the photographing lens 3 a of the camera 3. The camera position display unit 14 is configured, for example, by providing a plurality of LED display elements 14a around the photographing lens 3a, and is controlled to be turned on, blinked, and turned off as necessary.

  FIG. 3 is a block diagram illustrating an overall configuration of the face image photographing system illustrated in FIGS. 1 and 2. That is, the control unit 6 of the face image capturing system includes a video capture unit 21, a video acceleration unit 22, a CPU unit 23, an external bus control unit 24, a mass storage device 25, a reader / writer unit 27, and an I / O control unit 30. , A network adapter (LAN card or the like) 33, a sound control unit 34, or the like.

  The video capture unit 21 converts an analog signal (input image) from the camera 3 into a digital signal in real time, and transfers the digital signal to an MO 26 or a main memory 23b as a video memory described later.

A video accelerator unit 22 stores the image data from the video capture unit 21 in the MO 26 or the main memory 23b as a video memory, or performs a drawing process on the image data from the video capture unit 21 for display. The display control to the device 5 is performed.
The sound control unit 34 controls voice for guidance to the speaker 10.

  The CPU 23 stores a plurality of frames of image data from a CPU (Central Processing Unit) 23a, which performs various arithmetic processes and various controls on image data and the like, and the video capture unit 21, and performs arithmetic operations. A main memory 23b for temporarily storing results and the like and an interface (not shown) with various peripheral devices are provided.

  For example, a keyboard 28 and a mouse 29 are connected to the CPU unit 23 as input devices operated by the operator (photographer) 4 via the interface. On the keyboard 28, a shooting button 28a that is pressed when it is confirmed that the subject 1 is ready for shooting, a shooting confirmation button 28b that is pressed when the shot image is OK, and re-shooting is required. There is provided a shooting cancel button 28c that is pressed when this happens.

  When the icons corresponding to the shooting button 28a, the shooting confirmation button 28b, and the shooting cancel button 28c are displayed on the display 5 and the icons are selected using the cursor moved by the mouse 29, the keyboard 28 The function is the same as when the shooting button 28a, shooting confirmation button 28b, and shooting cancel button 28c are pressed.

The CPU 23 is connected to an image printing apparatus 31 as an output unit that prints out a captured face image, for example, via the interface.
The external bus control unit 24 controls an external bus (for example, SCSI) that connects various peripheral devices.
The large-capacity storage device 25 stores various OSs, applications, control data, image data, and the like, and includes a hard disk device (HDD).

  The reader / writer unit (MO driver) 27 reads / writes data from / to a magneto-optical disk (MO) or the like as a portable storage medium 26 that stores output data of the control table 6 and the like.

  The I / O control unit 30 controls the in-confirmation display unit 11, the end display unit 12, the exit display unit 13, and the camera position display unit 14, respectively.

  A network adapter (such as a LAN card) 33 is for performing mutual communication with other various devices (such as an image printing apparatus and an image filing apparatus) on the same network 32.

  FIG. 4 shows the physical flow of photographing among the face image photographing system, the subject 1 and the photographing applicant 10 waiting for photographing as viewed from above. The person to be photographed 1 usually moves (advances) in the direction of the solid arrow a shown in FIG. 4 when photographing is completed. However, if the person to be photographed 1 moves in the direction of the broken arrow b, the next person to be photographed 10 is replaced. There is a problem.

(First embodiment)
Here, as a first embodiment for preventing such troubles, an embodiment will be described in which the subject 1 is guided in the order of viewpoint alignment, standby, end, and traveling direction.

  The processing operation of this apparatus will be described below with reference to the flowcharts shown in FIGS. The processing described below is mainly executed under the control of the CPU 23a in the CPU unit 23.

  First, in step S1, the in-confirmation display unit 11, the end display unit 12, and the exit display unit 13 are turned off, and other initialization of the apparatus is performed, and the process proceeds to step S2. In step S <b> 2, the camera position display unit 14 is lit in red, for example, so that the photographed person 1 can see the photographing lens 3 a of the camera 3. Encourage as much as possible.

  Next, in step S3, the photographer 4 confirms that the subject 1 is ready for photographing in step S1, and presses the photographing button 28a. Next, in step S4, the camera position display unit 14 is set in a blinking state, for example, to prompt the subject 1 to continue to watch the camera 3 as it is, and the process proceeds to step S5.

  In step S5, an image taken by the subject 1 from the camera 3 is captured via the video capture unit 21, temporarily stored in the main memory 23b in the CPU unit 23, and the process proceeds to step S6. In step S6, the input image stored in the main memory 23b in the CPU unit 23 is displayed on the display 5, and the process proceeds to step S7.

  In step S7, since the image capture is completed, the blinking camera position display unit 14 is turned off, and the process proceeds to step S8. In step S8, the in-confirmation display unit 11 is blinked yellow as shown in FIG. 7, and the process proceeds to step S9. In step S9, the photographer 4 determines whether or not the image displayed in step S6 is blinking or whether the face position is shifted, and determines whether re-shooting is necessary. Meanwhile, the person to be photographed 1 needs to stand by in preparation for re-photographing. Therefore, the in-confirmation display unit 11 is blinked yellow in step S8.

  If it is determined in step S9 that re-shooting is necessary, the process proceeds to step S10, and the photographer 4 depresses the shooting cancel button 28c to return to step S2 and repeats the same operation as described above.

  If it is determined in step S9 that the photographed image is OK, the process proceeds to step S11, and the photographer 4 presses the photographing confirmation button 28b. When the shooting confirmation button 28b is pressed, the process proceeds to step S12, and the in-confirmation display unit 11 is turned off. Next, in step S13, the end display unit 12 blinks green as shown in FIG. 8, and in step S14, the arrow provided on the exit display unit 13 blinks as shown in FIG. As a result, the subject 1 is prompted to proceed in the exit direction since the photographing is completed.

Next, in step S15, the input image stored in the main memory 23b in the CPU unit 23 in step S6 is converted for output, and in addition to other personal information of the person to be photographed 1, depending on the purpose, In the connected image printing apparatus 31 or an image printing apparatus (not shown) connected from the network adapter 33 to the network 32, printing is performed on a recording medium (paper, card, etc.), or the network 32 is connected. The image is stored and output to an image filing device (not shown) as an image storage means, and the main memory 23b, the large-capacity storage device 25, or the portable storage medium 26 in the CPU unit 23 is stored as an image. By using it as a means, it is stored (saved output). This completes all the photographing processes for one person to be photographed.
It should be noted that whether the photographed face image is printed out or stored is selected depending on the purpose, and the method of outputting the face image is arbitrary.

  As described above, according to the first embodiment, the camera position display unit 14 that displays the position at which the camera 3 is viewed with respect to the person to be photographed when the camera 3 is photographing, An in-confirmation display unit 11 for performing a prompting display, an end display unit 12 for displaying the end of shooting for the photographed person, and an exit display unit 13 for performing a display for prompting the traveling direction after the shooting for the photographed person are provided. Thus, the shooting situation can be guided to the person to be photographed 1 by displaying the shooting situation (position where the camera 3 is viewed, shooting standby, shooting end, direction of travel after the end of shooting) for the person to be shot 1. It is possible to minimize the delay of the photographing queue of the photographing applicant 10 as much as possible. In addition, such guidance display can surely provide guidance display not only for healthy people but also for hearing-impaired people who are hard to hear.

  The present invention is not limited to the above-described embodiment, and various modifications can be made without departing from the gist of the present invention. For example, as shown in FIG. 9, the attachment positions of the end display unit 12 and the exit display unit 13 can be changed in accordance with the matrix direction of the photographing applicant 10 to the apparatus and the exit direction of the subject 1. Also good.

  In the example of FIG. 9, when the direction of travel of the subject 1 who has finished photographing is set to the left side with respect to the drawing of the present apparatus, the end display unit 12 translates to the left side with respect to the drawing as it is, 11 is attached to the left side with respect to the drawing of FIG. 11, and the exit display unit 13 is rotated 180 degrees so that the direction is reversed, and is attached to the left side with respect to the drawing of the end display unit 12.

  Further, the camera position display unit 14 displays the position where the photographing lens 3a of the camera 3 is viewed with respect to the photographed person 1. However, the present invention is not limited to this. “Please look at the photographic lens” may be displayed in text, or both of them may be used.

(Second Embodiment)
Next, as a second embodiment, an embodiment in which a face image of a certain size is obtained by measuring the size of the face of the subject 1 and performing a zooming process based on the measured size. Will be described.

  FIG. 10 shows an input image (face image) 41 from the camera 3, and a face outline region 42 detected from the input image 41 and the face size measured from the face outline region 42. It is.

  The processing operation of this apparatus will be described below with reference to the flowchart shown in FIG. 11 and FIG. The processing described below is mainly executed under the control of the CPU 23a in the CPU unit 23.

  First, in step S21, the number of retries counted by a counter (not shown) and other data are initialized, and the process proceeds to step S22. In step S22, an input image (face image of the person to be photographed 1) 41 from the camera 3 is captured via the video capture unit 21, temporarily stored in the main memory 23b in the CPU unit 23, and the process proceeds to step S23. In step S23, details of the detection method including the input image 41 in the main memory 23b to the hairstyle of the person to be photographed 1 will be described later, and the face outline region 42 having a color component different from the color component of the background plate 9 will be described later. Is detected, and the process proceeds to step S24.

  In step S24, the face size value (area, number of pixels) is measured by the face size measurement process, which will be described in detail later, from the face contour area 42 obtained in step S23, and the process proceeds to step S25. Here, the processing in step S24 corresponds to the measuring means in the present invention.

  In step S25, it is determined whether or not the face size measurement value measured in step S24 is within a specified range. The specified range here is a value calculated backward from the input corresponding range of the zooming process to be performed later. For example, when the input corresponding range of zooming processing is a zoom ratio of 0.5 to 1.5, the target face size value / face size measurement value is 0.5 to 1.5. It is within the specified range. Here, the process of step S25 corresponds to the first determination means in the present invention.

  If it is determined in step S25 that the face size measurement value is within the specified range, the process proceeds to step S26. In step S26, a ratio between the target face size value and the face size measurement value is obtained, and the obtained ratio is set as a zoom ratio. Note that the target face size value is the most appropriate face size value when an output photograph is made, and is determined based on, for example, an average of hundreds of face image data. . Here, the processing in step S26 corresponds to the zoom ratio setting means in the present invention.

  Next, in step S27, based on the zoom ratio set in step S26, the CPU unit 23 electronically zooms the input image from the camera 3, and the process proceeds to step S28. Here, the processing in step S27 corresponds to the zooming processing means in the present invention. In this example, the case where the zooming process is performed electronically has been described. However, the zooming process may be performed by moving the lens of the camera 3.

In step S28, the input image stored in the main memory 23b in the CPU unit 23 is converted for output, and other personal information (name, date of birth, ID number, etc.) of the person to be photographed 1 and according to the purpose. The image printing apparatus 30 connected to the CPU unit 23 or the image printing apparatus (not shown) connected from the network adapter 32 to the network 31 prints out on a recording medium (paper, card, etc.) The image data is stored and output to an image filing device (not shown) as an image storage means connected on the network 21. Furthermore, the main memory 23b in the CPU unit 23, the large-capacity storage device 25, or a portable storage device. By using the medium 26 as image storage means, it is stored (saved and output). This completes all the photographing processes for one person to be photographed.
Note that it is only necessary to select whether the photographed face image is printed out or stored in a predetermined area such as an identification card according to the purpose, and the method of outputting the face image is arbitrary.

  If it is determined in step S25 that the measured face size measurement value is out of the specified range, a retry process is performed. That is, first, in step S29, the number of retries is incremented, and the process proceeds to step S30. In step S30, it is checked whether or not the number of retries is within a predetermined value set in advance. If it is within the predetermined value, the process proceeds to step S31. Here, the processes in steps S29 and S30 correspond to the second determination means in the present invention.

  In step S31, if the measured face size measurement value exceeds the specified range (the face is too large), a message prompting the subject 1 to move away from the face is displayed on the display unit 5, In the opposite case, a message prompting the user to approach the face is displayed on the display 5 and the process returns to step S22, and the process from step S22 is performed again as a retry process.

  If it is determined in step S30 that the number of retries exceeds the specified value, it is determined that the image is inappropriate when the output photograph is selected, and the process proceeds to step S32 to perform error processing, and the photographing process is temporarily terminated. Here, the processing in step S32 corresponds to the error processing means in the present invention.

  Next, with reference to the flowchart shown in FIG. 12 and FIG. 10, the first embodiment of the face size measurement process in step S24 of FIG. 11 will be described.

  First, in step S33, a region 43 where both pupils will necessarily exist is calculated as a relative position region from the face contour region 42 obtained in step S23. A part having the same shape, for example, a black and round part is extracted. The area 43 is calculated on the basis of a value obtained by obtaining an area where both pupils always exist with respect to the face outline area in hundreds of face image data, for example.

  Next, in step S34, each black and round part extracted in step S33 is compared with a previously registered pupil sample, and a part having the highest degree of similarity is extracted. Since there are two pupils, left and right pupil samples are prepared, and the above determination is performed by comparing the left and right pupil samples, and the coordinates of the left pupil region 44 and the right pupil region 45 are extracted.

It should be noted that pupil samples may be prepared for either of the left and right pupils, and for the other, the prepared samples may be reduced by processing such as reversal, so that the prepared samples can be reduced.
The above-mentioned pupil sample is obtained by normalizing images obtained by photographing several hundred pupil portions, for example, for pattern comparison for obtaining the degree of similarity.

Next, in step S35, a center distance 46 between the left pupil part 44 and the right pupil part 45 extracted in step S34 is obtained, and a face size value is calculated from the obtained distance 46.
The face size is calculated based on, for example, an average of face size values with respect to the center distance between the left and right pupil parts in hundreds of face image data.

  Next, a second embodiment of the face size measurement process in step S24 of FIG. 11 will be described with reference to the flowchart shown in FIG. 13 and FIG.

  First, in step S36, a region 43 where both pupils are sure to exist is calculated as a relative position region from the face contour region 42 obtained in step S23. A part having the same shape, for example, a black and round part is extracted.

  Next, in step S37, each black and round part extracted in step S36 is compared with a previously registered pupil sample, and a part having the highest degree of similarity is extracted. Since there are two pupils, pupil samples are prepared for the left and right pupils, and the above determination is performed by comparison with both the left and right pupil samples, and the left pupil region 44 and the right pupil region 45 are extracted.

Next, in step S38, a region 47 where the mouth will surely exist is calculated as a relative position region from each center position of the left pupil part 44 and the right pupil part 45 extracted in step S37. A region having red pixels similar to the lips is extracted from the region 47.
This region 47 is calculated based on, for example, a value obtained by obtaining a region including the lips with respect to the center position between the left and right pupil parts in hundreds of face image data.

Next, in step S39, each part extracted in step S38 is compared with the mouth sample registered in advance, and the part having the highest degree of similarity is extracted as the mouth part 48. Next, in step S40, an area 49 between the left pupil part 44 and right pupil part 45 extracted in step S37 and the mouth part 48 extracted in step S38 is obtained. Calculate the magnitude value.
The face size is calculated based on, for example, an average of face size values with respect to the area between the left and right pupil parts and the mouth part in hundreds of face image data.

  Next, the face outline region detection processing in step S23 will be described using the flowcharts shown in FIGS. 14 to 16 and the image data examples shown in FIGS. 17 to 25. FIG. The processing described below is mainly executed under the control of the CPU 23a in the CPU unit 23.

  First, in step S41, the color component of the background board 9 is measured in the measurement range (search range, search area) of the input image 41 in the main memory 23b, and the process proceeds to step S42. In step S42, it is determined whether or not the result measured in step S41 is a color component that is likely to be the background board 9, and the process proceeds to step S43. At this time, the non-blue background (non-blue background) is determined as the face image portion of the subject 1.

  In step S43, a parietal height position detection process, which will be described later, is performed to obtain a vertical top line length direction: head top, and the process proceeds to step S44. In step S44, it is confirmed whether or not the top position is within a specified range, and the process proceeds to step S45.

  In step S45, a left and right face center position detection process, which will be described later, is performed, a face center position lateral line number: center is obtained from the left head position head left and right head position head right, and the process proceeds to step S46. . In step S46, it is confirmed whether the face center position is within the specified range, and the process proceeds to step S47. At this time, the left head side position head left, the right head side position head right, the top height head top, and the average number of lines FACE SIZE of the vertical length of the face of the plurality of subjects 1 The face outline region 42 described above is detected.

In step S47, the position where the horizontal line number: center and the vertical line number: headtop intersect is set as the face contour reference position, and the process proceeds to step S48. In step S48, the processing result is set to "the face outline range is within the specified range", and the process proceeds to step 24.
If the result is NG in step S42, step S44, or step S46, the face contour range is set to be outside the specified range as a processing result, and the process returns.

The measurement range (search range, search area) in step 41 and the method for detecting the top position head top and the face outline area 42 in step 43 will be specifically described.
That is, in the area occupied by the body of the person to be photographed 1 in the input image 41 shown in FIG. 17, the position of the face to be trimmed is always on the upper side unless the person to be photographed 1 stands upright. Above is a blue background area.

Thus, the upper and lower parts (y direction) in the input image 41 are excluded from the search range (search area) A. The parameters at this time will be described with reference to FIG.
The parameter for excluding the upper part is OVER HEAD.
The parameter for excluding the lower part is Y RANGE.

In addition, since the subject 1 sits in front of the camera 3, the lateral position variation is within a certain range, so that both sides (x direction) of the horizontally long image are excluded from the search range A. The parameters at this time will be described with reference to FIG.
The parameter for excluding the left part is assumed to be X START.
The parameter for excluding the right part is assumed to be X STOP.
Further, the area of the subject 1 in the image is almost continuous.

Based on this, the following parameters are set.
Search interval for pixels in the horizontal (X) direction: x interval
Search interval for pixels in the vertical (Y) direction: y interval
Further, assuming that the line in which the subject 1 does not exist in the search area A is 0 and the line in which the photographer 1 exists is 1, the vertical direction (Y direction) 0, 0, 0, 0... 0, 0, 1, 1, 1,..., Data arranged in ascending order of 0 and 1.

  In the horizontal direction (X direction), the first half can be regarded as data arranged in ascending order, and the second half can be regarded as data arranged in descending order. In this case, assuming that the line where the subject 1 does not exist in the search area A is 0 and the line where the photographer 1 exists is 1, 0, 0, 0, 0... 0 as shown in FIG. , 0, 1, 1, 1,..., 1, 1, 1, 0, 0,.

As a result, as shown in FIG. 20, the switching line from 0 to 1 (0 → 1) in the Y direction is detected as the top height (top position: vertical line number in the vertical direction) head top. A switching line from 0 to 1 (0 → 1) in the X direction is detected as a left head side position head left, and a switching line from 1 to 0 (1 → 0) in the X direction is detected as a right head side position head right. Detected.
An intermediate line between the left head position head left and the right head position head right is detected as the center position center of the face.

  The above-described switching line can be obtained by a sequential search from the end, but since the search target is data in ascending order (or descending order), the search area A can be searched more efficiently by performing a binary search. it can.

  Also, the left head side position head left, the right head side position head right, the top height head top, and the average number of lines FACE SIZE of the vertical length of the face of the plurality of subjects 1 are shown in FIG. As shown in FIG. 21, the face outline region 42 described above is detected.

The prescribed range of the face contour region 42 is determined from each value of the horizontal line number: center and the vertical line number: headtop as the face contour reference position. Is a range in which the trimming size can be acquired, and is expressed by the following expression.
X FMG SIZE / 2 <center <(X STILL SIZE-X FMG SIZE / 2)
OVER HEAD <head top <(Y STILL SIZE -Y FMG SIZE + OVER HEAD) = Y RANGE
That is, as shown in FIG. 22, the center position center of the face with respect to the input image 41 is determined from the value of ½ of the number of lines in the horizontal direction of the framing area (trimming area) X FMG SIZE. The range of the value obtained by subtracting ½ of the number of lines in the horizontal direction of the framing area (trimming area) X FMG SIZE from the number of lines X STILL SIZE in the horizontal direction is defined as the specified range.

Also, as shown in FIG. 22, the top position head top for the input image 41 is determined from the vertical line number OVER HEAD of the input image 41 to the framing area (Y STILL SIZE in the vertical direction of the input image 41). Trimming area) The range of the value obtained by subtracting the number of horizontal lines of Y FMG SIZE and adding the vertical line number OVER HEAD of the input image 41 is defined as the specified range.
Note that fmg x is the trimming upper left corner X coordinate, and fmg y is the trimming upper left corner Y coordinate.

  Next, the top height position detection process in step S43 will be described with reference to the flowchart shown in FIG. 16 and the image data shown in FIG.

  First, in step S51, it is confirmed that the top of the subject 1 is in a specified range (OVER HEAD + 1 line to Y RANGE line) where at least the top of the photographer 1 may exist.

  Among the pixels composing the horizontal line of the vertical line number: OVER HEAD in the input image, the pixels different from the color components of the background plate 9 (that is, the face portion of the subject 1) are counted, and the number is equal to or greater than the specified value. Check whether or not.

Further, in the same way, it is confirmed whether or not there are pixels different from the color component of the background plate 9 among the pixels constituting the YRANGE horizontal line, and the pixels different from the color component of the background plate 9 on the OVER HEAD line. Is less than the specified value, and the pixel different from the color component of the background plate 9 on the Y RANGE line is confirmed to be greater than or equal to the specified value.
This prescribed value is a numerical value for ignoring the portion of the subject 1 when the hair of the subject 1 is fluffing sideways or a foreign object different from the color is attached to the background plate 9.

  The vertical line number of the input image: OVER HEAD is set to i and Y RANGE is set to j, and the process proceeds to step S52. In step S52, OVER HEAD is set to i, Y RANGE is set to j, and the process proceeds to step S53.

  In step S53, when it is confirmed that there are more lines than 0 lines between the i line and the j line, the process proceeds to step S54. In step S54, the center line (i + j) / 2 of the i line and the j line is set to k, and the process proceeds to step S55.

  In step S55, pixels that are different from the color components of the background plate 9 among the pixels constituting the horizontal line of k are counted, and it is determined whether the number is equal to or greater than a specified value. This specified value is a numerical value for ignoring the portion of the subject 1 when the hair of the subject 1 is fluffing sideways or when a minute foreign object different from the color is attached to the background plate 9.

  If the determination result in step S55 is yes, k is set to j in step S56, and the process returns to step S53. If the determination result in step S55 is no, k is set to i in step S57. Return to S53.

  In this way, from the search area from OVER HEAD to Y RANGE of the input image, this search area is halved for each processing loop, and j−i is less than 1 (when there is no difference in the number of lines, 0 Line), that is, if the determination result in step S53 is no, the process proceeds to step S58.

  In step S58, i on the parietal portion Y line is set to the parietal position head top, and the process proceeds to step S59. In step S59, the top position head top is set within the specified range as a processing result, and the process proceeds to step 44.

  If the determination result in step S51 is yes, that is, if there is no face image data in Y RENGE and there is face image data in OVER HEAD, if it is determined that the face image is not within the specified range, the process goes to step S60. move on. In step S60, the top position is set to be outside the specified range as a processing result, and the process proceeds to step 44.

  Next, the detection processing of the center position of the face in the left-right direction in step S45 will be described with reference to the flowchart shown in FIG. 16 and the image data shown in FIGS.

  First, in step S61, the leftmost vertical line number 0 of the input image 41 shown in FIG. 24 is set to i, the center vertical line number (X STILL SIZE / 2) of the input image is set to j, and the process proceeds to step S62. . In step S62, it is confirmed whether there are more lines than 0 lines between the i line and the j line. If yes, the process proceeds to step S63.

  In step S63, the vertical line (i + j) / 2 at the center of the i line and the j line is set to k, and the process proceeds to step S64. In step S64, out of the pixels constituting the vertical line of k, pixels having a color different from the color component of the background plate 9 measured in step S41 are counted, and it is determined whether the number is equal to or greater than a predetermined value. This specified value is a numerical value for ignoring the portion of the subject 1 when the hair of the subject 1 is fluffing sideways or when a minute foreign object different from the color is attached to the background plate 9.

  If the determination result in step S64 is yes, k is set to j in step S65, and the process returns to step S62. If the determination result in step S64 is no, k is set to i in step S66. Return to S62.

In this way, the search area is halved for each processing loop from the search area in the left half of the input image, and when j-i becomes less than 1, that is, the determination result in step S62 is no The process proceeds to step S67.
In step S67, when j-i becomes less than 1 in step S62, i is set to the left face vertical line: head left, and the process proceeds to step S68.

  In step S68, the center vertical line number ((X START + X STOP) / 2) of the input image 41 shown in FIG. 25 is set to i, the right end vertical line number (X STOP) of the input image is set to j, step Proceed to S69. In step S69, it is confirmed whether there are more lines than 0 lines between the i line and the j line, and if no, the process proceeds to step S70.

  In step S70, the vertical line (i + j) / 2 at the center of the i line and the j line is set to k, and the process proceeds to step S71. In step S71, pixels having a color different from the color component of the background plate 9 measured in step S41 among the pixels constituting the k vertical line are counted, and it is determined whether the number is equal to or greater than a predetermined value. This specified value is a numerical value for ignoring the portion of the subject 1 when the hair of the subject 1 is fluffing sideways or when a minute foreign object different from the color is attached to the background plate 9.

  If the determination result in step S71 is yes, k is set to i in step S72, and the process returns to step S69. If the determination result in step S71 is no, k is set to j in step S73. Return to S69.

  In this way, the search area is halved for each processing loop from the search area in the right half of the input image, and when j-i becomes less than 1, that is, the determination result in step S69 is yes The process proceeds to step S74.

  In step S74, when j-i becomes less than 1 in step S69, i is set to the face right edge vertical line: head right, and the process proceeds to step S75. In step S75, as shown in FIG. 26, the center position is calculated based on the face left end vertical line head left set in step S67 and the face right end vertical line head right set in step S74. The value is set as the value of center, and the process proceeds to step 46.

  The face contour region 42 including the hairstyle of the subject 1 in the step 23 includes the face contour reference position (center, head top), head left: left head side position, head right: right head side position, FACE. SIZE: Determined by the average number of lines of the vertical length of the faces of a plurality of subjects 1.

As shown in FIG. 27, the region 43 where both pupils are sure to exist in step S21 is abscissa: x eye area posi, ordinate based on the face outline reference position (center, head top). The area of the horizontal size: X EYE AREA SIZE and the vertical size: Y EYE AREA SIZE with y eye area posi as the left vertex is calculated. The abscissa: x eye area posi and the ordinate: y eye area posi are as follows.
x eye area posi = center- (X EYE AREA SIZE / 2)
y eye area posi = head top + Y EYE POSI SIZE- (Y EYE AREA SIZE / 2)
That is, with respect to the center (abscissa of the face outline reference position), the width of the predetermined area 43 (the number of horizontal lines indicating the minimum area where both pupils will always exist: X EYE AREA SIZE) The coordinates obtained by subtracting half of the) are the abscissas (x eye area posi) of the left end portion of the region 43.

  The average vertical line number (Y EYE POSI SIZE) from the top position to the pupil position is added to the headtop (the vertical coordinate of the face contour reference position), and the vertical width of the predetermined region 43 (always) The coordinate obtained by subtracting half of the number of vertical lines (Y EYE AREA SIZE) indicating the minimum area where both pupils will be present becomes the ordinate (y area areposi) of the upper end of the area 43.

  In the above-described embodiment, the positions of both pupils and mouths in the face of the subject are detected from the captured face image, and the face position is determined from the detected positions of both eyes and mouth. Although the case where the size is measured has been described, the present invention is not limited to this, and the positions of both eyes, the mouth, and the nose in the face of the subject are determined from the captured face image. Each face may be detected, and the size of the face may be measured from the detected positions of both eyes, the mouth, and the nose.

Therefore, in the conventional face image photographing device, when photographing the subject, the size of the face is determined by a human system, and the face size is too large or too small. The operator (photographer) adjusts the size (zooming).
For example, when the shooting cycle is as fast as several seconds, the human system determination has a heavy load on the operator (photographer) of the imaging apparatus at the time of shooting, and the probability of erroneous determination is high.
In particular, if an erroneous determination is made after taking an image of a wrong judgment and printing it out, then the subject who is not already there must be recalled and taken again. In addition, there is a problem that the cost of an output medium (such as paper) accompanying printing output is wasted.

  On the other hand, according to the second embodiment described above, the above-mentioned drawbacks can be avoided, and when at least the face image of the subject is photographed, the face size of the input face image is automatically recognized. Thus, the zooming process can be automatically performed so that the size of the face of the output image is always uniform.

(Third embodiment)
Next, as a third embodiment, an embodiment will be described in which the state of the subject 1 and the mouth is monitored to recognize the shootable state and perform automatic shooting. It should be noted that the calculation of the face contour area in the second embodiment is performed in the same manner.
28 and 29 show an input image (face image) from the camera 3, and regions and coordinates used during each processing during the photographing processing.

  The processing operation of this apparatus will be described below with reference to the flowchart shown in FIG. 30 and FIGS. 28 and 29. The processing described below is mainly executed under the control of the CPU 23a in the CPU unit 23.

  First, in step S81, the number of retries counted by a counter (not shown) and other data are initialized, and the process proceeds to step S82. In step S82, the input image 41 from the camera 3 is captured via the video capture unit 21, temporarily stored in the main memory 23b in the CPU unit 23, and the process proceeds to step S83.

  In steps S83 and S84, detection processing is performed to determine whether or not the subject 1 is present in front of the camera 3 based on the input image 41 in the main memory 23b. That is, for example, by searching for a region having a pixel different from the background color in the input image 31 (that is, the region of the subject), the subject 1 exists and even the hairstyle of the subject 1 is reached. It is determined whether the position of the included face outline region 42 is within an appropriate range for output. Here, the processes in steps S83 and S84 correspond to the subject detection means in the present invention.

  If the subject 1 is not present in front of the camera 3 or if the position of the face contour region 42 is inappropriate, the process returns to step S82 and the same operation as described above is repeated. If the subject 1 is present in front of the camera 3 and the position of the face outline region 42 is appropriate, the process proceeds to step S85. In step S85, in order to urge the subject 1 in front of the camera 3 to prepare for shooting, a guidance message is output from the display 5 and the speaker 10, and the process proceeds to step S86. Here, the processing in step S85 corresponds to the photographing guide means in the present invention.

  In step S86, after inputting the image 41, the process waits for a specified time for determining that the person to be photographed 1 is stationary (prepared for shooting) in the determination process of the shooting enabled state, which will be described later, and proceeds to step S87. move on. In step S87, the input image 41 'continuously following the previous input image 41 is again taken from the camera 3, temporarily stored in the main memory 23b in the CPU unit 23, and the process proceeds to step S88.

  In steps S88 and S89, it is determined whether or not the subject 1 is ready to shoot by performing a shootable state determination process using the input image 41 and the input image 41 'in the main memory 23b. Details of this photographing enabled state determination process will be described later. Here, the processing in steps S88 and S89 corresponds to the recognition means in the present invention.

As a result of the determination, if the subject person 1 is ready to shoot, the process proceeds to step S90, and is suitable for outputting a face image out of the input images 41 or 41 'stored in the main memory 23b in the CPU unit 23. The selected image is converted for output, and together with other personal information of the photographed person 1, the image printing apparatus 31 connected to the CPU unit 23 or the network adapter 33 is connected to the network 32 according to the purpose. In a connected image printing apparatus (not shown), it is printed out on a recording medium (paper, card, etc.) or stored in an image filing apparatus (not shown) as an image storage means connected on the network 32. The image is output, and further, the main memory 23b in the CPU unit 23, the mass storage device 26, or the portable storage medium 27 is stored as an image. By using it as a means, it is stored (saved output). This completes all the photographing processes for one person to be photographed. Here, the process of step S90 corresponds to the selection means in the present invention.
It should be noted that whether the photographed face image is printed out or stored is selected depending on the purpose, and the method of outputting the face image is arbitrary.

  If the result of the determination is that the subject 1 is not ready to shoot, the process proceeds to step S91 and retry processing is performed. That is, first, in step S91, the number of retries is incremented, and the process proceeds to step S92. In step S92, it is checked whether or not the number of retries is within a predetermined value set in advance. If it is within the predetermined value, the process returns to step S82, and the processes from step S82 are performed again as the retry process. Here, the processing of steps S91 and S92 corresponds to the determination means in the present invention.

  If the number of retries exceeds the specified value in the determination in step S92, it is determined that the image is inappropriate when the output photograph is selected, and the process proceeds to step S93 to perform error processing and end the photographing process. Here, the processing in step S93 corresponds to the error processing means in the present invention.

  Next, with reference to the flowchart shown in FIG. 31 and FIGS. 28 and 29, the determination process of the photographing enabled state in steps S88 and S89 in FIG. 30 will be described.

First, in step S101, a region 43 where both pupils will necessarily exist is calculated as a relative position region from the face contour region 42, and a part having the same shape as the pupil, for example, Extract black and round parts.
The area 43 is calculated on the basis of a value obtained by obtaining an area where both pupils are always present with respect to the face outline area in hundreds of face image data, for example.

  Next, in step S102, each of the black and round parts extracted in step S101 is compared with a pupil sample registered in advance, and the degree of similarity of each part exceeds the specified value. Extract high parts. Since there are two pupils, left and right pupil samples are prepared, and the above determination is performed by comparing the left and right pupil samples, and the coordinates of the left pupil region 44 and the right pupil region 45 are extracted.

Next, in step S103, a region 47 where the mouth is sure to exist is calculated as a relative position region from the center positions of the left pupil region 44 and the right pupil region 45 extracted in step S102. A region having red pixels similar to the lips is extracted from the region 47.
The area 47 is calculated based on a value obtained by calculating and averaging an area including the lips with respect to the center position between the left and right pupil parts in, for example, hundreds of face image data.

  Next, in step S104, the part extracted in step S103 is compared with the mouth sample registered in advance, and the highest part among the parts having a degree of similarity exceeding a specified value is determined as the coordinate 48 of the mouth part. Extract as Next, in step S105, between the pixels in the prescribed peripheral region 50 of the left pupil coordinate 44 in the input image 41 of FIG. 28 and the region 52 of the same size in the input image 41 ′ of FIG. An absolute value of luminance difference is obtained, and a total value SumDef1 in the entire area is obtained.

  Next, in step S106, between the pixels in the specified peripheral region 51 of the right pupil coordinate 45 in the input image 41 of FIG. 28 and the region 53 of the same size in the input image 41 ′ of FIG. An absolute value of luminance difference is obtained, and a total value SumDef2 in the entire area is obtained.

  Next, in step S107, between each pixel in the prescribed peripheral region 53 of the coordinate 48 of the mouth part in the input image 41 of FIG. 28 and the region 54 of the same size in the input image 41 ′ of FIG. An absolute value of luminance difference is obtained, and a total value SumDef3 in the entire area is obtained.

Next, in step S108, the sum values SumDef1, SumDef2, and SumDef3 of the luminance difference absolute values obtained in steps S105 to 107 are respectively compared with specified values set for them, and are all equal to or less than the specified values. Judgment is made.
As a result of this determination, if all of the three numerical values are equal to or less than the predetermined value, it is determined that the photographing is possible, and if one of the three numerical values exceeds the predetermined value, it is determined that the photographing is not possible. .

  If the subject 1 is ready for photographing, the face and the eyes and mouth, which are moving parts in the face, are stationary, so the above three numerical values are lower than the prescribed values. On the contrary, if the face is moving, the above three numerical values are higher than the prescribed values. In addition, when there is a blink, SumDef2 and SumDef3 each have a higher value than the specified value. When the conversation or the mouth is moving, SumDef3 is higher than the specified value.

The specified time in step S86 described above is a motion monitoring time for allowing the above three numerical values to be sufficiently quantified when the subject 1 is not ready for photographing and has a behavior. .
Therefore, in the conventional face image photographing device, when photographing the subject, the operator (photographer) determines the photographing preparation state (photographable state) of the subject and the photographing is performed.

  However, for example, the shooting cycle is as fast as a few seconds, and the operator (photographer) needs to observe the state of the subject while operating the imaging device. The load is large, and the probability of misoperation is high.

  In particular, if an erroneous determination is made after taking an image of a wrong judgment and printing it out, then the subject who is not already there must be recalled and taken again. In addition, there is a problem that the cost of an output medium (such as paper) accompanying printing output is wasted.

  On the other hand, according to the above-described third embodiment, the above-described drawbacks are avoided, and the eyes and mouth are monitored by a plurality of images obtained by continuously capturing the image of the image-captured person. By recognizing the possible state, it is possible to automatically recognize the photographable state of the person to be photographed without automatically determining the human system, and automatically perform photographing.

  In the above embodiment, a case where it is recognized whether the subject is ready to shoot by monitoring the state of the eyes and mouth in the face of the subject with a plurality of captured images. Although described above, the present invention is not limited to this, and it is possible to recognize whether the subject is ready to shoot by monitoring the state of eyes, mouth and nose in the face. .

(Fourth embodiment)
Next, as a fourth embodiment, an embodiment will be described in which the state of the pupil of the person to be photographed 1 is recognized based on a photographed image and it is determined whether the pupil is normally open. This embodiment is for processing after photographing. It should be noted that the calculation of the face contour area in the second embodiment is performed in the same manner.

  FIG. 32 identifies an input image (face image) 61 from the camera 3 and an approximate position area 63 of the pupil (eyes) that is a relative position from the face outline position 62 in the input image 61. The manner in which the position areas 64 and 65 of both eyes are extracted from the area 63 is shown.

  The processing operation of this apparatus will be described below with reference to the flowchart shown in FIG. 33 and FIG. The processing described below is mainly executed under the control of the CPU 23a in the CPU unit 23.

  First, in step S111, the number of retries and other data are initialized, and the process proceeds to step S112. In step S112, an input image (face image) 61 from the camera 3 is captured via the video capture unit 21, and temporarily stored in the main memory 23b in the CPU unit 23, and the process proceeds to step S113. In step S113, the face contour position 62 including the hairstyle is detected from the input image 61 in the main memory 23b, and the process proceeds to step S114.

  In step S114, it is checked whether or not the detected face contour position 62 is within a predetermined range, and if it is not within the predetermined range, the process proceeds to step S115 to perform error processing and return to the initial state. Return.

If it is within the specified range in step S114, the process proceeds to step S116. In step S116, the approximate position area 63 of the pupil (eyes), which is the relative position, is calculated from the face contour position 62, and the round position area (64, 64) having the same shape as the pupil is calculated from the approximate position area 63. 65) is extracted, and the flow advances to step S117.
The approximate position area 63 is calculated on the basis of a value obtained by obtaining an area where both pupils always exist with respect to the face outline area in hundreds of face image data, for example.

  In step S117, it is checked whether or not the number of extracted round-shaped position areas is equal to or larger than a predetermined number (2) set in advance. If the number is not the predetermined number, the pupil is not normally opened. And the process proceeds to step S118. In step S118, after waiting for a predetermined time (about 0.1 second) set in advance, the process proceeds to step S119.

In step S119, it is checked whether or not the number of retries is within a predetermined value set in advance, and if it is within the predetermined value, the process proceeds to step S120. In step S120, the number of retries is incremented, and the process returns to step S112. That is, if the number of retries is within the specified number, the process waits for a specified time (about 0.1 second) and returns to step S112 to input the face image again as a retry.
If it is determined in step S119 that the value is equal to or greater than the specified value, the process proceeds to step S121 to perform error processing and return to the initial state.

  If the prescribed number is satisfied in the check in step S117, it is determined that the pupil is normally open, and the process proceeds to step S122. In step S122, the extracted shape around the position areas 64 and 65 is compared with the sample data of the pupil stored in the main memory 23b in the CPU unit 23 in advance, and the extracted part having the highest degree of similarity is determined. The pupil is extracted, and the process proceeds to step S123.

In step S123, it is checked whether or not the similarity degree of the most similar part extracted in step S122 is equal to or higher than a preset threshold value. If it is equal to or lower than the threshold value, the process proceeds to step S119, and the retry process similar to the above is performed. Do. If the result of the check is greater than or equal to the threshold value, the pupil position is determined to be normally open, and the process proceeds to step S124.
In this case, since there are two pupils, sample data is prepared for the left and right pupils, and the above determination is made by comparing with both sample data of the left and right pupils to recognize the state of both the left and right pupils.

In step S124, the input image captured in step S112 is converted for output, and together with other personal information of the person 1 to be photographed, the image printing apparatus 31 connected to the CPU unit 23 or a network adapter depending on the purpose. An image printing apparatus (not shown) connected from 33 to the network 32 prints out on a recording medium (paper, card, etc.) or an image filing apparatus (not shown) as image storage means connected to the network 32. The main memory 23b in the CPU unit 23, the large-capacity storage medium 25, or the portable storage medium 26 is used as an image storage means to store (save) it. Output. This completes all processing.
Whether the face image is printed out or saved is selected according to the purpose, and the method of outputting the face image is arbitrary.

  By the way, when the subject 1 is photographed, if there is blinking or blinking, it is determined in steps S116 and S117, and a process of waiting for the pupil to open is performed while performing a retry process.

  If the subject 1 is not a normally open pupil whose pupil is open but is not looking away or looking forward, the determination in steps S122 and S123 is made. I wait to become a pupil. Moreover, the process of step S122, S123 operate | moves also as a correction process with respect to what was wrongly determined by the process of step S116, S117.

  Furthermore, the processing in steps S122 and S123 also acts on an abnormally open pupil (surprised eye) that cannot be determined by the processing in steps S116 and S117, and performs retry processing.

  Therefore, in the conventional face image photographing device, when the subject is photographed, the human system determines whether or not the pupil (eyes) is open, and the image is not suitable as an output photograph such as blinking eyes. If there were, they were taking measures such as re-taking.

However, when the shooting cycle is as fast as several seconds, the human system determination has a heavy load on the operator (photographer) of the shooting apparatus at the time of shooting, and the probability of erroneous determination is high.
In particular, if an erroneous determination is made after taking an image of an erroneous determination and printing it out, it is necessary to recall the imaged person who is not already there and take the image again. In addition, there is a problem that the cost of an output medium (such as paper) accompanying printing output is wasted.

On the other hand, according to the above-described fourth embodiment, the above-described drawbacks are avoided, images that are not suitable as output photographs such as blinking are automatically excluded, and a facial image that can always output the best image. An object is to provide a photographing method and a face image photographing device.
As described above, images that are not suitable as output photographs such as blinking are automatically excluded, and the best image can always be output.

(Fifth embodiment)
Next, as a fifth embodiment, a description will be given of an embodiment in which several consecutive frames of image data of a plurality of frames are stored, and one image is selected as a photographing result from the stored images.

  FIG. 34 specifies an input image (face image) 71 from the camera 3 and an approximate position area 73 of the pupil (eyes) that is a relative position from the face contour position 72 in the input image 71, and this approximate position. Both pupil position areas 74 and 75 are extracted from the area 73, and the image cutout area 77 is determined based on the relative position from the center position (coordinates) 76 of the both pupil position areas 74 and 75. .

  The processing operation of this apparatus will be described below with reference to the flowcharts shown in FIGS. 35 and 36 and FIG. The processing described below is mainly executed under the control of the CPU 23a in the CPU unit 23.

The photographing process of this apparatus is started by waiting for the photographer (operator) 4 to be ready for the photographed person 1 and pressing the photographing button 28 a in the keyboard 28.
First, in step S131, the number of retries, the frame number, and other data are initialized, and the process proceeds to step S132. In step S132, an image capture start command is output, and the process proceeds to steps S133 and S135. In step S133, it is confirmed that a start instruction for image capture processing has been output, and the process proceeds to step S134.

  In step S134, the input image (face image) 71 from the camera 3 is continuously captured from the frame numbers “0” to “n” via the video capture unit 21, and the CPU as a continuous image (moving image) of a plurality of frames. The data is temporarily stored in a main memory 23b as a work memory in the unit 23. This capture process is performed while the image capture process start command is output. That is, when images from frame numbers “0” to “n” are captured, the output of the image capture processing start command is stopped.

  On the other hand, in step S135, in parallel with the processing in steps S133 and S134, the process waits for the completion of capturing of the image of the corresponding frame number (for example, frame number “0”, which is initially set in step S131). When the capture is completed, the process proceeds to step S136. In step S136, the face contour position 72 including the hairstyle is detected from the input image 71 of the frame number “0” stored in the main memory 23b in the CPU unit 23, and the process proceeds to step S137.

In step S137, it is checked whether or not the detected face contour position 72 is within a predetermined range. If the detected face contour position 72 is not within the predetermined range, the process proceeds to step S138 to perform error processing and return to the initial state. Return. If it is within the specified range, the process proceeds to step S139. In step S139, the approximate position area 73 of the pupil (eyes), which is the relative position, is calculated from the face outline position 72, and the position area 74 having a round shape that is the same shape as both pupils is calculated from the approximate position area 73. 75 is extracted, and the process proceeds to step S140.
The area 73 is calculated on the basis of a value obtained by obtaining an area where both pupils are always present with respect to the face outline area in hundreds of face image data, for example.

  In step S140, it is checked whether or not the extracted images (pupils) of the round-shaped position regions 74 and 75 are in a state suitable for print output. If the image is suitable for print output such as blinking or blinking. If not, the process proceeds to step S141.

In step S141, it is checked whether or not the number of retries is within a preset specified value. If it is within the specified value, the process proceeds to step S142. In step S142, the number of retries and the frame number are incremented, and the process returns to step S135. That is, if the number of retries is within the specified number, the process returns to step S135 to retry with the next frame number “1” image.
If it is determined in step S11 that the number of retries is equal to or greater than the specified value, the process proceeds to step S143 to perform error processing, and the process returns to the initial state.

If it is determined in step S140 that there is no blink or blinking in the image and the image is suitable for print output, the process proceeds to step S144. In step S144, a center position (coordinates) 76 between the position areas 74 and 75 is calculated from the position areas 74 and 75 of both eyes extracted in step S139, and an image cutout area 77 is calculated based on the relative position from the center position 76. The image in the image cutout region 77 is cut out from the input image, and the process proceeds to step S145.
This region 77 is calculated based on a value obtained by calculating and averaging image cut-out regions with respect to the center position between the left and right pupil parts in hundreds of face image data, for example.

  In step S145, the image cut out in step S144 is recorded on a recording medium (paper, card) in a printing unit (not shown) connected to the network 32 from the LAN card 33 according to the purpose, together with other personal information of the person to be photographed 1. To a storage device such as a filing device (not shown) connected on the network 32, a large-capacity storage medium 25 such as a main memory 23b or a hard disk in the CPU 23a, or a portable storage medium 26 The process proceeds to step S146. In step 146, the process waits for the completion of the capture of the image of frame number “n”.

  In step S147, the n frames of continuous images temporarily stored in the main memory 23b in the CPU unit 23 are stored as backup images in the large-capacity storage medium 25, and the process ends.

  Based on the result of the print output in step S145, for example, the photographed person 1 was open, turned sideways, or misjudged in the processing of steps S139 and S140, etc. If the photographer 4 determines that there is no image, a suitable image is selected from the continuous images for n frames that are backed up and output again. Hereinafter, this will be described in detail with reference to the flowchart shown in FIG.

  First, in step S151, the photographer 4 inputs the re-output symmetrical reference number registered at the time of shooting from the keyboard 28, and proceeds to step S152, and the backup image corresponding to the reference number is stored in the mass storage medium 25. Search from. If there is no corresponding image as a result of this search (S153), the process proceeds to step S154, a message indicating that there is no image with the corresponding number is displayed on the display 5, and the process ends.

  If there is a corresponding image as a result of the search (S153), the process proceeds to step S155, all the frames of the corresponding backup image are read and displayed on the display 5, and the process proceeds to step S156. In step S156, the photographer 4 selects an image suitable for the proof photograph from the images of all the frames displayed on the display 5, and inputs the frame number from the keyboard 28, whereby the process returns to step S157. move on.

  In step S157, an image corresponding to the frame number input from the keyboard 28 is retrieved from the large-capacity storage medium 25 and read out, and the read image is printed out in the same process as in step S15, and re-output processing is performed. finish.

  Therefore, in the conventional face image photographing device, when the subject is photographed, it is determined by the human system whether or not the image is normal, and if the image is not suitable as an output photograph such as blinking , I was taking measures such as re-taking.

However, when the shooting cycle is as fast as several seconds, the human system determination has a heavy load on the operator (photographer) of the shooting apparatus at the time of shooting, and the probability of erroneous determination is high.
In particular, if an erroneous determination is made after taking an image of a wrong judgment and printing it out, then the subject who is not already there must be recalled and taken again. In addition, there is a problem that the cost of an output medium (such as paper) accompanying printing output is wasted.

  On the other hand, according to the above-described fifth embodiment, even if the above-mentioned drawbacks are avoided and defective image shooting is performed, a non-defective image can be reproduced again without recalling the photographed person and shooting again. Can be output.

  The above-described re-output processing program has been described on the assumption that the re-output processing program is in the present apparatus. However, the re-output process program is connected to the re-output process dedicated terminal device connected to the same network 32 via the apparatus and the network adapter 33. It may be provided.

  The n frames of continuous images stored in backup in the large-capacity storage medium 25 are erased by obtaining information on completion of issuance from a certification photo issuance management system separately provided on the same network, for example. It has become.

(Sixth embodiment)
Next, as a sixth embodiment, an embodiment in which a face image is extracted based on clicking a position between eyebrows as a reference position in photographed image data will be described.

  FIG. 38 extracts an area 82 where a pupil will exist based on the input image (face image) 81 from the camera 3 and the coordinates of the inter-brow 83 as the center of the face in the input image 81; It shows how the image area 84 of the output photo size is extracted.

  The processing operation of this apparatus will be described below with reference to the flowcharts shown in FIGS. 39, 40, and 41 and FIG. The processing described below is mainly executed under the control of the CPU 23a in the CPU unit 23.

In this embodiment, a description will be given of a process of shooting when a portion between the eyebrows of the face in the display image is clicked instead of the shooting button 28a.
First, in step S161, an input image (face image) 81 from the camera 3 is taken in via the video capture unit 21, and the process proceeds to step S162. In step S162, the input image 81 is displayed on the display 5 via the video acceleration unit 22, and the process proceeds to step S163. In step S163, it is determined whether or not there is a click from the mouse 29. If it is determined that there is no click, the process returns to step S1 again. That is, when there is no click, the display 5 is displayed in a state close to a moving image.

When it is determined in step S163 that there is a click, the determination process (S164) described in FIG. 40 and the image output process (S165) described in FIG. 41 are performed, and the process returns to step S161.
Step S164 is processing for determining whether or not the image has no blinking when the mouse 29 is clicked, and will be described with reference to FIG.

  First, in step S171, the number of retries and other data are initialized, and the process proceeds to step S172. In step S172, the input image 81 captured from the camera 3 is temporarily stored in the buffer number corresponding to the number of retries in the main memory 23b in the CPU unit 23, and the process proceeds to step S173.

In step S173, a region 82 where a pupil will exist is calculated based on the coordinates 83 between the eyebrows clicked with the mouse 29 on the image in the Nth image buffer, and the process proceeds to step S174. That is, the approximate position area 82 of the pupil (eye), which is the relative position, is calculated from the coordinates 83 of the position between the eyebrows. In step S174, the position area (84, 85) having the same shape as the pupil is extracted from the area 82, and the process proceeds to step S175.
The region 82 is calculated on the basis of a value obtained by averaging and calculating a region where both pupils always exist with respect to the coordinates of the position between the eyebrows in, for example, hundreds of face image data.

  In step S175, it is checked whether or not the number of extracted round-shaped position areas is equal to or larger than a predetermined number (2) set in advance. If the number is not the predetermined number, the pupil is not normally opened. It is determined that the subject 1 was blinking or blinking when the subject 1 was photographed, and the process proceeds to step S176.

  In step S176, it is checked whether or not the number of retries is within a predetermined value set in advance, and if it is within the predetermined value, the process proceeds to step S177. In step S177, after waiting for a predetermined time (about 0.1 second) set in advance, the process proceeds to step S178.

  In step S178, the number of retries is incremented, and the process returns to step S172. That is, if the number of retries is within the specified number of times, the process waits for a specified time (about 0.1 second) and returns to step S172 to input the face image again as a retry.

Thereby, when the subject 1 is photographed, if it is blinking or blinking, it is determined in the process of step S175, and while waiting for the eyes to open while performing the retry.
If the subject is not a normally open eye that has eyes open but is not looking away or looking forward, the determination in step S177 is made and the eyes are normally open. Wait for Further, the process of step S177 also operates as a correction process for those erroneously determined in the process of step S175.

Further, the process of step S177 also acts on abnormally open eyes (surprised eyes) that cannot be determined by the process of step S175, and performs a retry process.
If it is determined in step S176 that the value is equal to or greater than the specified value, the process proceeds to step S177 to perform error processing, and the process returns to the initial state.

  If the prescribed number is satisfied in the check in step S175, it is determined that the pupil is normally open, and the process proceeds to step S178. In step S178, the extracted shapes around the position areas 84 and 85 are compared with the sample data of the pupil stored in advance in the Nth image buffer of the main memory 23b in the CPU unit 23, and the degree of similarity is compared. The extracted part with the highest left is extracted for the left and right pupils, and the process proceeds to step S179.

In step S179, it is checked whether or not the similarity degree of the most similar part extracted in step S178 is equal to or higher than a preset threshold value. If it is equal to or lower than the threshold value, the process proceeds to step S176, and the retry process similar to that described above is performed. Do. As a result of the check, if it is equal to or greater than the threshold value, it is determined that the pupil position is normally open, and the process proceeds to image output processing in step S165.
In this case, since there are two pupils, sample data is prepared for the left and right pupils, and the above determination is made by comparing with both sample data of the left and right pupils to recognize the state of both the left and right pupils.

  Next, the image output processing in step 165 after the determination processing will be described with reference to the flowchart shown in FIG.

First, in step S181, it is determined whether the immediately preceding determination processing result is an error. If the result of this determination is that there is an error, the buffer number is “0”, that is, the image buffer that records the image closest to the timing at which the shooting was clicked is selected (step 182), and processing proceeds to step S183.
If it is determined in step S181 that the immediately preceding determination process is not an error, the image buffer having the currently registered buffer number is selected, and the process proceeds to step S183.

  In step S183, an image area 84 having an output size is extracted from the image buffer image selected in step S181 or S182 based on the clicked coordinates, and the process proceeds to step S184.

In step S184, if necessary, the clipped image is compressed and secured in the large-capacity storage medium 25 or the portable storage medium 26, or via the network adapter 33, not shown on the network 32. Output to a printing press or filing device.
In this embodiment, a mouse is used as a pointing device, but there is also a method of directly clicking the screen with a touch panel.

Therefore, in the conventional face image photographing device, the steps from photographing an image to outputting are assured and secured in the storage medium by pressing the photographing button, capturing a wide range of images including the face. Trim around the face in the image and output the clipped part.
For this reason, the shooting operation process is long and complicated, and it takes a long time to output after shooting.

  On the other hand, according to the above-described sixth embodiment, it is possible to avoid the above-described drawbacks and to perform a complicated operation of trimming a captured image when capturing at least a face image of the subject. By directly indicating the position of the face on the display screen, an image that is not suitable as an output photograph such as blinking can be automatically excluded, and the best image can always be output.

(Seventh embodiment)
Next, as a seventh embodiment, the captured image data is displayed on the monitor, and when the image capture is instructed based on the monitor display, the image displayed on the monitor immediately before this instruction is adopted. An embodiment to be described will be described.

FIG. 42 shows an input image (face image) 91 from the camera 3.
The processing operation of this apparatus will be described below with reference to the flowchart shown in FIG. 43 and FIG. The processing described below is mainly executed under the control of the CPU 23a in the CPU unit 23.

In this embodiment, a process of outputting a monitor image immediately before the depression based on the depression of the photographing button 28a will be described.
First, in step S191, an input image (face image) 91 from the camera 3 is captured via the video capture unit 21, and the input image 81 passes through the video acceleration unit 22 and is stored in the main memory 23a of the CPU unit 23. Recording in the buffer unit proceeds to step S192. In step S192, the input image 81 is displayed on the display 5, and the process proceeds to step S193. In step S193, it is determined that the shooting button 28a on the keyboard 28 has been pressed. If it is determined that the shooting button 28a has not been pressed, the process returns to S191 again. That is, when the shooting button 28a is not pressed, the display 5 is displayed in a state close to a moving image. In addition, an image before the current photographed image is recorded in the buffer portion of the main memory 23a.

  When it is determined in step S193 that the shooting button 28a has been pressed, the image displayed on the display 5 when the shooting button 28a stored in the buffer unit is pressed is determined as a shot image. (Step S194). The image determined to be a captured image is secured in the large-capacity storage medium 25 or the portable storage medium 26, or is output to a printing machine or filing device (not shown) on the network 32 via the network adapter 33 (step). S195).

  Therefore, in the conventional face image photographing device, when the photographed image is determined to be appropriate while looking at the monitor of the photographed image, the photograph button is pressed, and the image is determined by the signal (timing) when the photograph button is pressed. The captured image is taken as the captured image.

  In such a conventional method, the time taken for pressing the shooting button, the time required for capturing the image, etc. between the monitor image determined to be an appropriate image and the image captured as the shot image As a result, a different image, which is delayed in time from the image determined to be appropriate, is captured. For this reason, there is a case where the image is significantly different from the determined appropriate image, and it is necessary to perform photographing again.

  On the other hand, according to the seventh embodiment described above, a confirmation instruction is given when the above disadvantage is avoided, at least a face image of the subject is photographed, and the monitor display of the photographed image is confirmed. By outputting the previous monitor screen, the best image can always be output.

  DESCRIPTION OF SYMBOLS 1 ... Subject, 2 ... Illuminator, 3 ... Video camera, 4 ... Photographer (operator), 5 ... Display (monitor display), 6 ... Control stand (control part), 9 ... Background board, 10 ... Speaker: 11... Confirming display section (second display section), 12... End display section (third display section), 13... Exit display section (fourth display section), 14. 1), 21... Video capture unit, 22... Video acceleration unit, 23... CPU unit, 23 a... CPU, 23 b... Main memory, 25. , 28 ... keyboard, 29 ... mouse, 31 ... image printing device (output means), 32 ... network.

Claims (2)

Image capturing means for capturing at least a face image of the subject and capturing as a plurality of frames of continuous images;
Image storage means for storing a continuous image of a plurality of frames captured by the image capture means as a backup image;
First determination means for recognizing a state of a pupil from images of predetermined frames in a plurality of consecutive frames stored in the image storage means and determining whether or not the pupil is suitable for printing out a face image; ,
A first print output unit that prints out the image of the predetermined frame when the first determination unit determines that the state of the pupil is suitable for the print output of the face image;
When the first determination unit determines that the pupil state is not suitable for the print output of the face image, the next frame of the image of the predetermined frame in the plurality of consecutive images stored in the image storage unit Retry control means for performing the determination process of the determination means again using the image of
A second determination unit that counts the number of times the retry process is performed by the retry control unit and determines whether or not the count value has reached a predetermined value;
When the second determination unit determines that the number of retry processes has reached a specified value, an error processing unit that performs error processing;
Image selection for selecting an image of a predetermined frame suitable for outputting a face image from a plurality of continuous images stored in the image storage means when the image printed out by the first print output means is not appropriate Means,
Second print output means for printing out the image selected by the image selection means;
A face image photographing apparatus comprising: Capturing at least a face image of the subject and capturing it as a continuous image of a plurality of frames;
Storing the captured multiple frames of continuous images as backup images;
A step of storing and outputting an image of a predetermined frame in the stored continuous image of a plurality of frames to a storage device such as a filing device or other storage medium;
If the output image is not appropriate, a step of selecting a predetermined frame image suitable for the output of the face image from the stored continuous image of the plurality of frames;
Outputting the selected image; and
A method for photographing a face image.

Images