JP2003108979A - Face area extraction method, device and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To make a face area easily extractable from the image of a person including the face, and to make the person easily confirmable. SOLUTION: The image of a person including the face recorded on a memory card 12 is displayed on a liquid crystal monitor 14. Designation of at least one reference point on the face area is received by an input means 15, and the face area is extracted from the image displayed on the monitor 14 with the designated reference point as reference. A minimum rectangular area including the extracted face area is set on the image displayed on the monitor 14, and is expanded such that the minimum rectangular area fits a screen size of the monitor 14. The expanded minimum rectangular area is displayed on the monitor 14.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a face area extraction method and apparatus for extracting a face area from an image including a human face, and a program for causing a computer to execute the face area extraction method.

[0002]

Reproducing image data acquired by a digital camera, a digital video or the like on a liquid crystal monitor of a digital camera or the like, a liquid crystal monitor of a mobile terminal such as a mobile phone or a PDA, or a liquid crystal monitor or a CRT monitor of a personal computer. Is being done. In particular, if the image is reproduced on a digital camera or a mobile terminal, the image can be viewed without being restricted by the place or the time, so that the image can be viewed and enjoyed when gathering with friends.

However, since the liquid crystal monitor attached to the digital camera or the portable terminal is small in size, when an image including a person is displayed, it is possible to specify which person is included in the displayed image or to identify the person. It is difficult to confirm a person, such as checking whether his eyes are closed. Also, although the monitor of a personal computer has a larger screen than the LCD monitor of a digital camera or a mobile terminal, if the image itself to be displayed is not so large,
It is difficult to confirm a person as in the case of displaying an image on a small monitor.

For this reason, a face area included in a displayed image is surrounded by a rectangular area, and the rectangular area is enlarged and displayed (prior art 1). Further, it is also performed to accept designation of one point on a face area included in an image and to enlarge and display the image centered on the one point (prior art 2). According to these methods, since the face area can be enlarged and displayed, the person can be surely confirmed.

[0005]

However, in the method of the above-mentioned prior art 1, since it is necessary to set a rectangular area including a face area on the image, the work is very troublesome. Although it is possible to automatically set a rectangular area, it is difficult to accurately surround the face area with the rectangular area because the image includes not only the face but also the background.
Further, in the method of the related art 2, since the enlargement ratio is preset, the enlarged face area may not fit in the monitor. In such a case, it is difficult to confirm the person because the entire face area cannot be grasped unless the screen is scrolled.

The present invention has been made in view of the above circumstances, and it is a first object of the present invention to easily extract a face area from an image including a person's face.

A second object of the present invention is to easily confirm a person.

[0008]

A face area extraction method according to the present invention accepts designation of at least one reference point on a face area in an image including a human face displayed on a monitor, and the designated reference point. It is characterized in that the face area is extracted from the image on the basis of points.

The "monitor" may be a monitor having a relatively small display size provided in a digital camera or a mobile terminal such as a mobile phone or a PDA, or a CRT monitor or a liquid crystal monitor such as a personal computer. .

The reference point can be specified by using any input method such as clicking the mouse, moving and confirming the point with the ten keys on the left, right, up and down, pen input, or input with the touch panel.

As a method for extracting a face area, a method is used in which an image is radially scanned around a designated reference point to extract an edge portion of a face, and an area surrounded by the edge portion is extracted as a face area. JP-A-5-27443
The method of extracting a face area by a neural network described in Japanese Patent No. 8 and Japanese Patent Laid-Open No. 5-307605 can be used. In addition, the image is divided into color regions based on the photometric data of the image, and one of the color regions is
Select one of the three areas as the area of interest, perform the contraction process of the selected area of interest to find the core of the area of interest, and find the circle with the largest area inscribed in the area of interest centered on the obtained core. A method of determining the end of division for a region and repeating the process to divide the region of interest into circles having a large area, and determining that the largest divided circle after the region of interest has been divided is the face region ( JP-A-5-165119
Issue) may be used.

In the face area extracting method according to the present invention, a minimum rectangular area including the extracted face area is set on the image to obtain a desired display image size,
The minimum rectangular area may be enlarged so that the minimum rectangular area matches the display image size.

In this case, the enlarged minimum rectangular area may be reproduced.

The "minimum rectangular area" means an area including the face area to the maximum extent in the area, and is preferably an area inscribed in the face area.

As the "desired display image size",
It may be the screen size of the monitor or the size of the window for displaying the image. It may also be the size of the recording paper to be printed out. Further, “obtaining a desired display image size” may be obtained from a keyboard, a touch panel, a dial button of a mobile phone or a pen input, but a digital camera having a communication function, a mobile phone When downloading image data from the image server that stores the image data and displaying the image on a display terminal such as a terminal or a personal computer, when accessing the image server from the display terminal, the monitor size, window size or recording paper The size may be sent to the image server. Further, the image server may instruct the display terminal to input the monitor size, window size, or recording paper size. In such a case, the transmitted or input monitor size, window size, or recording paper size may be acquired as the desired display image size. Also, as monitors are attached to digital cameras,
The display image size is known and stored in the memory of a digital camera or the like. In this case, the display image size may be acquired by reading the display image size stored in the memory.

"Enlarge the smallest rectangular area so that the smallest rectangular area fits the display image size" means to enlarge the smallest rectangular area so that the smallest rectangular area is displayed within the display image size to the maximum extent. Say. Specifically, one of the vertical and horizontal lengths of the minimum rectangular area may be enlarged to be substantially the same as the length of one side of the display image size. If the minimum rectangular area and the display image size have different aspect ratios, a margin may occur around the minimum rectangular area when the enlarged minimum rectangular area is reproduced. Therefore, in order to match the aspect ratios of the minimum rectangular area and the display image size, it is preferable to expand the area including the area around the minimum rectangular area. Further, the aspect ratio of the minimum rectangular area may be changed to match the aspect ratio of the display image size.

The term "enlarge" means to generate an image enlarged so as to fit the display image size with respect to the image displayed on the monitor, and the minimum rectangle from the image displayed on the monitor. Not only when extracting an area and enlarging the extracted minimum rectangular area to generate an image that matches the display image size, but also from an image that is the same as the image displayed on the monitor but has a different resolution, It also includes extracting a corresponding area and enlarging or reducing the area to generate an image that matches the display image size.

When displaying the enlarged minimum rectangular area on the monitor of a digital camera, on the monitor of a mobile terminal such as a mobile phone or PDA, or when printing out, it is reproduced on a monitor such as a personal computer. Compared to the case where the image is reproduced, whether the image is vertically or horizontally long, the image can be erected by rotating the mobile terminal or the recording paper itself, so there is no problem in observing. . Therefore, when the enlarged minimum rectangular area is displayed on the monitor of the digital camera, displayed on the mobile terminal, or printed out by the printer, the minimum rectangular area is enlarged as much as possible. It is preferable that the minimum rectangular area is rotated and then enlarged so that the long side direction of the display screen and the long side direction of the display screen coincide with each other. Even in this case, the aspect ratio of the minimum rectangular area may be changed according to the display screen size.

The face area extracting apparatus according to the present invention includes an input means for accepting designation of at least one reference point on a face area in an image including a human face displayed on a monitor.
Extraction means for extracting the face region from the image using the designated reference point as a reference is provided.

In the face area extracting device according to the present invention, setting means for setting a minimum rectangular area including the extracted face area on the image, and acquiring means for acquiring a desired display image size, You may further comprise the expansion means which expands the said minimum rectangular area so that the said minimum rectangular area may match the said display image size.

In this case, a reproducing means for reproducing the enlarged minimum rectangular area may be further provided.

The face area extracting method according to the present invention may be provided as a program for causing a computer to execute the method.

[0023]

According to the present invention, in an image including a human face displayed on a monitor, designation of at least one reference point on a face area is accepted, and the designated reference point is used as a reference from the image. The face area is extracted. Therefore, the face area can be extracted without the trouble of setting a rectangular area including the face area on the image. In addition, since the face area is extracted with reference to at least one reference point designated on the face area, it is not necessary to detect the position of the face area in the calculation of the face area extraction. The face area can be extracted.

Further, by setting the minimum rectangular area including the extracted face area, receiving the input of the desired display image size, and enlarging the minimum rectangular area so that the minimum rectangular area matches the display image size. The face area is enlarged and displayed. For this reason, the enlarged face area does not completely fill the display screen, and as a result, the entire face area can be grasped without scrolling the screen. Therefore, the person included in the displayed image can be easily confirmed.

[0025]

BEST MODE FOR CARRYING OUT THE INVENTION Embodiments of the present invention will be described below with reference to the drawings. FIG. 1 is a schematic block diagram showing the configuration of a digital camera including a face area extracting device according to the first embodiment of the present invention. As shown in FIG. 1, the digital camera 1 according to the first embodiment captures an image of a subject and obtains image data S0 representing the subject by an image capturing unit 11 including a lens, a shutter, a CCD, and the like, and the image capturing unit 11 acquires the image data S0. The recording means 13 for recording the recorded image data S0 in the memory card 12 and reading out the image data S0 recorded in the memory card 12, the liquid crystal monitor 14 for displaying the image, the ten keys on the left, right, up and down, and the enter button Input means 15 and face area extraction means 16
And a control means 17 for controlling the image pickup means 11, the recording means 13, the liquid crystal monitor 14, and the face area extraction means 16.

The face area extracting means 16 displays the image represented by the image data S0 on the liquid crystal monitor 14, accepts the designation of at least one reference point on the face area in the displayed image by the input means 15, and designates it. The face area is extracted using the extracted reference points as a reference, the minimum rectangular area including the extracted face area is set on the displayed image, and the minimum rectangular area is set to be approximately the same size as the liquid crystal monitor 14. The rectangular area is enlarged and the enlarged minimum rectangular area is displayed on the liquid crystal monitor 14. The face area extracting means 1
6 corresponds to the extraction means, the setting means, the acquisition means, and the expansion means.

Next, the operation of the first embodiment will be described. FIG. 2 is a flowchart showing the operation of this embodiment. In the present embodiment, it is assumed that the photographing of the subject has already been completed and the image data S0 representing the image including the face of the person is stored in the memory card 12. First,
By the selection of the user who is the operator of the digital camera 1,
An image for extracting the face area is displayed on the liquid crystal monitor 14 (step S1). This image is displayed on the LCD monitor 14
The thumbnail image in which the resolution of the image represented by the image data S0 recorded in the memory card 12 is lowered is displayed, and the image to be displayed is selected by the input unit 15. Here, it is assumed that an image including the faces of three persons is selected as shown in FIG. The control means 17 creates the image data representing the thumbnail image.

Next, as shown in FIG. 4, the user uses the ten keys of the input means 15 to specify one reference point on any face area. This designation is made by moving the pointer P displayed on the liquid crystal monitor 14 to the desired face area with the ten keys and pressing the confirm button.
Although only one reference point may be designated, a plurality of reference points are preferable in order to improve the extraction accuracy of the face area, which will be described later.

Then, it is judged whether or not the reference point is designated (step S2), and if the step S2 is affirmed,
The face area is extracted using the designated reference point as a reference (step S3). As a method of extracting the face area, a method of radially scanning an image around a designated reference point to extract an edge portion of a face and defining an area surrounded by the edge portion as a face area is disclosed in Japanese Patent Laid-Open No. 5-56 274438
It is possible to use a method of extracting a face area by a neural network described in Japanese Patent Publication No. 5-307605 and the like. Further, the image is divided into color regions based on the photometric data of the image, one region is selected from the color regions as a region of interest, and the contraction process of the selected region of interest is performed to obtain the core of the region of interest. , Find the circle with the largest area inscribed in the region of interest centered on the obtained core, determine the end of division for all regions of the region of interest, and repeat this to divide the region of interest into circles with the largest area. , A method of determining the largest divided circle as a face area after the attention area has been divided (Japanese Patent Laid-Open No. 5-165119)
Issue) may be used. As a result, as shown in FIG. 5, the face area including the designated reference point is extracted.

Next, as shown in FIG. 6, the minimum rectangular area R1 including the extracted face area is set (step S
4). As the minimum rectangular area, an area that is inscribed in the extracted face area or an area that surrounds the face area with a slight gap from the face area can be used. Then, the screen size of the liquid crystal monitor 14 is acquired (step S5). Since the liquid crystal monitor 14 is provided in the digital camera 1 in the present embodiment, it is stored in a memory (not shown) as a known one, and the screen size stored in this memory is read to determine the screen size. Is obtained.

Then, the minimum rectangular area R1 is enlarged so that the minimum rectangular area R1 matches the screen size (step S6). Specifically, the enlargement processing is performed by performing interpolation calculation on the image data representing the image in the minimum rectangular area R1. At this time, one of the vertical and horizontal lengths of the minimum rectangular area R1 may be enlarged so as to be substantially the same as the length of one side of the liquid crystal monitor 14. When the minimum rectangular area R1 and the liquid crystal monitor 14 have different aspect ratios, a blank space may occur around the minimum rectangular area during reproduction of the enlarged minimum rectangular area R1. For this reason,
In order to match the aspect ratios of the minimum rectangular area R1 and the liquid crystal monitor 14, it is preferable to expand the area including the area around the minimum rectangular area R1. In addition, the minimum rectangular area R1
The aspect ratio may be changed to match the aspect ratio of the liquid crystal monitor 14.

Then, the enlarged minimum rectangular area R1 is displayed on the liquid crystal monitor 14 (step S7). As a result, as shown in FIG. 7, the face area of the person to the left of the designated reference point is enlarged and displayed on the liquid crystal monitor 14. It should be noted that FIG. 7 shows an image enlarged to include a region around the minimum rectangular region R1 so as not to generate a blank space.

In this embodiment, since the faces of three persons are included in the image displayed on the liquid crystal monitor 14 as shown in FIG. 2, the user enlarges the face areas of other persons. May be desired. Therefore, following step S7, it is determined whether or not an instruction to select the next face area is given from the input means 15 (step S8). Step S8
If the result is affirmative, the process returns to step S1 to display the image shown in FIG. 3 on the liquid crystal monitor 14 again, and the processes of steps S2 to S8 are repeated to display another face area in an enlarged manner. If step S8 is denied, the process ends.

As described above, according to the present embodiment, in the image including the human face displayed on the liquid crystal monitor 14, the designation of at least one reference point on the face area is accepted, and the designated reference point is set. Since the face area is extracted from the image as a reference, the face area can be extracted without the trouble of setting a rectangular area including the face area on the image. Further, since the face area is extracted with reference to one designated reference point on the face area, it is not necessary to detect the position of the face area in the calculation for extracting the face area. Regions can be extracted.

Further, by setting the minimum rectangular area R1 including the extracted face area and enlarging the minimum rectangular area R1 so that the minimum rectangular area R1 fits the liquid crystal monitor 14, the face area is expanded and the liquid crystal is displayed. It will be displayed on the monitor 14. For this reason, the enlarged face area does not completely fill the display screen, and as a result, the entire face area can be grasped without scrolling the screen. Therefore, the person included in the displayed image can be easily confirmed.

Next, a second embodiment of the present invention will be described. FIG. 8 is a schematic block diagram showing the configuration of an image display system to which the face area extracting device according to the second embodiment of the present invention is applied. As shown in FIG. 8, the image display system according to the second embodiment includes a display terminal 2 such as a mobile phone, a PDA, a digital camera having a communication function, or a personal computer, and an image server 3 that stores image data S0. The image data S0 is downloaded in the display terminal 2 and the image is displayed on the monitor of the display terminal 2 by communicating between the two via the network 4.

The display terminal 2 includes a monitor 21 and an input means 2.
2, which displays an image on the monitor 21, and downloads image data S0 from the input means 22 and inputs one reference point on the face area. Here, when the display terminal 2 is a mobile phone, a dial button, numeric keys on the left, right, up, and down serve as the input unit 22, and when the display terminal 2 is a PDA, a pen for pen input, a touch panel for touch input,
Various input keys and the like serve as the input means 22. When the display terminal 2 is a personal computer, a keyboard, a mouse or the like serves as the input means 22.

The image server 3 includes a communication IF 31 for communicating via the network 4, a large capacity hard disk 32 for storing the image data S0, and a face area extracting means 33.
And a control means 34 for controlling the communication IF 31 and the face area extraction means 33.

The large-capacity hard disk 32 has 1
It is assumed that high-resolution image data S0 and low-resolution image data S1 are stored for one image.

Next, the operation of the second embodiment will be described. FIG. 9 is a flowchart showing the operation of the second embodiment. First, when the user operating the display terminal 2 gives an instruction to download the desired image data S0 on the display terminal 2 using the input means 22 (step S1).
If 1 is positive, an instruction to download the image data S0 is transmitted to the image server 3 via the network 4 (step S12). The image server 3 receives the instruction and receives the image data S from the large capacity hard disk 32.
The low resolution image data S1 of 0 is transmitted to the display terminal 2 via the network 4 (step S13). The display terminal 2 receives the low-resolution image data S1 and displays it on the monitor 21 (step S14). here,
It is assumed that an image including the faces of three persons is selected as shown in FIG.

Next, as shown in FIG. 4, the user uses the ten keys of the input means 22 to specify one reference point on any face area. This designation is made by moving the pointer P displayed on the monitor 21 by the input means 22 onto a desired face area and determining the position by a predetermined input.

Then, it is judged whether or not the reference point is designated (step S15), and if the result at step S15 is affirmative, the information indicating the position of the designated reference point is transmitted to the image server 3 (step S16). The face area extracting means 33 of the image server 3 extracts the face area based on the designated reference point (step S17). Then, as shown in FIG. 6, the minimum rectangular area R1 including the extracted face area is set (step S18). The processes of steps S17 and S18 are performed in the same manner as the processes of steps S3 and S4 in the first embodiment.

Subsequently, the screen size of the monitor 21 of the display terminal 2 is acquired (step S19). In the present embodiment, the acquisition of the screen size may be performed by prompting the display terminal 2 to input the screen size, and when the display terminal 2 issues an instruction to download the image data S0, the screen size is simultaneously transmitted. It may be acquired in this way.
If the image is displayed in the window instead of the entire screen of the monitor 21, the window size may be transmitted from the display terminal 2 to the image server 3 in place of the screen size.

Then, the minimum rectangular area R1 is enlarged so that the minimum rectangular area R1 matches the screen size of the monitor 21 (step S20). Specifically, the enlargement process is performed by performing interpolation calculation on the high-resolution image data S0 representing the image in the minimum rectangular area R1.
In addition, the image data in the enlarged minimum rectangular area R1 is S
Let L. Then, the image data SL representing the enlarged minimum rectangular area R1 is transmitted to the display terminal 2 (step S2
1), the image represented by the image data SL is the monitor 21
Is displayed (step S22).

Here, also in the second embodiment, as shown in FIG.
Since the face areas of the three persons are included in the image displayed on the monitor 21 as shown in, the user may desire to enlarge the face areas of the other persons. Therefore, following step S22, it is determined whether or not there is an instruction to select the next face area from the input unit 22 (step S2).
3). When the result in step S23 is affirmative, the process returns to step S14, the image shown in FIG. 3 is displayed again on the monitor 21, and the processes in steps S14 to S23 are repeated to enlarge and display another face area. If step S23 is denied, the process ends.

When the enlarged minimum rectangular area R1 is displayed on a digital camera having a communication function, on a monitor of a mobile terminal such as a mobile phone or PDA, or when it is printed out, it is displayed on a personal computer or the like. Compared with the case of playing on a monitor, the image played can be upright by rotating the portable terminal or the recording paper itself, regardless of whether it is vertically long or horizontally long. there is no problem. Therefore, when displaying the enlarged minimum rectangular area on the digital camera, on the monitor of the mobile terminal, or when printing out with the printer, the minimum rectangular area is enlarged so that it is enlarged as much as possible. It is preferable to rotate and expand the minimum rectangular area so that the long-side direction of and the long-side direction of the display screen coincide with each other. When enlarging, the aspect ratio of the minimum rectangular area R1 may be changed according to the display size.

In the first and second embodiments described above, with respect to an image including the faces of a plurality of persons, a reference point on one face area is designated and enlarged, and then on the next face area. Although the reference point of is specified and expanded,
The designation of the reference point may be accepted for a plurality of face areas desired to be enlarged first, and the plurality of face areas may be collectively enlarged. In this case, the image enlarged by the instruction from the input means 15 and 22 is displayed on the liquid crystal monitor 14 or the monitor 21.
It is sufficient to sequentially switch to and display.

Further, in the first and second embodiments, the extracted face area is enlarged and displayed, but the image processing to be performed on the image data S0 is performed based on the color information in the extracted face area. It is also possible to obtain the image processing conditions for the above and perform the image processing on the image data S0 according to the obtained image processing conditions.

[Brief description of drawings]

FIG. 1 is a schematic block diagram showing a configuration of a digital camera including a face area extracting device according to a first embodiment of the present invention.

FIG. 2 is a flowchart showing the operation of the first embodiment.

FIG. 3 is a diagram showing an image including faces of three persons.

FIG. 4 is a diagram showing a state in which a reference point is specified on a face area.

FIG. 5 is a diagram showing a state in which a face area is extracted.

FIG. 6 is a diagram showing a state in which a minimum rectangular area is set.

FIG. 7 is a diagram showing an enlarged face area.

FIG. 8 is a schematic block diagram showing the configuration of an image display system to which a face area extracting device according to a second embodiment of the present invention is applied.

FIG. 9 is a flowchart showing the operation of the second embodiment.

[Explanation of symbols]

1 digital camera 2 display terminals 3 image server 4 network 11 Imaging means 12 memory cards 13 Recording means 14 LCD monitor 15,22 Input means 16, 33 Face area extracting means 17,34 Control means 22 monitors 31 Communication IF 32 large capacity hard disk

Claims (9)

[Claims] 1. An image including a person's face displayed on a monitor receives designation of at least one reference point on a face area, and extracts the face area from the image with the designated reference point as a reference. A face area extracting method characterized by: 2. A minimum rectangular area including the extracted face area is set on the image, a desired display image size is acquired, and the minimum rectangular area matches the display image size. The face area extracting method according to claim 1, wherein the minimum rectangular area is enlarged. 3. The face area extracting method according to claim 2, wherein the enlarged minimum rectangular area is reproduced. 4. An image including a human face displayed on a monitor, input means for accepting designation of at least one reference point on a face area, and the face from the image based on the designated reference point. A face area extracting device comprising: an extracting unit that extracts an area. 5. A setting means for setting a minimum rectangular area including the extracted face area on the image, an acquisition means for acquiring a desired display image size, and the minimum rectangular area being the display image size. The face area extracting apparatus according to claim 4, further comprising an enlarging unit that enlarges the minimum rectangular area so as to fit. 6. The face area extracting device according to claim 5, further comprising reproducing means for reproducing the enlarged minimum rectangular area. 7. A procedure of accepting designation of at least one reference point on a face area in an image including a person's face displayed on a monitor, and the face area from the image based on the designated reference point. A program for causing a computer to execute the procedure for extracting. 8. A procedure of setting a minimum rectangular area including the extracted face area on the image, a step of obtaining a desired display image size, and the minimum rectangular area conforming to the display image size. 8. The program according to claim 7, further comprising the step of enlarging the minimum rectangular area. 9. The program according to claim 8, further comprising a step of reproducing the enlarged minimum rectangular area.