JP2002051316A - Image communication terminal - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an image communication terminal in which the camera section tracks the user's position without using a large scale tracking mechanism and the user can be imaged at a good position. SOLUTION: The image communication terminal comprises a section 7 for extracting the position and size of a face region in an image picked up at a camera section 4, a section 3 for displaying the image to a user, a section 9 performing bi-directional communication of the image with an opposite information processor, and a transmission data processing section 8 for outputting an image in a rectangular transmission region being set shiftably in an image picked up at the camera section 4 to the communicating section 9. An effective region shifting integrally with the transmission region is set in an image picked up at the camera section 4 and only when the face region deviates from the effective region, position of the transmission region is shifted depending on the position and size of the face region.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image communication terminal, and more particularly, to a camera in which a user takes a picture of himself or another person in the vicinity, and transmits the photographed picture to the other party. The present invention relates to an image communication terminal for performing a conversation.

[0002]

2. Description of the Related Art As is well known, various types of image communication terminals, such as a video telephone, a video conference system, and a video mail, which perform a conversation while transmitting an image to a partner, are known. In any of these forms, in order for the user to transmit the figure of himself or another person nearby (hereinafter simply referred to as “user”) to the other party, the user is built in or externally connected to the image communication terminal. The camera unit and the user who is the subject must always be in an appropriate positional relationship.

In order to maintain this proper positional relationship,
A method is conceivable in which a mechanism for moving the optical axis, a zoom mechanism, and the like are provided in the camera unit, and the camera unit follows the movement of the user. However, according to this method, the camera unit and related mechanisms required for the tracking operation become large, and the size and cost of the image communication terminal cannot be reduced. In particular, it is not practical to provide such a mechanism in an image communication terminal such as a mobile terminal or a portable (video) telephone where portability is important.

On the other hand, a method is also conceivable in which information relating to the position of the user with respect to the camera unit is provided from the image communication terminal to the user, and the user adjusts itself to the camera unit so as to maintain the appropriate positional relationship. .

More specifically, as a first technique, a part of a screen is used to show a figure of the user (the user himself) by a picture-in-picture method or a screen division method. I have. However, with this approach,
A significant portion of the screen is occupied to reflect yourself,
As a result, there is a problem that the opponent's figure becomes small and it becomes hard to see. In addition, as a second method, switching between an image of one's own and an image of the other party while displaying the image has been conventionally performed. However, in this method, since the screen is frequently switched, there is a problem that the user is worried about the switching and cannot concentrate on the conversation. In addition, the first
According to both the second and the third techniques, the environment is far away from the normal conversation (conversation between the user and the other party with the knees), and the user cannot prohibit an unnatural feeling.

Therefore, in order to deal with such a problem,
Japanese Patent Application Laid-Open No. Hei 8-251561 discloses a technique that does not display a user's own appearance and can omit a follow-up mechanism of a camera unit. According to the technique disclosed in this publication, a user himself / herself is photographed by a camera unit, the position of the user is detected, and it is determined whether or not the detected position deviates from a photographing range. Then, only in the case of deviation, the user is notified of the deviation by one of the following methods. (1) The appearance of the other party is displayed on almost the entire screen, and when the user deviates, the user is notified by changing the image of the other party (for example, deforming the other party's figure). . (2) A character display area as well as an area for displaying the other party's figure is secured in the screen. And, when it deviates, by displaying the message of departure in the character display area,
Notify the user to that effect.

[0007]

However, in any of the above methods (1) and (2), no notification is given to the user unless the position of the user deviates from the photographing range. Further, when the user uses common sense, the user does not depart from the shooting range so frequently. Therefore, in most cases (that is, when the user does not deviate), the user cannot confirm his / her position with respect to the shooting range.

Further, in the above method (1), when the user deviates, the partner's appearance suddenly changes, and the user may be surprised and interrupt the conversation. Further, in the above method (2), a character display area to a certain extent is required in order to prevent characters (messages) to be displayed from being crushed. For this reason, the image display area is reduced by being pressed by the character display area, and the other party is likely to be small and difficult to see. In addition, in any of the above methods (1) and (2),
The size of the user on the screen is not known at all, and the suitability of the user for the camera unit in the near and far directions is unknown.

SUMMARY OF THE INVENTION It is therefore an object of the present invention to provide an image communication terminal in which a camera section can follow a user's position without using a large-scale follow-up mechanism and can photograph a user at a favorable position. . A further object of the present invention is
It is an object of the present invention to provide an image communication terminal that allows a user to always check his or her image (photographing position) while ensuring a natural conversation in which the other party is easily displayed.

[0010]

Means for Solving the Problems and Effects of the Invention A first invention is an image communication terminal for transmitting an image of a user photographed by a camera unit to a partner, and an input unit for receiving an input from the user. A camera unit for photographing the user, a face extraction unit for extracting the position and size (face area) of the user's face from the image photographed by the camera unit, and a display unit for displaying the image to the user And a communication unit that performs at least image communication with the other party's information processing device, and transmission of a rectangle that is smaller than the area of the image captured by the camera unit and is set to be movable within the area of the image. A transmission data processing unit that outputs an image of the region to the communication unit, and an effective region that moves integrally with the transmission region is set in the region of the image captured by the camera unit. The extracted face area deviates from the effective area When, characterized in that to move the set position of the transmission region in accordance with the position of the face region.

As described above, according to the first aspect, it is determined whether or not the face area has deviated from the effective area. If the face area has deviated from the effective area, transmission is performed in accordance with the position of the face area. Move the position of the area. As a result, the transmission area follows the movement of the face area, and the self-portrait appropriately framed is transmitted to the other party only when the user is at an approximate position without worrying about the degree of reflection. . Moreover, there is no need for a large tracking mechanism such as an optical axis moving unit or a zoom unit of the camera unit, and the portability of the image communication terminal is not impaired. Further, if the face area is within the effective area, the transmission area does not move, so that the image transmitted to the other party, particularly the background image of the user, does not frequently blur, and it is possible to prevent the other party from getting sick.

[0012] A second invention is an invention according to the first invention, wherein the effective area is smaller than the transmission area, and
It is set in the transmission area.

As described above, according to the second aspect, since the face area always deviates from the effective area before deviating from the transmission area, the face area protrudes outside the transmission area and a part of the face is missing. Such a situation can be avoided.

[0014] A third invention is an invention according to the first invention, wherein the transmission data processing unit, when the extracted face area deviates from the effective area, positions the face area at the center of the transmission area. In such a case, the transmission area is moved.

A fourth invention is an invention according to the first invention, wherein the transmission data processing section, when the extracted face area deviates from the effective area, places the face area above the center of the transmission area. The transmission area is moved so as to be located in the direction.

A fifth invention is an invention according to the fourth invention, wherein, when the extracted face area deviates from the effective area, the transmission data processing unit sets the face area to the center or the center of the transmission area. It is characterized in that the transmission area is moved by switching in accordance with the transmission mode information input from the input unit so as to be located further upward.

As described above, according to the third aspect, the face area is moved so as to be located at the center of the transmission area.
Facial framing with face-up can be realized. Also,
According to the fourth aspect, since the face area is moved so as to be located above the center of the transmission area, favorable framing of bust-up can be realized. Further, according to the fifth aspect, the face-up / bust-up framing can be selected according to the user's preference.

A sixth invention is an invention according to the fourth invention, wherein the display unit monitors and displays an image and a face area in the transmission area in accordance with information input from the input unit. The user can adjust the position of the transmission area in the vertical and horizontal directions by inputting to the input unit with reference to the monitor display.

As described above, according to the sixth aspect, the user monitors the image in the transmission area and the face area, and adjusts the position of the transmission area as appropriate, so that the self-portrait can be arbitrarily framed. Can be sent to the other party.

A seventh aspect of the present invention is an image communication terminal for transmitting an image of a user photographed by a camera section to a partner, comprising: an input section for receiving an input from the user; and a camera section for photographing the user. A face extraction unit that extracts the position and size (face area) of the user's face from an image captured by the camera unit;
A display unit for displaying an image to the user, a communication unit for communicating at least the image to the information processing device of the other party, and a region smaller than the region of the image taken by the camera unit and within the region of the image A transmission data processing unit that outputs an image of a rectangular transmission area set to be movable to the communication unit, and an effective area that moves integrally with the transmission area is included in the area of the image captured by the camera unit. Is set, and the transmission data processing unit
When the extracted face area deviates from the effective area, the set position of the transmission area is moved according to the position of the face area,
In addition, based on the image brightness of the extracted face region, the image brightness of the transmission region is corrected and output to the communication unit so that the visibility of the face in the image captured by the camera unit is improved. Features.

An eighth invention is an invention according to the seventh invention, wherein the transmission data processing section corrects the color tone in addition to the image luminance of the transmission area and outputs the corrected color tone to the communication section. .

A ninth aspect of the present invention is an image communication terminal for transmitting an image of a user photographed by a camera unit to a partner, comprising: an input unit for receiving an input from the user; and a camera unit for photographing the user. A face extraction unit that extracts the position and size (face area) of the user's face from an image captured by the camera unit;
A display unit for displaying an image to the user, a communication unit for communicating at least the image to the information processing device of the other party, and a region smaller than the region of the image taken by the camera unit and within the region of the image A transmission data processing unit that outputs an image of a rectangular transmission area set to be movable to the communication unit, and an effective area that moves integrally with the transmission area is included in the area of the image captured by the camera unit. Is set, and the transmission data processing unit
When the extracted face area deviates from the effective area, the set position of the transmission area is moved according to the position of the face area,
In addition, a value of an exposure level of the camera unit is set based on the extracted image brightness of the face region so that the visibility of a face in an image captured by the camera unit is improved.

As described above, according to the seventh to ninth aspects, it is possible to transmit an image in which the user's face is always visible to the other party even in the case of backlight. Thus, it is possible to have a conversation with the other party using the image communication terminal even outdoors, without worrying about the surrounding lighting environment.

A tenth aspect of the present invention is an image communication terminal for transmitting an image of a user photographed by a camera unit to another party. A face extraction unit for extracting the position of the user's face, a display unit for displaying the image received from the other party to the user, and a user's position in the image captured by the camera unit based on the position of the extracted face. A notification control unit that notifies the user of the position of the face and a communication unit that performs at least image communication with the partner information processing device are provided.

As described above, according to the tenth aspect, the user is notified of his / her own position in the photographed image, so that even if his / her figure does not deviate from the screen, his / her own You can proceed with a conversation with the other party with confidence while checking the position of. Even if the user deviates from the screen, the partner's appearance does not suddenly change, so that the user can calm down, return to the correct position while referring to the notification, and continue the conversation. In addition, since there is no need to provide a mechanism for following the user in the camera unit, the weight and power consumption of the image communication terminal can be reduced. For this reason, the present invention can be suitably used for devices in which portability is important, such as mobile (video) phones and mobile terminals.

An eleventh invention is an invention according to the tenth invention, wherein the face extraction unit extracts the size of the face together with the position of the user's face, and the notification control unit captures the image by the camera unit. The user is notified of the position and size of the user's face in the displayed image.

As described above, according to the eleventh aspect, by extracting and notifying the size of the face area, the user can obtain information on both the position and the size of the face area. Therefore, the user refers to this information,
The position on the screen and the position in the perspective direction can be appropriately maintained. In addition, the user can check at what position on the screen and at what size the user is without having to obtain a self-portrait.

A twelfth invention is an invention according to the tenth invention, wherein the notification control unit causes the display unit to display only the position of the extracted face or a mark indicating the position and size. Features.

As described above, according to the twelfth aspect, the user can concentrate on the conversation in the same manner as a normal conversation while watching the other party displayed on the display unit. In addition, the user can confirm his / her position by referring to the simple landmark.

A thirteenth invention is a invention according to the twelfth invention, wherein the mark is displayed on an image received from the other party.

As described above, according to the thirteenth aspect, since the mark appears on the figure of the partner, it is not necessary to secure a wide screen area only for the mark, and the figure of the partner can be made larger and larger. It can be displayed easily. Moreover, the user does not need to change his / her eyes to see the landmarks, and is less tired even after a long conversation.

A fourteenth invention is a invention according to the twelfth invention, wherein the mark is displayed outside an image received from the other party.

As described above, according to the fourteenth aspect, by removing the mark from the image of the partner, the marker does not interfere with the image of the partner, and the partner can be seen more clearly.

A fifteenth invention is a invention according to the twelfth invention, wherein the notification control unit determines the position of the extracted face by:
The notification is provided via a position notification unit provided separately from the display unit.

As described above, according to the fifteenth aspect, by providing the position notification unit separately from the display unit, the entire screen of the display unit can be assigned to the display of the other party's figure. The figure can be displayed more widely and easily.

A sixteenth invention is an invention according to the tenth invention, wherein a method of notifying a user performed by the notification control unit can be switched according to an instruction from the user. .

As described above, according to the sixteenth aspect, the user can select a favorite notification method.

A seventeenth invention is an invention according to the first to sixteenth inventions, wherein the face extracting unit detects an edge portion (corresponding to a contour of a person, a contour of a face, or the like) from an image taken by the camera unit. And an edge extraction unit that generates an image (edge image) of only the edge portion,
A template storage unit that stores a plurality of templates provided concentrically at the center point in various sizes of similar and different sizes, and coordinate positions and voting on the edge image for each size of the template. And a voting result storage unit for storing the numbers of the pixels in association with each other, and sequentially moving the center point of the template to each pixel position of the edge portion, and for each of the moved pixel positions, all the pixels forming the shape of each size. For each coordinate position corresponding to the position, a voting unit that increases or decreases the number of votes stored in the voting result storage unit, respectively, and is included in the target image based on each voting number stored in the voting result storage unit. And an analysis unit for obtaining the position and size of the face to be obtained.

As described above, according to the seventeenth aspect, the position of the face can be detected at high speed only by the voting process (basically only the addition) with a light processing load and the evaluation thereof. In addition, since a template having a plurality of similar and concentric shapes is used, a substantial approximation is made as to which one of these shapes is closer to the edge portion that may include the face. Therefore, the size of the face can be extracted at high speed.
As described above, since the processing load can be greatly reduced, the face can be extracted almost in real time even with the processing capability of the current personal computer level. In addition, the point of the face area in the target image, the number of face areas, etc.
The face can be detected uniformly in a wide range of target images without any problem before extraction, and the versatility is extremely high.

An eighteenth invention is an invention according to the seventeenth invention, wherein the predetermined shape is a circle.

As described above, according to the eighteenth aspect, since the shape group is a circle, the distance from the center point of the template to all the pixels of the shape is always constant, and the accuracy of the voting result is reduced. Can be kept high.

A nineteenth invention is an invention according to the first to sixteenth inventions, wherein the face extracting unit inputs a predetermined template image, obtains an edge normal direction vector of the image, and A template image processing unit that generates an evaluation vector from the normal direction vector, orthogonally transforms the evaluation vector, and an image captured by the camera unit, inputs an edge normal direction vector of the image, and obtains the edge normal An input image processing unit that generates an evaluation vector from the direction vector and orthogonally transforms the evaluation vector, and sums up the corresponding spectral data for each of the orthogonally transformed evaluation vectors generated for each of the template image and the captured image. A product-sum unit for calculating, and an inverse orthogonal transform unit for performing an inverse orthogonal transform on a result of the product-sum calculation to generate a map of similarity values, the evaluation vector Includes corresponding image of edge normal direction vectors and the even double angle convert components, calculation formula of the similarity value, the orthogonal transform and inverse orthogonal transformation, characterized in that both those having linearity.

As described above, according to the nineteenth aspect, the edge normal direction vector of the template image and the edge normal direction vector of the image (input image) shot by the camera unit Even if the sign of the inner product (cos θ) of the angle θ formed by the inversion is inverted, the similarity is not affected and the matching can be evaluated properly.

A twentieth aspect is the invention according to the nineteenth aspect, wherein the face extraction unit calculates the value calculated based on the angle when the edge normal direction vector is expressed in polar coordinates in the expression of the evaluation vector. Is used.

The twenty-first invention is an invention according to the first to sixteenth inventions, wherein the face extraction unit is adapted to determine that the position and size extracted as a face from the image taken by the camera unit are truly true. A face / non-face judging unit for judging whether the face is a face or not is further provided, and the extraction result is output only when the face is judged.

A twenty-second invention is an invention according to the seventeenth invention, wherein the face extraction unit converts the image captured by the camera unit into a face based on the contents stored in the voting result storage unit. A face / non-face judging unit for judging whether or not the extracted position and size are truly a face is further provided, and the extraction result is output only when the face and the non-face are judged to be a face.

A twenty-third aspect is an invention according to the nineteenth aspect, wherein the face extraction unit converts the image captured by the camera unit into a face based on the similarity value generated by the inverse orthogonal transform unit. A face / non-face judging unit for judging whether or not the extracted position and size are truly a face is further provided, and the extraction result is output only when the face and the non-face are judged to be a face.

As described above, according to the twenty-first to twenty-third aspects, a stable face area can be extracted even when the actual face is other than the first face area candidate. Further, even when there is no face in the image, it can be determined that there is no face, so that it is possible to automatically detect a case where it is not necessary to move and display the position of the face.

A twenty-fourth invention is an invention according to the twenty-first invention, wherein the face / non-face determination unit uses an image feature obtained from an area extracted as a face from an image taken by the camera unit. The face / non-face determination is performed based on the determination result of the support vector function.

A twenty-fifth aspect is the invention according to the twenty-fourth aspect, wherein the face / non-face judging section comprises an edge normal direction obtained from an area extracted as a face from an image taken by the camera section. It is characterized in that a vector is an image feature.

A twenty-sixth invention is an invention according to the twenty-fourth invention, wherein the face / non-face judging section judges an edge normal obtained from an area extracted as a face from an image taken by the camera section. It is characterized in that the histogram is used as an image feature.

[0052]

Embodiments of the present invention will be described below with reference to the drawings. (First Embodiment) FIG. 1 is a block diagram showing a configuration of an image communication terminal according to a first embodiment of the present invention. Figure 1
In the image communication terminal according to the first embodiment, the input unit 2, the display unit 3, the camera unit 4, the display control unit 5,
Self-image memory 6, face extraction unit 7, transmission data processing unit 8
, A communication unit 9, a reception data processing unit 10, and a partner image memory 11. First, an outline of each configuration of the image communication terminal according to the first embodiment will be described.

As shown in FIG. 1, in the image communication terminal of the present embodiment, the input unit 2, the display unit 3, and the camera unit 4 face the user 1. The input unit 2 includes a keyboard (including a numeric keypad), a mouse, and the like, and is used by the user 1 to input a transmission mode and other necessary information.
The display unit 3 is configured by an LCD or the like, and displays an image of the other party, a mark according to an instruction of the display control unit 5, and the like to the user 1 on a screen. Although the mark is described in detail later, it is an index by which the user 1 can confirm the position and size of his / her face on the screen. The camera unit 4 includes an optical system such as a lens and C
It is composed of an electric system such as a CD and is used for photographing the user 1. Images taken by the camera unit 4 (hereinafter referred to as
The target image is stored in the self-image memory 6 for each frame. The display control unit 5 controls screen display of the display unit 3 (mainly, display of a received partner image). In addition, the display control unit 5 responds to information input from the input unit 2,
A mark based on the face area extracted by the face extracting unit 7 is displayed on the screen of the display unit 3.

The face extracting unit 7 examines the position and size of the existing face with respect to the target image stored in the self-image memory 6, and uses the information as a face area for the display control unit 5 and the transmission data processing unit 8. Output to Regarding the face extraction unit 7, a method applicable to the present invention will be described later in detail. The transmission data processing unit 8 sets a transmission area according to the position of the face area extracted by the face extraction unit 7. Then, the transmission data processing unit 8 sends out the image data in the transmission area among the target images stored in the own image memory 6 to the communication unit 9 according to the transmission mode instructed from the input unit 2. The communication unit 9 communicates at least image data with a partner information processing device (including an image communication terminal) via a communication path. The communication mode here is arbitrary, and may be, for example, communication between slave units not via a base station such as an extension phone, or synchronous or asynchronous communication via a base station such as a videophone. . The reception data processing unit 10 processes the other party's image data received via the communication unit 9,
It is stored in the partner image memory 11 for each frame.

In the present embodiment, the case where the communication unit 9 performs two-way communication will be described as an example. However, the present invention is also applied to video mail or the like in which the user 1 unidirectionally communicates image data to the other party. Can be applied. In this case, the information processing device of the other party may have only a configuration for receiving the transmitted image data and displaying the image data on the screen.

Next, a tracking process performed by the transmission data processing unit 8 in accordance with the position of the face area will be described with reference to FIGS. First, the relationship between the photographing region 30 of the camera unit 4 and the transmission region 31 of the image transmitted from the communication unit 9 is as follows.
Generally, it is as shown in FIG. The transmission area 31 is a rectangular area smaller than the imaging area 30. The camera unit 4 captures an image of the subject (user 1) in a capturing area 30 wider than the transmission area 31, but only the image in the transmission area 31 is transmitted from the image communication terminal to the other party. In the example of FIG.
Is the length A in the x direction and the length B in the y direction.
Is a length L in the x direction and a length M in the y direction. Also, L <A
And M <B, and the lengths A, B, L, and M are fixed. In the example of FIG. 3, the upper left point (x
1, y1) is used as a reference point. The reference point is movable within the photographing area 30, and the position of the transmission area 31 is uniquely determined by determining the reference point. Note that a point other than the upper left point of the transmission area 31 may be used as a reference.

On the other hand, in the present embodiment, the position and size of the face area extracted by the face extraction unit 7 are represented by circular marks R. The center of the mark R is the center of the face area, and the mark R
Is equivalent to the size of the face area. The mark R is
The shape may be other than a circle.

In the state of FIG. 3, the face area indicated by the mark R deviates to the right of the transmission area 31. Therefore, the mark R
If the transmission area 31 is moved to the right as indicated by an arrow in the figure based on the above, preferable framing is achieved. Therefore, in the present embodiment, the transmission area 31 is moved so that the mark R is included therein. FIG. 4 shows a state (upper left point (x2, y2)) after the transmission area 31 has been moved. Here, in the present embodiment, as shown in FIG. 4, an effective area 32 is further set inside the transmission area 31 so that the effective area 32 and the transmission area 31 move integrally. Then, it is checked whether or not the mark R has deviated from the effective area 32 instead of the transmission area 31. If the mark R has deviated, the transmission area 31 and the effective area 32 are moved as shown in FIGS.

Here, when the effective area 32 is made narrower, the probability that the mark R deviates from the effective area 32 increases, and it becomes easy for the opponent to get sick. Therefore, as shown in FIG. 4, it is desirable to suppress the movement of the transmission area 31 in order to widen the effective area 32. Even in this case, the face area is at a position that is easy to see.

In addition, in the present embodiment, the position of the mark R immediately after the movement of the transmission area 31 can be switched according to the transmission mode (bust-up mode or bust-up mode). FIG. 4 shows an example of an image display method in the bust-up mode in which the mark R is positioned slightly above the transmission area 31 with respect to the center in the x direction and the center in the y direction. In the face-up mode, the mark R is displayed in the transmission area 31.
Is an image display method located at the center in the x direction and the center in the y direction. Furthermore, in the present embodiment, as shown in FIG. 5, it is possible to offset the mark R from these modes in a desired direction. According to this, for example,
It is possible to respond to various requests, for example, when the user 1 wants to show the thing brought with him / her together to the other party.

Next, each process of the follow-up processing performed by the transmission data processing section 8 will be described with reference to FIG. First, the user 1 inputs a transmission mode (bust-up mode / face-up mode) from the input unit 2 (step S201).
Next, the user 1 is photographed by the camera unit 4 and stored in the self-portrait memory 6 as a target image (step S20).
2). At the time of this shooting, the user 1 has a large shooting area 3
It is sufficient to be at a position where the face is captured within 0. next,
The face extraction unit 7 extracts a face area (face position and size) in the target image, and outputs the extracted face area to the transmission data processing unit 8 (Step S203).

When the face area is extracted, the transmission data processing unit 8 adjusts the transmission area 31 to the face area according to the transmission mode (step S204). Specifically, as shown in FIG. 4, the upper left point of the transmission area 31 is determined so that the face area is included in the transmission area 31. Next, an effective area 32 is set in the transmission area 31 (step S205), and the image in the transmission area 31 of FIG.
Is displayed on the monitor (step S206). In step S206, the image display of the user 1 may be omitted, and only the mark R may be displayed. Next, user 1
The framing displayed on the monitor is input using the input unit 2 (whether the transmission area 31 is locked) or not (step S207). When the user 1 desires the offset of the transmission area 31, the input unit 2 adjusts the position of the transmission area 31 by accepting the input of the movement information (step S215). Thereafter, the process returns to step S205, and asks for confirmation of user 1 again.

When framing is completed in step S207, image communication with the other party is started (step S207).
208). Note that an appropriate interrupt processing unit may be provided so that the processes of steps S201 to S207 can be performed even during communication. When the communication is started, the communication unit 9
And the other party's image memory 1 via the reception data processing unit 10
1 is displayed on the screen of the display unit 3 (step S209). Here again, the camera unit 4
Captures the user 1 (step S210), and extracts the face
Extracts a face area (step S211), and the transmission data processing unit 8 checks whether the face area has deviated from the effective area 32 (step S212).

Here, if it deviates as shown in FIG. 6, the transmission data processing unit 8 moves the upper left point of the transmission area 31 in accordance with the transmission mode as in step S204 (step S213). It is again checked whether the face area extracted again by the face extraction unit 7 has deviated from the effective area 32 (steps S211 and S212). on the other hand,
If there is no deviation, the transmission data processing unit 8 continues the communication without moving the transmission area 31. If the user wants to communicate with peace of mind while confirming his or her image quality, the user's own image may be displayed on the screen together with the image of the other party using, for example, a picture-in-picture method. Then, steps S209 to S21
3 is repeated until the communication ends (step S2).
14).

As described above, according to the image communication terminal according to the first embodiment of the present invention, without using a large-scale follow-up mechanism, without substantially impairing the portability of the image communication terminal, the Shooting and image communication can be performed following the movement of the user. In other words, the user is taken with preferable framing without worrying about the degree of reflection, and the self-portrait is transmitted to the other party. Further, if the face area is within the effective area, the transmission area does not move, so that the image transmitted to the other party, especially the background image on the user side, does not blur frequently, and it is possible to prevent the other party from getting sick. .

As is well known, some cameras used for the camera unit 4 have a function of automatic exposure correction. Automatic exposure correction is a function that automatically corrects the brightness of an image so that the brightness is optimal, and generally changes the brightness of each pixel in an image based on the entire image or the average brightness of several points. Done in However, when the average luminance of the face area is lower than the average luminance of the entire target image as in the case of backlight or the like, there remains a problem that the face of the user 1 becomes black. Therefore, as a countermeasure in such a case, the transmission data processing unit 8 improves the visibility of the face based on the brightness of the target image captured by the camera unit 4 based on the face area extracted by the face extraction unit 7. After correcting the luminance as described above, the data may be transmitted to the communication unit 9.

More specifically, the transmission data processing section 8 stores in advance the ideal value of the average luminance inside the face area (ideal average luminance a). Then, the transmission data processing unit 8 obtains the average luminance I inside the face area extracted by the face extraction unit 7 and converts the luminance Y1 of the target image captured by the camera unit 4 into a new luminance Y2.
, For each pixel of the target image, Y2 = Y
1 × (a / I) is applied. This makes it possible to correct the inside of the face area to have the ideal average luminance a. Also,
Using this ideal average luminance a, not only luminance but also hue may be similarly changed. In addition, in addition to this, the transmission data processing unit 8 determines that the average luminance I
In such a case, the face area may have an exposure level of the camera unit 4 to be set such that the face area has the ideal average luminance a. In this case, the transmission data processing unit 8 notifies the camera unit 4 of the exposure level with respect to the average luminance I in the face area, so that the brightness of the face area can be corrected to the ideal value. Become.

In this way, even in the case of backlight, it is possible to transmit an image in which the face of the user 1 is always visible to the other party. Thus, it is possible to have a conversation with the other party using the image communication terminal even outdoors, without worrying about the surrounding lighting environment.

(Second Embodiment) In the first embodiment described above, the image communication terminal automatically adjusts to the user's movement by using a simple follow-up mechanism. A method for transmitting a proper image to the other party has been described. Next, in the second embodiment, an appropriate image in which the user is captured in a frame is displayed by displaying the image so that the user can move in accordance with the image communication terminal without using the following mechanism. A method that can be transmitted to the server will be described.

FIG. 7 is a block diagram showing a configuration of an image communication terminal according to the second embodiment of the present invention. 7, the image communication terminal according to the second embodiment includes an input unit 22.
, Display unit 3, camera unit 4, display control unit 25, self-portrait memory 6, face extraction unit 7, transmission data processing unit 8
, A communication unit 9, a reception data processing unit 10, and a partner image memory 11. First, an outline of each configuration of the image communication terminal according to the second embodiment will be described.

As shown in FIG. 7, in the image communication terminal of this embodiment, the input unit 22, the display unit 3, and the camera unit 4 face the user 1. The input unit 22 includes a keyboard (including numeric keys) and a mouse, and is used by the user 1 to input a notification mode, a transmission mode, and other necessary information. In the present embodiment, the input unit 22 includes a numeric keypad that can be turned on (or blinked). The display unit 3 is configured by an LCD or the like, and displays an image of the other party, a mark according to an instruction of the display control unit 25, and the like to the user 1 on a screen. The landmarks will be described in detail later.
Is an index by which the position and size of one's face on the screen can be confirmed. Note that the input unit 22 and the display unit 3
A notification unit 12 is configured to notify the user 1 of the position and size of the face of the user 1 in the image transmitted to the other party.
The camera unit 4 includes an optical system such as a lens and an electric system such as a CCD, and is used for photographing the user 1. The image (target image) captured by the camera unit 4 is stored in the self-image memory 6 for each frame. Display control unit 25
Controls the screen display of the display unit 3 (mainly, the display of the received partner image). Further, the display control unit 25 displays a mark on the screen of the display unit 3 based on the face area extracted by the face extraction unit 7 according to the notification mode input from the input unit 22, Or turn on the numeric keypad.

The face extraction unit 7 examines the position and size of the existing face with respect to the target image stored in the self-image memory 6, and uses the information as a face area as the display control unit 25 and the transmission data processing unit 8 Output to Regarding the face extraction unit 7, a method applicable to the present invention will be described later in detail. The transmission data processing unit 8 sends the target image stored in the self-image memory 6 to the communication unit 9 as it is or after performing the processing described below in accordance with the transmission mode instructed from the input unit 22. The communication unit 9 communicates at least image data with a partner information processing device (including an image communication terminal) via a communication path. The communication mode here is arbitrary,
For example, communication between slave units not via a base station such as an extension telephone may be used, or synchronous communication or asynchronous communication via a base station or the like such as a videophone may be used. The reception data processing unit 10 processes the other party's image data received via the communication unit 9 and stores the data in the other party's image memory 11 for each frame.

Next, an example of a mark displayed on the screen of the display unit 3 by the display control unit 25 will be described with reference to FIGS. Note that these examples can be used in appropriate combination. First, FIG. 8A to FIG.
This is an example in which only the face position (here, the center of the face area extracted by the face extraction unit 7) is displayed as a mark R on the screen of the display unit 3. An area indicated by a rectangle in the figure is a screen of the display unit 3, on which an image of the other party is displayed. FIG.
In (c), the mark R is displayed superimposed on the image of the other party. In FIG. 8D, the mark R is displayed outside the image of the other party. The display of these marks R may be updated in synchronization with the frame of the image of the other party, or may be updated asynchronously. In FIG. 8A, a cross line is used as the mark R, and the intersection of the lines indicates the position of the user 1's face.
FIG. 8B illustrates an example in which an arrow is used as the mark R, and a point specified by both arrows indicates the position of the face of the user 1. FIG. 8C is a diagram in which a figure of a cross or an X mark is used as the mark R, and the position of the figure indicates the position of the face. FIG. 8D shows a case where the vertical and horizontal rulers displayed outside the frame of the image of the other party are used as the mark R, and the point specified by the mark on the vertical ruler and the mark on the horizontal ruler is determined by the user. 1 shows the position of the face.

Next, FIGS. 9A to 9C show the position and size of the face of the user 1 (the entire face area extracted by the face extracting unit 7) on the screen of the display unit 3. This is an example of displaying with R. In FIG. 9A, two vertical and horizontal parallel lines are used as the mark R, and a rectangular area surrounded by the parallel lines indicates the position and the size of the face of the user 1. FIG.
(B) uses a vertical and horizontal ruler displayed outside the frame of the image of the other party as a mark R, and an area specified by a width mark on the vertical ruler and a width mark on the horizontal ruler is determined. The position and size of the face of the user 1 are shown. In FIG. 9C, a circle (or an ellipse) approximating the face area is used as the mark R, and the circle area indicates the position and size of the face of the user 1.

Note that these marks R may be displayed without depending on the image of the other party, or may be displayed depending on them. As the former, for example, the mark R is displayed in a predetermined color (eg, black) regardless of the image of the other party.
As the latter, for example, when the mark R to be displayed is difficult to recognize on the image of the other party, changing the luminance of the pixel displaying the mark R or changing (inverting) the RGB value thereof. In any case, it is desirable to display these marks R so as not to disturb the image of the other party.

FIG. 10 shows an example in which the approximate position of the face of the user 1 is displayed not by the display unit 3 but by the input unit 22. As shown in FIG. 10, a ten-key that can be turned on is used as the mark R, and the user 1 can be notified of the position of the face by turning on one of the ten-keys. In FIG. 10, since the key “3” is lit,
It can notify that the position of the face is "upper right" of the screen. Similarly, the key “1” is “upper left” on the screen, the key “5” is “middle” on the screen, and the key “9” is “lower right” on the screen.
Thus, a rough position display can be performed. Even such a notification of the approximate position is sufficiently practical.

In the present embodiment, the notification modes given from the input unit 22 to the display control unit 25 depend on the notification modes shown in FIGS.
It is possible to switch which method of FIG. 10 notifies the position of the face. Further, these notifications may be made all the time, or may be made only when the user 1 instructs the notification with the input unit 22. As a method of notifying the approximate position, a sound or light may be used instead of turning on the numeric keypad of the input unit 22 shown in FIG. For example, in the case of sound, the interval or frequency emitted from the speaker may be changed according to the position of the face, and in the case of light, the brightness or the interval of blinking may be changed according to the position of the face. Conceivable.

Next, an example of an image of the user 1 transmitted by the transmission data processing unit 8 via the communication unit 9 will be described with reference to FIG. In the present embodiment, an image to be transmitted to the other party can be selected by a transmission mode provided from the input unit 22 to the transmission data processing unit 8. Here, on the user 1 side (self side),
It is assumed that a mark R (a combination of FIGS. 9A to 9C) as shown in FIG. 11A is displayed. At this time, the transmission data processing unit 8 can transmit its own image to the other party in various modes depending on the transmission mode. For example, if the transmission mode is “normal”, the transmission data processing unit 8 transmits the image acquired by the camera unit 4 as it is, as shown in FIG. If the transmission mode is "marked",
As shown in FIG. 11C, the transmission data processing unit 8 refers to the face area extracted by the face extraction unit 7, creates a self-portrait in which the obtained image is combined with the mark R, and transmits the self-portrait to the other party. Further, if the transmission mode is “face only”, as shown in FIG. 11D, the transmission data processing unit 8 converts the own image obtained by cutting out only the face area extracted by the face extraction unit 7 from the acquired image,
Send to the other party.

The image processing based on the transmission mode is as follows.
Since it can be easily realized by a well-known technique, a detailed description thereof will be omitted. Here, if the self-image is transmitted with “marks” as shown in FIG. 11C, even if an image (for example, an image in the dark) where it is difficult to understand where the user is, is transmitted to the other party. It is possible to accurately grasp the user's own position. In addition, as shown in FIG. 11D, if the self-portrait is transmitted with “face only”, the background is not reflected, so that a part that the other party does not want to see can be hidden, and privacy can be protected. Note that even if the background is hidden in this way, the facial expression and the like are transmitted to the other party, so that there is no problem in conversation. Note that the transmission modes described above can be based on any other discriminating method as long as they are unique to each other.

As described above, according to the image communication terminal according to the second embodiment of the present invention, the positional relationship on the screen of the user is simply and appropriately expressed using the mark based on the extracted face area. can do. Therefore, the user can confirm the position of his / her face without worrying not only when the position of his / her face deviates from the screen but also when the position of his / her face does not deviate from the screen. Conversation with the other party can be advanced. Further, since the follow-up mechanism is omitted as compared with the first embodiment, the portability of the image communication terminal can be improved.

(Detailed Example of Face Extraction Unit 7) Next, a specific example of the face extraction unit 7 applicable to the image communication terminals according to the above-described first and second embodiments of the present invention will be described. Three ways will be described. In addition to the three methods described below, the face extraction unit 7 uses various well-known methods such as a method based on color information, a method that focuses on a face portion such as eyes and mouth, and a method based on template matching. Can be applied.

<Embodiment 1> FIG. 12 is a block diagram showing the structure of the face extraction unit 7 of Embodiment 1. In FIG.
The face extraction unit 7 includes an edge extraction unit 51, a template storage unit 52, a voting result storage unit 53, a voting unit 54, and an analysis unit 55.

The edge extracting section 51 extracts an edge portion from the target image captured by the camera section 4 and generates an image of only the edge portion (hereinafter referred to as an edge image). Here, the edge part is a part (pixel) corresponding to the outline of a person, the outline of a face, or the like, and is a part that becomes a high-frequency component in a target image. For the edge extraction unit 51, it is preferable to use a Sobel filter or the like that extracts a high-frequency component from the target image.

The template storage unit 52 stores data of a plurality of templates provided concentrically at the center point in various shapes having similar shapes and different sizes. The shape of this template includes a circle,
An ellipse, regular polygon, polygon, or the like can be used, but it is most preferable to use a circle in which the distance from the center point to the shape line (each pixel forming the shape) is always constant. Thereby, the accuracy of a voting result described later can be increased. Hereinafter, in the first embodiment, as shown in FIG. 13, a case will be described in which a template having a plurality of concentric circles having a center point P and different radii is used. Here, the plurality of circles t1 to tn (n is an arbitrary integer) constituting the template may have a configuration in which the radius changes at regular intervals as in the template shown in FIG. A configuration in which the radius changes may be used. In addition, the plurality of circles t1 to tn forming the template may have a line width of one dot (corresponding to one pixel of the target image), or a part or all of the line width of two dots or more ( That is, it may be configured in a ring shape). In the following description, circles and rings are collectively referred to simply as “circles”.

The plurality of circles t1 to tn are collectively treated as one template and
However, in the actual processing, the circles t1 to tn constituting the template are handled independently.
For this reason, pixel data forming each of the circles t1 to tn is stored in the template storage unit 52, for example, in a table format.

In the voting result storage unit 53, an area for storing the result of a voting process performed in the voting unit 54 described below (hereinafter referred to as a voting storage area) forms a template stored in the template storage unit 52. It is provided for each shape of each size. In this example, since the shape of each size is a circle t1 to tn, the voting result storage unit 53 is provided with n voting storage areas for the circles t1 to tn. This voting storage area has a range corresponding to the target image.

The voting section 54 performs a voting process on the edge image generated by the edge extracting section 51 by using the template stored in the template storage section 52.
FIG. 14 is a flowchart illustrating a procedure of the voting process performed by the voting unit 54. Referring to FIG. 14, voting unit 54
First, the voting result storage unit 53 is accessed, and components (voting values) representing the coordinates in each voting storage area are all initialized to zero (step S601). Next, the voting unit 54
The center point P of the template is set at the top pixel position of the edge portion in the edge image (step S602). The top pixel position may be, for example, the position of the pixel of the edge portion detected first by sequentially scanning the edge image from the upper left to the upper right or the lower left.

Next, the voting unit 54 initializes a counter i for specifying a shape (a circle t1 to tn in this example) constituting the template to “1” (step S603).
Next, for the circle t1 specified by the counter i (= 1), the voting unit 54 acquires the xy coordinates of all the pixels forming the circle t1 on the edge image (step S6).
04). Then, the voting unit 54 performs voting by adding “1” to each of the acquired components representing the xy coordinates in the voting storage area for the circle t1 provided in the voting result storage unit 53 (step S605). When this process ends, the voting unit 54 increments the counter i by one to set i = 2 (step S607). Next, the voting unit 54 determines the circle t2 specified by the counter i (= 2).
For x on the edge image of all pixels forming the circle t2
The y coordinate is obtained (step S604). Then, the voting unit 54 performs voting by adding “1” to each of the acquired components representing the xy coordinates in the voting storage area for the circle t2 provided in the voting result storage unit 53 (step S605).

Similarly, the voting unit 54 increments the counter i one by one until i = n (steps S606 and S607), and repeats the above for the circles t3 to tn, which are all the shapes forming the template. The voting process of steps S604 and S605 is repeated. As a result, the voting process at the head pixel position is performed on each of the voting storage areas for each of the circles t1 to tn. Further, the voting unit 54 sets the center point P of the template at the pixel position next to the edge portion, and repeatedly performs the processing of steps S603 to S607.
The process is performed once for all the pixels at the edge portion in the edge image (steps S608 and S609). That is, the voting process performed by the voting unit 54 is performed such that the center point P of the template crawls all pixels at the edge portion.

For example, by performing the voting process on the edge image shown in FIG. 15, the number of votes as shown in FIG. 16 is stored in the n voting storage areas provided in the voting result storage unit 53. . FIG. 16 shows a case where the voting process is performed at a part of the pixel position of the edge portion in order to make the drawing easy to see. In FIG. 16, the solid circle corresponds to the coordinate component voted based on the shape of each size of the template (circles t1 to tn) in step S605, and the number of coordinates is “1”. Further, as described above, since the number of votes is cumulatively added, the portion where the solid circles intersect in FIG. 16 (indicated by a black circle in the figure) indicates that the greater the number of intersections, the higher the number of votes. I have.

Therefore, if the above-mentioned voting process is applied to the edge portion representing the contour of a face approximated to a circle or an ellipse having a center point, a high number of votes will be concentrated near the center point. Therefore, by determining a portion where high voting values are concentrated, the center of the face can be specified. Further, such a phenomenon in which high voting values are concentrated appears more remarkably when a circular shape having a radius equal to or very close to the minimum width of the edge representing the face contour is used in the template. . Therefore, it is possible to specify the size of the face by determining in which circular voting storage area this phenomenon is noticeable. This point is similar to the generalized Hough transform. However, the face extraction method of the present invention is different from the generalized Hough transform in that a template having concentric shapes of a plurality of sizes can be used to specify the size together with the center point of the edge at a time. Different.

In step S601, the components representing the coordinates in each voting storage area are all initialized to a predetermined maximum value. In step S605, the components representing the xy coordinates obtained are each set to "1". May be subtracted for voting. In this case, it is possible to identify the center of the face by determining the portion where the low voting value is concentrated, and to determine in which circular voting storage area this concentration phenomenon is prominent, Can be specified. In step S605, the value by which the number of votes is added or subtracted may be other than “1”, and the value can be set freely.

Next, a method for specifying the face area of the target image based on the voting result stored in the voting result storage unit 53 will be described. After the voting process by the voting unit 54 is completed, the analysis unit 55 evaluates the cluster based on the voting result stored in the voting result storage unit 53 and determines the position and size of the face included in the target image. Ask. FIG.
9 is a flowchart illustrating a procedure of an analysis process performed in Step 5.

Referring to FIG. 17, analysis section 55 first
Shapes that make up the template (in this example, circles t1 to t
The counter j for specifying n) is set to "1" (step S701). Next, the analysis unit 55 sets the counter j
For the circle t1 specified by (= 1), the number of votes is determined by a predetermined threshold G (for example, 200) with reference to the voting result stored in the voting storage area related to the circle t1 in the voting result storage unit 53. And the like are extracted (step S702). The threshold value G can be arbitrarily determined based on the definition of the target image and the desired extraction accuracy. Next, the analysis unit 55 performs clustering on only the extracted components (step S703),
The variance value and the covariance value of each of the clustered regions are calculated (step S704). The similarity in this clustering may be determined using any of the Euclidean square distance, the standardized Euclidean square distance, the Mahalanobis' generalized distance, or the Minkowski distance. Also,
To form a cluster, the shortest distance method (SLINK: single
linkage clustering method, longest distance method (CLIN)
K: complete linkage clustering method or group averaging method (UPGMA: unweighted pair-group method using
arithmetic averages) may be used.

Next, the analysis unit 55 compares the variance value and covariance value of each of the clustered areas with a predetermined threshold value H (step S705). If each value is less than the threshold value H in step S705, the analysis unit 55 regards the center point of the area as the center point of the face and determines the center point of the circle t1 indicated by the counter j (= 1) at this time. The size (diameter) is set as the minor axis length of the face (step S706), and the length obtained by adding a fixed value (determined empirically) to the minor axis length is determined as the major axis length of the face (step S707). Then, the analysis unit 55 calculates the determined center point, the short axis length and the long axis length,
The result is stored as an analysis result (step S708). on the other hand,
If each value is equal to or greater than the threshold value H in step S705, the analysis unit 55 determines that the center point of the area is not the center point of the face, and proceeds to the next process.

When this process is completed, the analyzing unit 55 increments the counter j by one to j = 2 (step S710). Next, the analysis unit 55 sets the counter j (=
With respect to the circle t2 specified by 2), with reference to the voting result stored in the voting storage area related to the circle t2 in the voting result storage unit 53, only the component whose number of votes exceeds a predetermined threshold G is extracted. (Step S702). Next, the analysis unit 55 performs clustering only on the extracted components (step S703), and calculates the variance value and the covariance value of each of the clustered regions (step S704). Next, the analysis unit 55 compares the variance value and the covariance value of each of the clustered regions with a predetermined threshold value H (Step S705). If each value is less than the threshold value H in step S705,
The analyzing unit 55 regards the center point of the area as the center point of the face, sets the size of the circle t2 indicated by the counter j (= 2) at this time as the short axis length of the face (step S706), and Is determined as the major axis length of the face (step S707). Then, the analysis unit 55 additionally holds the determined center point, short axis length, and long axis length as an analysis result (step S708). On the other hand, step S
If each value is equal to or greater than the threshold value H in 705, the analysis unit 5
No. 5 judges that the center point of the area is not the center point of the face, and proceeds to the next processing.

In the same manner, the analysis unit 55 increments the counter j by one until j = n (steps S709 and S710), and the voting result storage unit 53
The analysis processing of steps S702 to S708 is repeatedly performed on the voting storage area for each of the circles t3 to tn stored in. Thereby, the analysis result of the face area extraction in the voting storage area for each of the circles t1 to tn can be obtained. This analysis result is output to the display control units 5 and 25 and the transmission data processing unit 8.

As described above, in the face extracting unit 7 of the first embodiment,
The face position can be extracted at high speed only by the light voting process (basically only the addition process) and the evaluation of the number of votes. In addition, since a template having similar and concentric shapes of a plurality of sizes is used, a substantial approximation is made as to which edge of the face area is closer to which of these shapes. , And the size of the face can be extracted at high speed.

<Embodiment 2> Next, as Embodiment 2, the amount of processing is reduced by performing pattern matching in the space after the orthogonal transformation, and a limited amount of processing such as that of a mobile phone is required. A method effective in the terminal will be described.
FIG. 18 is a block diagram illustrating a configuration of the face extraction unit 7 according to the second embodiment. In FIG. 18, the face extraction unit 7 includes a template image processing unit 80, an input image processing unit 90,
01, an inverse orthogonal transform unit (inverse FFT) 102, and a map processing unit 103. According to the method of the second embodiment, in the template image processing unit 80 and the input image processing unit 90, the template image and the input image (target image) are respectively subjected to orthogonal transformation having linearity, and after integrating them, the inverse orthogonal transformation is performed. Thus, a similarity value L is obtained.

Here, in the second embodiment, F
The case where FT (fast discrete Fourier transform) is used will be described, but Hartley transform, number-theoretic transform, and the like can also be used. When using these other conversion methods, the part described as “Fourier Transform” in the following description,
What is necessary is just to read these conversion methods. In both the template image processing unit 80 and the input image processing unit 90, the inner product of the edge normal direction vector is used, and the closer the direction of the edge normal direction vector is, the higher the correlation is obtained. Moreover, this inner product is evaluated using an even-numbered double-width representation. In the following, for the sake of simplicity, the case of a double angle will be described as an example of an even number double angle. However, the same effect as that of the second embodiment can be obtained in other even number doubles such as a quadruple angle and a 6 times angle.

First, the template image processing section 80 will be described. In FIG. 18, a template image processing unit 80 includes an edge extraction unit 81 and an evaluation vector generation unit 82
, An orthogonal transform unit (FFT) 83, a compression unit 84, and a recording unit 85.

The edge extracting section 81 performs differentiation processing (edge extraction) on the input template image in each of the x direction and the y direction, and outputs an edge normal direction vector of the template image. In the second embodiment,
In the x direction,

(Equation 1) Using the Sobel filter

(Equation 2) Sobel filter is used. From these filters (1) and (2), an edge normal direction vector of the template image defined by the following equation (3) is obtained.

(Equation 3)

The evaluation vector generation unit 82 receives the edge normal direction vector of the template image from the edge extraction unit 81, performs the following processing, and outputs the evaluation vector of the template image to the orthogonal transformation unit 83. First, the evaluation vector generation unit 82 normalizes the edge normal direction vector of the template image with respect to the length using the following equation (4).

(Equation 4) This is because when the imaging conditions such as illumination fluctuations change, the strength (length) of the edge is easily affected, but the angle of the edge is hardly affected. Thus, in the second embodiment, as described later, the input image processing unit 90 normalizes the edge normal direction vector of the target image to the length “1”. At the same time, the template image processing unit 80 also normalizes the edge normal direction vector of the template image to the length “1”. Further, as is well known, the double angle formula of the following equation (5) holds for the trigonometric function.

(Equation 5) Using this double-angle formula, the edge vector is normalized based on the following equation (6).

(Equation 6)

Hereinafter, equation (6) will be described. First, the constant a is a threshold value for removing minute edges,
The reason why the vector smaller than the constant a is set to the zero vector is to remove noise and the like. Next, a description will be given of a point that each xy component is a dependent function of cosine and sine related to a double angle of each xy component of Expression (4). Here, if the angle between the evaluation vector T of the template and the evaluation vector I of the target image is θ, and the inner product thereof, that is, cos θ, is used as a similarity scale, the following problem occurs. For example, if the template image is as shown in FIG.
It is assumed that the target image is as shown in FIG. Here, in the image of the background portion in FIG. 19B, the left half is brighter than the target, and the right half is darker than the target.
Looking at only the image, when the center of the template image in FIG. 19A coincides with the center of the target image in FIG. 19B, the object completely matches. There must be. Then, assuming that the edge normal direction vector is positive for the outward direction from the image of the target object, the same direction (outward / inward direction) as viewed from the target object in both the bright background portion and the dark background portion in FIG. Orientation). However, at this time, if the luminance of the background portion in FIG.
As shown by the arrow in FIG. 9B, the directions are opposite (outside of the object in a bright background portion, inward of the object in a dark background portion). In such a case, when the maximum similarity value is supposed to be the same, the similarity value does not always become a high value, and misrecognition is likely to occur.

The above points will be described in more detail with reference to FIG. When the inner product cos θ of the angle θ between the evaluation vector T of the template image and the evaluation vector I of the target image is used as a similar value, as described above, the luminance variation of the background image around the target object causes The direction of the evaluation vector can be either the I direction or the diametrically opposite I ′ direction. For this reason, there are two possible inner products that are the similarity scales, cos θ and cos θ ′. Moreover, θ + θ ′ = π, and cos θ
= Cos (π-θ ') =-cosθ. That is, c
When osθ is used as a similarity measure, the similarity value may be reduced in a case where the function should originally act to increase the similarity value. In addition, when it is necessary to operate to reduce the similar value, the similar value may be increased.

Therefore, in the second embodiment, the cosine (cos2θ) of the double angle of θ is used in the expression of the similar value. In this case, even if cos θ ′ = − cos θ, cos 2θ ′ = cos 2θ from the double angle formula of Expression (5). That is, when it is necessary to act to increase the similarity value, the similarity value becomes high without being affected by the background portion. Therefore, even if the image of the background portion has a luminance variation, it is possible to properly evaluate the image matching.
The above points are similarly established not only in the case of the double angle, but also in the case of the quadruple angle, the 6 times angle and the like. By this, even number double-width evaluation,
A pattern can be stably extracted regardless of the luminance condition of the background. In addition to this expression, here, the value of θ expressed as cos θ = Tx, sin θ = Ty from the combination of the values of Tx and Ty (that is, the phase angle when the edge normal direction vector is expressed in polar coordinates) , It is also possible to represent not one of Tx and Ty but one value. In addition, when θ is not 0 to 360 degrees but is represented by, for example, 8 bits, and a negative value is represented by a binary number as a two's complement representation (that is, −128 to 127),
When 1 is added to 128, it becomes 0, which is a cyclic expression. For this reason, in the double angle calculation and the similarity value calculation regarding θ, when it exceeds 127, the process of setting to −128 is automatically performed.

Next, the calculation of the similarity value will be described. More specifically, in the second embodiment, the similarity value L is defined by the following equation (7).

(Equation 7) Note that the evaluation vectors are (Vx, Vy), (Tx, Ty)
However, when Vθ and Tθ are used, the following equation (8) is obtained.

(Equation 8) Here, the case where the evaluation vector has one element is also referred to as a vector.

Here, since the equations (7) and (8) consist only of addition and integration, the similarity value L is linear for each evaluation vector of the target image and the template image. Therefore, when the equations (7) and (8) are Fourier-transformed, according to the discrete correlation theorem of the Fourier transform,

(Equation 9)

(Equation 10) Becomes Note that in equations (9) and (10),
“〜” Indicates a Fourier transform value, and “*” indicates a complex conjugate.

Further, if the equation (9) or (10) is subjected to inverse Fourier transform, a similar value L of the equation (7) or (8) can be obtained. Then, the following two points become clear from Expressions (9) and (10). 1. In the transformed value after the orthogonal transformation, the Fourier transform value of the template image and the Fourier transform value of the target image may be simply summed up. 2. It is not necessary to calculate the Fourier transform value of the template image and the Fourier transform value of the target image at the same time, and the Fourier transform value of the template image may be calculated prior to the Fourier transform value of the target image. .

Therefore, in the second embodiment, the recording unit 85 is provided in the template image processing unit 80, and the output of the compression unit 84 is stored before the input of the target image. Thus, after the target image is input to the input image processing unit 90, the template image processing unit 80 does not need to perform any processing on the template image. Therefore, the processing capability of the image communication terminal is reduced by the input image processing unit 90 and the integrating unit 1.
It is possible to concentrate on the processing subsequent to 01, and the processing can be further speeded up.

Next, a configuration subsequent to the evaluation vector generation unit 82 will be described. As shown in FIG. 18, in the template image processing unit 80, the evaluation vector of the template image output from the evaluation vector generation unit 82 is Fourier-transformed by the orthogonal transformation unit 83, and is output to the compression unit 84. The compression unit 84 reduces the evaluation vector after the Fourier transform and stores it in the recording unit 85. As shown in FIG. 21, the evaluation vector after the conversion includes various high and low frequency components in both the xy directions. According to experiments performed by the present inventors, sufficient accuracy can be obtained by performing processing on low-frequency components (for example, halves on the low frequency side in both xy directions) without performing processing on all frequency components. I know that. In FIG. 21,
Areas not hatched (-a≤x≤a, -b≤y≤
b) is the original area, and the hatched area (-a / 2 ≦
(x ≦ a / 2, −b / 2 ≦ y ≦ b / 2) is the region after the reduction. That is, the processing amount becomes 1/4. In this way, the number of processing targets can be reduced, and higher-speed processing can be realized. Note that the compression unit 84 and the recording unit 85 can be omitted when the data amount is small or when high speed is not required.

Next, the input image processing section 90 will be described. 18, the input image processing unit 90 includes an edge extraction unit 91, an evaluation vector generation unit 92, an orthogonal transformation unit (FFT) 93, and a compression unit 94. The input image processing unit 90 performs the same processing as the template image processing unit 80. That is, the edge extraction unit 91 outputs the edge normal direction vector of the target image using Expressions (1) and (2). Further, the evaluation vector generating unit 92 receives the edge normal direction vector of the target image from the edge extracting unit 91, performs the same processing as the evaluation vector generating unit 82 of the template image processing unit 80, and generates an evaluation vector. The evaluation vector of the target image output from the evaluation vector generation unit 92 is Fourier-transformed by the orthogonal transformation unit 93 and is output to the compression unit 94. The compression unit 94 reduces the evaluation vector after the Fourier transform and outputs the result to the integration unit 101. Here, the compression unit 94 reduces processing targets to the same frequency band as the compression unit 84 of the template image processing unit 80.

Next, a description will be given of the integrating unit 101 and subsequent units. When the processing of the template image processing unit 80 and the input image processing unit 90 is completed, the accumulation unit 101
4, the Fourier transform values of the respective evaluation vectors of the template image and the target image are input. Therefore, the integrating unit 10
1 performs a product-sum operation by Expression (9) or Expression (10),
The result (Fourier transform value of similarity value L) is converted to inverse orthogonal transform unit 10
Output to 2. The inverse orthogonal transform unit 102 performs an inverse Fourier transform of the Fourier transform value of the similar value L, and generates a map L of the similar value L.
(X, y) is output to the map processing unit 103. The map processing unit 103 extracts a point (peak) having a high value from the map L (x, y) and outputs the position and the value. The map processing unit 103 and subsequent units can be freely configured as needed.

Now, assuming that the size of the target image is A (= ^2γ ) and the size of the template image is B, in order to sequentially scan the template image on the target image and obtain the correlation value at each position, The number of times of product = 2AB is required. The number of calculations here is
Evaluate based on the number of products with high computational cost. On the other hand, in the second embodiment, two FFTs by the orthogonal transform units 83 and 93, a product-sum calculation by the integrating unit 101, and one
The number of times of inverse FFT is required, and the number of times of product = 3 {(2γ-4) A + 4} + 2A is sufficient. Comparing the numbers of these calculations, for example, A = 256 × 256 = 2 ¹⁶ and B = 60 × 60
, The number of product calculations according to the second embodiment is about 1/1.
00, which enables very high-speed processing, which leads to a reduction in the amount of processing.

As described above, in the face extracting unit 7 of the second embodiment,
The position of the face can be extracted with a small amount of processing. Therefore, even in a scene where a limited amount of processing is required, such as a portable image communication terminal, the position and size of the face can be extracted. Also, by performing the double-width representation, the shooting location and time are not limited as in a portable image communication terminal, and even in a scene where all shooting conditions must be assumed,
It is possible to stably extract a face.

<Embodiment 3> According to the face extraction methods of Embodiments 1 and 2, even if no face exists in the target image, a portion close to the face is forcibly extracted as a face area. Therefore, as a third embodiment, a method for further determining whether the position and size of the face extracted by the face extraction methods of the first and second embodiments are truly a face will be described.

In order to realize this, the analysis unit 55 of the first embodiment shown in FIG. 12 or the second embodiment shown in FIG.
A configuration (face / non-face determination unit) for determining whether or not the extracted face area is a true face is provided at the subsequent stage of the map processing unit 103. In the case where a face / non-face determination unit is provided after the analysis unit 55 of the first embodiment, in the simplest case, a threshold value for determining a face / non-face is determined in advance, and the area output from the analysis unit 55 If the value obtained from the voting value and the size of the face exceeds this threshold, the area is determined to be a face. here,
The value obtained from the voting value and the size of the face is a value obtained by dividing the voting value by the size of the face. Performing such processing is
This is because the voting value proportional to the face size is normalized by the face size. Also, the map processing unit 103 according to the second embodiment
When a face / non-face determination unit is provided after
A threshold for determining a face / non-face is determined in advance, and if the similarity value of the area output from the map processing unit 103 exceeds this threshold, the area is determined to be a face. In the above-described first and second embodiments, the case where only one face area is output from the face extracting unit 7 has been described. The face / non-face determination of the third embodiment can be applied.

The face area determined as not a face by the face / non-face determination unit is not output from the face extraction unit 7 to the display control unit 5 and the transmission data processing unit 8. When the face area is not output from the face extraction unit 7, the transmission data processing unit 8 in the first embodiment uses the transmission area 31 at the previous time without moving the position of the transmission area 31. If the face area is not output for a certain period of time, the transmission area 31 is set at the initial position (for example, the center of the shooting area 30).

On the other hand, there is a method of determining a face / non-face using a support vector function instead of the above-described determination method using a threshold value. The outline of the face / non-face determination using the support vector function will be described below. Note that the support vector itself is a known technique, and is described in the document “Support Vec.
Discrimination of multiple categories by tor Machines (IEICE IEICE PRMU98-36 (19
98-06)) ".

FIG. 22 is a block diagram showing components of the face extraction unit 7 of the third embodiment that are added to the configurations of the first and second embodiments. 22, the additional configuration according to the third embodiment includes an image size normalizing unit 111, a feature vector extracting unit 112, a face / non-face determining unit 113, and a face / non-face learning dictionary 114. The configuration of FIG. 22 is added after the analysis unit 55 of the first embodiment or after the map processing unit 103 of the second embodiment.

The image size normalizing section 111 includes an analyzing section 55
Alternatively, the image of the face area portion output from the map processing unit 103 is cut out from the target image. Then, the image size normalization unit 111 determines, for the cut-out image (hereinafter, referred to as a face area candidate image), image features (for example,
After obtaining edge strength, color value, luminance value, etc., normalization is performed to a fixed size. Here, the face area candidate image is 10 ×
An example of enlargement or reduction (that is, normalization) to the size of 10 pixels will be described. The feature vector extracting unit 112 acquires the luminance information of the normalized face area candidate image as one of the feature data. In this example, since the image is normalized to a 10 × 10 pixel image, a 100-dimensional feature vector xi
(0 ≦ i <100) is obtained.

Here, the feature vector extracting unit 112 may extract the edge normal direction vector as a feature vector. Specifically, the face area candidate image is
Apply a bell filter and a Y-direction sobel filter,
A direction vector is calculated based on the intensity in the X direction and the intensity in the Y direction at each pixel. In this calculation, since the angle and the strength are calculated as the values, the strength is ignored and only the angle is extracted. Then, normalization in each direction is performed on the basis of 256 gradations, and is used as a feature vector. Also,
The feature vector extraction unit 112 may calculate a normalized histogram for each angle inside the face area candidate image, and extract the histogram of the edge normal as a feature vector.

The face / non-face judging section 113 judges the face / non-face of the face area by the following formula using the characteristic images and parameters prepared in the face / non-face learning dictionary 114 in advance.

[Equation 11] Where K () is the kernel function and αi is the corresponding L
The aggregate coefficient, yi indicates teacher data, and +1 is applied when the learning dictionary is a face and -1 when the learning dictionary is not a face.
The kernel function includes a polynomial K (Si, Xi) = (Si · Xi + 1) and a two-layer neural network K (Si, Xi) = tanh in addition to the above equation (12).
(Si · Xi−δ) can be used.

FIG. 23 shows the result of face / non-face discrimination. face·
The non-face determination unit 113 determines that the face area candidate image is a face image when the result of the above expression (12) is greater than 0, and determines that the image is a non-face image when the result is smaller than 0. Similarly, face / non-face determination is performed on other face area candidate images. In the example of FIG. 23, the image 121 is determined to be a face image, and
124 is determined to be a non-face image. In the face / non-face learning dictionary 114, a face image and a non-face image are prepared as teacher data, and a dictionary is created using the same feature data used for identification.

As described above, in the face extracting unit 7 of the third embodiment,
Even when the actual face is other than the first candidate for the face area, stable extraction of the face area is possible. Further, even when there is no face in the image, it can be determined that there is no face, so that it is possible to automatically detect a case where it is not necessary to move and display the position of the face.

[Brief description of the drawings]

FIG. 1 is a block diagram illustrating a configuration of an image communication terminal according to a first embodiment of the present invention.

FIG. 2 is a flowchart illustrating a procedure of a tracking process performed by a transmission data processing unit 8;

FIG. 3 is a diagram illustrating a relationship between a photographing area 30 and a transmission area 31.

FIG. 4 is a diagram illustrating a relationship between a photographing area 30 and a transmission area 31.

FIG. 5 is a diagram illustrating a relationship between a shooting area 30 and a transmission area 31.

FIG. 6 is a diagram illustrating a relationship between a shooting area 30 and a transmission area 31.

FIG. 7 is a block diagram illustrating a configuration of an image communication terminal according to a second embodiment of the present invention.

FIG. 8 is a diagram showing an example of a mark displayed on the screen of the display unit 3;

FIG. 9 is a diagram illustrating an example of a mark displayed on a screen of a display unit 3;

FIG. 10 is a diagram showing an example of a mark notified using the numeric keypad of the input unit 22.

FIG. 11 is a diagram illustrating an example of an image of the user 1 displayed on the screen of the information processing device of the other party.

FIG. 12 is a block diagram illustrating a configuration of a face extraction unit 7 according to the first embodiment.

FIG. 13 is a diagram illustrating an example of a template stored in a template storage unit 52;

FIG. 14 is a flowchart illustrating a procedure of a voting process performed by a voting unit.

FIG. 15 is a diagram illustrating an example of an edge image extracted by an edge extraction unit 51.

FIG. 16 is a diagram illustrating the concept of the number of votes stored in a voting storage area of a voting result storage unit 53 by a voting process.

FIG. 17 is a flowchart illustrating a procedure of an analysis process performed by the analysis unit 55;

FIG. 18 is a block diagram illustrating a configuration of a face extraction unit 7 according to the second embodiment.

FIG. 19 is a diagram illustrating an example of a template image and a target image input to edge extraction units 81 and 91.

FIG. 20 is a diagram for explaining the sign reversal of the inner product.

FIG. 21 is a diagram illustrating a process of compressing an evaluation vector.

FIG. 22 is a block diagram illustrating a part of the configuration of a face extraction unit 7 according to the third embodiment.

FIG. 23 is a diagram illustrating an example of a face / non-face discrimination result performed by the face / non-face discrimination unit 113.

[Explanation of symbols]

 DESCRIPTION OF SYMBOLS 1 ... User 2,22 ... Input part 3 ... Display part 4 ... Camera part 5,25 ... Display control part 6 ... Self-image memory 7 ... Face extraction part 8 ... Transmission data processing part 9 ... Communication part 10 ... Reception data processing part DESCRIPTION OF SYMBOLS 11 ... Partner image memory 12 ... Notification part 30 ... Photographing area 31 ... Transmission area 32 ... Effective area 51, 81, 91 ... Edge extraction part 52 ... Template storage part 53 ... Voting result storage part 54 ... Voting part 55 ... Analysis part 82 , 92 ... Evaluation vector generation unit 83, 93 ... Orthogonal transformation unit (FFT) 84, 94 ... Compression unit 85 ... Recording unit 101 ... Integration unit 102 ... Inverse orthogonal transformation unit (inverse FFT) 103 ... Map processing unit 111 ... Image size Normalizing section 112 Feature vector extracting section 113 Face / non-face determining section 114 Face / non-face learning dictionary 121 to 124 Image R Marker t1 to tn Circle shape

──────────────────────────────────────────────────続 き Continued on the front page (51) Int.Cl. ⁷ Identification symbol FI Theme coat ゛ (Reference) H04N 1/46 H04N 7/14 7/14 1/46 Z (72) Inventor Yuji Takada Kadoma, Osaka 1006 Kadoma Matsushita Electric Industrial Co., Ltd. (72) Inventor Masafumi Yoshizawa 1006 Kadoma, Kadoma, Osaka Prefecture Matsushita Electric Industrial Co., Ltd. (72) Inventor Tetsuya Yoshimura 1006 Kadoma, Kazuma, Osaka Pref. Matsushita Electric Industrial Co., Ltd. (72) Katsuhiro Iwasa 1006 Kadoma, Kazuma, Kadoma, Osaka Pref. BA02 CA01 CA08 CA12 CA16 CB01 CB08 CB12 CB16 CC01 CE03 CE17 DA07 DB02 DB06 DB09 DC16 DC32 5C064 AA01 AA02 AB02 AB04 AC04 AC12 AC15 AD01 AD08 AD14 5C079 HA01 HB01 LA07 LB11 MA17 5L096 CA02 EA21 FA06 FA59 FA69 JA09

Claims (26)

[Claims] 1. An image communication terminal for transmitting an image of a user photographed by a camera unit to a partner, an input unit for receiving an input from the user, a camera unit for photographing the user, and the camera unit A face extraction unit that extracts the position and size of a user's face (hereinafter, referred to as a face area) from an image captured in step 1, a display unit that displays an image to the user, A communication unit that communicates at least an image, and an image of a rectangular transmission area that is smaller than an area of the image captured by the camera unit and that is set to be movable within the image area, to the communication unit. A transmission data processing unit for outputting, wherein an effective area that moves integrally with the transmission area is set in an area of an image captured by the camera unit, and the transmission data processing unit is configured to extract the face Area misses the effective area When, characterized in that moving the set position of the transmission region in accordance with the position of the face area, the image communication terminal. 2. The image communication terminal according to claim 1, wherein the effective area is smaller than the transmission area and set within the transmission area. 3. The transmission data processing section, when the extracted face area deviates from the effective area, moves the transmission area so that the face area is located at the center of the transmission area. The image communication terminal according to claim 1, wherein: 4. The transmission data processing unit moves the transmission area such that when the extracted face area deviates from the effective area, the face area is located above the center of the transmission area. The image communication terminal according to claim 1, wherein 5. The input data processing unit, wherein when the extracted face area deviates from the effective area, the input unit controls the input area so that the face area is located at the center of the transmission area or in an upward direction from the center. The image communication terminal according to claim 4, wherein the transmission area is moved by switching according to transmission mode information input from the terminal. 6. The display unit displays an image in the transmission area and the face area on a monitor in accordance with information input from the input unit, and a user refers to the monitor display, The image communication terminal according to claim 4, wherein the position of the transmission area can be adjusted in the vertical and horizontal directions by an input to the input unit. 7. An image communication terminal for transmitting an image of a user photographed by a camera unit to a partner, an input unit for receiving an input from the user, a camera unit for photographing the user, and the camera unit A face extraction unit that extracts the position and size of a user's face (hereinafter, referred to as a face area) from an image captured in step 1, a display unit that displays an image to the user, A communication unit that communicates at least an image, and an image of a rectangular transmission area that is smaller than an area of the image captured by the camera unit and that is set to be movable within the image area, to the communication unit. A transmission data processing unit for outputting, wherein an effective area that moves integrally with the transmission area is set in an area of an image captured by the camera unit, and the transmission data processing unit is configured to extract the face Area misses the effective area In this case, the setting position of the transmission area is moved in accordance with the position of the face area, and based on the extracted image brightness of the face area, the face of the face in the image shot by the camera unit is extracted. An image communication terminal, wherein the image communication terminal corrects the image brightness of the transmission area and outputs the corrected image brightness to the communication unit so that visibility is improved. 8. The image communication terminal according to claim 7, wherein the transmission data processing unit corrects a color tone in addition to an image luminance of the transmission area and outputs the corrected color tone to the communication unit. 9. An image communication terminal for transmitting an image of a user photographed by a camera unit to a partner, an input unit for receiving an input from the user, a camera unit for photographing the user, and the camera unit A face extraction unit that extracts the position and size of a user's face (hereinafter, referred to as a face area) from an image captured in step 1, a display unit that displays an image to the user, A communication unit that communicates at least an image, and an image of a rectangular transmission area that is smaller than an area of the image captured by the camera unit and that is set to be movable within the image area, to the communication unit. A transmission data processing unit for outputting, wherein an effective area that moves integrally with the transmission area is set in an area of an image captured by the camera unit, and the transmission data processing unit is configured to extract the face Area misses the effective area In this case, the setting position of the transmission area is moved in accordance with the position of the face area, and based on the extracted image brightness of the face area, the face of the face in the image shot by the camera unit is extracted. An image communication terminal, wherein a value of an exposure level of the camera unit is set so that visibility is improved. 10. An image communication terminal for transmitting an image of a user photographed by a camera unit to a partner, comprising: a camera unit for photographing the user; and a face of the user based on the image photographed by the camera unit. A face extracting unit for extracting the position of the user, a display unit for displaying an image received from the other party to the user, and a user's face in the image taken by the camera unit based on the extracted position of the face. An image communication terminal, comprising: a notification control unit that notifies a user of the position of the information processing unit; and a communication unit that performs at least image communication with a partner information processing device. 11. The face extraction unit also extracts a size of a face together with the position of the face of the user, and the notification control unit determines a position of the face of the user in an image captured by the camera unit. The image communication terminal according to claim 10, wherein the size is notified to a user. 12. The image communication terminal according to claim 10, wherein the notification control unit causes the display unit to display only the position of the extracted face or a mark indicating the position and size. . 13. The image communication terminal according to claim 12, wherein the mark is displayed on an image received from a partner. 14. The image communication terminal according to claim 12, wherein the mark is displayed outside an image received from a partner. 15. The image communication according to claim 12, wherein the notification control unit notifies the position of the extracted face via a position notification unit provided separately from the display unit. Terminal. 16. The image communication terminal according to claim 10, wherein a method of notifying a user performed by the notification control unit can be switched according to an instruction from the user. 17. The face extraction unit extracts an edge portion (pixels corresponding to the outline of a person, a contour of a face, and the like) from an image captured by the camera unit, and extracts an image of only the edge portion (hereinafter, referred to as an image). An edge extraction unit that generates an edge image), a template storage unit that stores a plurality of templates provided concentrically at a center point in various shapes having similar shapes and different sizes, and A voting result storage unit for storing the coordinate position on the edge image and the number of votes in association with each other for each shape of each size constituting the template, and sequentially storing the center point of the template at each pixel position of the edge portion. For each of the moved pixel positions, the coordinates stored in the voting result storage unit for each coordinate position corresponding to the position of all pixels forming the shape of each size. A voting unit for increasing or decreasing the number, and an analyzing unit for calculating a position and a size of a face included in the target image based on the voting number stored in the voting result storage unit. Item 17. The image communication terminal according to any one of Items 1 to 16. 18. The image communication terminal according to claim 17, wherein the predetermined shape is a circle. 19. The face extraction unit inputs a predetermined template image, finds an edge normal direction vector of the image, generates an evaluation vector from the edge normal direction vector, and orthogonally transforms the evaluation vector. A template image processing unit, an image captured by the camera unit, an edge normal direction vector of the image obtained, an evaluation vector generated from the edge normal direction vector, and an input for orthogonally transforming the evaluation vector An image processing unit; a product-sum unit for calculating the product-sum of the corresponding spectrum data for each of the orthogonally transformed evaluation vectors generated for each of the template image and the captured image; and inverting the result of the product-sum calculation. An inverse orthogonal transformation unit that performs orthogonal transformation to generate a map of similarity values, wherein the evaluation vector is the edge of a corresponding image. Line direction vector includes the even combination angle converted components, calculation formula of the similarity value, the orthogonal transform and inverse orthogonal transformation, characterized in that both those having linearity, claims 1 to 16
The image communication terminal according to any one of the above. 20. The face extraction unit according to claim 19, wherein, in the expression of the evaluation vector, a value calculated based on an angle when the edge normal direction vector is expressed in polar coordinates is used. Image communication terminal. 21. The face extraction unit, wherein a position and a size extracted as a face from an image captured by the camera unit are:
17. The apparatus according to claim 1, further comprising a face / non-face determination unit that determines whether the face is truly a face, and outputs an extraction result only when the face is determined. Image communication terminal. 22. The face extracting unit, wherein a position and a size extracted as a face from an image taken by the camera unit are based on the content stored in the voting result storage unit.
18. The image communication terminal according to claim 17, further comprising a face / non-face determination unit that determines whether the face is truly a face, and outputs an extraction result only when the face is determined. 23. The face extraction unit, wherein the position and size extracted as a face from an image captured by the camera unit are based on the similarity value generated by the inverse orthogonal transform unit.
20. The image communication terminal according to claim 19, further comprising a face / non-face determination unit that determines whether or not the image is a true face, and outputs an extraction result only when the image is determined to be a face. 24. The face / non-face determining unit uses an image feature obtained from an area extracted as a face from an image captured by the camera unit, based on a support vector function determination result. Determining a face,
The image communication terminal according to claim 21. 25. The image processing apparatus according to claim 25, wherein the face / non-face determination unit uses an edge normal direction vector obtained from an area extracted as a face from an image captured by the camera unit as the image feature. Item 25. The image communication terminal according to item 24. 26. The image processing apparatus according to claim 26, wherein the face / non-face determination unit uses, as the image feature, a histogram of an edge normal obtained from a region extracted as a face from the image captured by the camera unit. Item 25. The image communication terminal according to item 24.