KR20100001409A - Method for recognizing face expression and control method of robot for appreciating image contents using the same - Google Patents

Abstract

PURPOSE: A method for recognizing the facial expression of a user and a control method of the robot for visual contents impression using the same are provided to operate the image contents according to the recognized face pose by recognizing face pose and expression of a user. CONSTITUTION: A face range is extracted from an inputted image. A pose of the extracted face range is presumed(S101). In case the pose examines the front, a face shape and a texture are extracted(S103). The face shape and the texture are generated in an average form(S104). The average face form and the texture are processed in retinex(S105). A first face parameter about each facial expression learns based on the average face form and the texture of the retinex about each facial expression(S106).

Description

Recognition method of facial expression and control method of robot for viewing video contents using the same {METHOD FOR RECOGNIZING FACE EXPRESSION AND CONTROL METHOD OF ROBOT FOR APPRECIATING IMAGE CONTENTS USING THE SAME}

The present invention relates to a facial expression recognition method and a control method of a robot for image content appreciation using the same. More particularly, the present invention provides a method for recognizing a facial pose and a facial expression of a user who views image content such as a cartoon, As a result, a facial expression recognition method for manipulating video content such as cartoons according to user's preferences, and grasping a change in user's emotion as a recognized facial expression in the process of viewing the video content and reflecting it in the video content while viewing the video content And it relates to a control method of a robot for viewing video content using the same.

Human-Robot Interaction (HRI) technology applied to the existing content robot developed due to the development of the sensor is a technology that recognizes the touch of the user or controls the content of the content by using a sensored glove.

The technology of recognizing a user's touch is using a touch screen that is commercially available recently. It is configured to detect the user's touch and control the content even if the user touches the monitor screen even if a conventional input device such as a mouse or keyboard is not provided separately. For example, when the video content is a cartoon, the user can perform a control such as turning a page of the cartoon by touching the monitor screen.

In addition, a technology using a sensor attached glove is a technology that the user wears the sensor attached glove, the sensor detects the movement of the glove when the user shakes the hand. Accordingly, in the case of cartoon content, control such as turning over a page of a cartoon according to the detection of a sensor may be performed. This technology is also used in film to reproduce human behavior in computer graphics.

However, the conventional Human-Robot Interaction (HRI) technology has a problem in that the user must touch the touch screen and purchase expensive gloves.

In addition, even using the conventional Human-Robot Interaction (HRI) technology, content such as cartoons can be enjoyed simply by turning pages, so that the cartoon can be expanded while a certain cut of the cartoon is changed or the character arrangement is changed according to the user's taste. It was required to be able to appreciate the technology.

In addition, if the user's emotion changes, such as joy, sadness, anger, fear, etc., can be reflected in the video content, the viewing of more realistic video content It will be possible.

Accordingly, the present invention recognizes a face pose and facial expression of a user who views video content such as a cartoon, manipulates the video content such as a cartoon according to the user's taste according to the recognized face pose, and the user in the viewing process of the video content. The object of the present invention is to provide a facial expression recognition method for recognizing a change of emotion as a recognized facial expression and reflecting it in image content, and to recognize the image content and a control method of a robot for image content using the same.

According to the present invention, (a) extracting a face region from the input image to estimate the pose of the extracted face region; (b) extracting a face shape and a face texture of the extracted face area when the pose of the extracted face area is estimated to stare at the front; (c) warping the extracted face shape and face texture to a predetermined average shape to generate an average face shape and an average face texture; (d) Retinex processing the average face shape and the average face texture; (e) performing steps (a) to (d) on a plurality of facial expressions, respectively, and performing the Retinex-treated average face shape and the Retinex treatment on each of the facial expressions; Learning first facial parameters for each facial expression based on the averaged facial textures; (f) Generating a second face parameter based on a Retinex-treated average face shape and average face texture generated by performing steps (a) to (d) on the input image. Making a step; (g) estimating a facial expression for the second facial parameter as one of the first facial parameters having the greatest similarity with the second facial parameter. .

Here, the learning of the first face parameter and the generation of the second face parameter may be performed by applying an AAM (Active Appearance Model) to the Retinex-treated average face shape and the average face texture. It is characterized in that the generated.

When the AAM is applied, a Principal Component Analysis (PCA) technique may be applied to the average face shape and the average face texture.

(G) step (g1) calculating a first Gaussian distribution for the first facial parameter; (g2) calculating a second Gaussian distribution for the second facial parameter; (g3) calculating a distance difference between the first Gaussian distribution and the second Gaussian distribution as the similarity.

The step (a) may include: (a1) determining a pixel corresponding to the human skin through color in the input image; (a2) determining whether a ratio of the pixels meets a threshold value in a sub-window of a predetermined size of the image; (a3) determining whether an image is a face image in the subwindow through an Adaboost algorithm when the pixels clustered in the subwindow meet a threshold; (a4) if the determination result is a face image in the sub-window, setting the sub-window as the first face region sub-window; (a5) The manifold may be classified and stored according to the face poses by classifying and storing the plurality of first face area sub-windows having various face poses which are determined by repeating steps (a1) to (a4). Forming a space; (a6) performing a step (a1) to (a3) to set a corresponding subwindow as a second face region subwindow when it is a face image; (a7) comparing the pixel data of the first face region subwindow with the pixel data of the first face region subwindow stored in the manifold space and comparing the pixel data of the first face region subwindow to the pixel data of the second face region subwindow as one closest to the pixel data of the second face region subwindow. Estimating a pose of the region.

On the other hand, the above object is, according to another embodiment of the present invention, displaying the image content on the display unit; Photographing a person located in front of the display unit; Estimating a facial expression of a photographed person by applying the facial expression recognition method to the photographed image; It may also be achieved by a control method of a robot for image content viewing, comprising changing a color of at least one of an object and a background included in the image content according to the estimated facial expression.

The method may further include changing the shape of the object according to the estimated facial expression.

The video content may include cartoon content.

As described above, according to the present invention, a face pose and a facial expression of a user who enjoys a video content such as a cartoon are recognized, a video content such as a cartoon is manipulated according to a user's taste according to the recognized face pose, and a viewing process of the video content. In the present invention, it is possible to grasp the change in the user's emotion with the recognized facial expression and to reflect the same on the image content, thereby enjoying the image content.

Hereinafter, with reference to the accompanying drawings will be described in detail the present invention. Here, before describing the present invention, the video content used in the present invention is described as an example of a cartoon that can be enjoyed while turning a page, but the video content according to the present invention is not limited to a cartoon and other games, etc. Of course, it can be applied to the content.

1 is a view for explaining a process of learning a facial expression in the face recognition method and a control method of the image content viewing robot 300 (see FIG. 8) according to the present invention, Figure 2 is a face recognition method and This is a view for explaining a method of extracting a facial expression of an actual user using the facial expressions learned in the control method of the image content viewing robot 300 and applying the same to the control of the image content viewing robot 300.

Referring to FIG. 1, a process of learning a facial expression is described. An image including a facial expression of a user is photographed (S100). Then, the face region is extracted from the image input through the imaging, and the pose of the face region is estimated from the extracted face region (S101).

Hereinafter, the extraction of the face region and the estimation of the pose in step S101 will be described in detail with reference to FIGS. 3 to 7.

First, referring to FIG. 3, an image of a user is photographed through a photographing apparatus such as a camera 320 (see FIG. 8) (S11). Then, in order to extract the skin region of the user from the captured image, the skin color pixel is determined in the image (S12). The color of each pixel of the image is expressed in RGB system, but it is converted into YCrCb system as shown in FIGS. 5A to 5C. Here, the YCrCb scheme is a color resolution used to digitize luminance (Y), color difference signals (BY, RY), and R · G · B signals of component digital video, where Y is Distribution of Luminance Component, and Cr and Cb are Red and Blue Distribution of Crominance Component. Since converting the RGB system to the YCrCb system is a technique known to those skilled in the art, a detailed description thereof will be omitted.

Table 1 below is a table showing the YCrCb threshold in the case of recognizing the color of the pixel as the skin color.

TABLE 1

Y 0 to 225 Cb 77-127 Cr 135 to 165

5A to 5C, the YCrCb threshold value of the pixel corresponding to the skin color when converted to the YCrCb system may be displayed as shown in [Table 1]. That is, if the YCrCb value of the color of the pixel of the image matches the threshold of [Table 1], the pixel is regarded as a pixel representing the skin of the user.

In addition, the image photographed by the photographing apparatus is divided into a plurality of sub-windows. Therefore, if the pixels matching the thresholds are grouped in [Table 1] in the subwindow (S13), it is regarded as a candidate of the face image.

Here, it is required to determine whether the image in the subwindow considered as a candidate of the face image is a face image. This is because an image in the subwindow considered as a candidate of the face image may be an image of the user's hand. Accordingly, in the present invention, it is determined whether the image is a face image in the sub-window through the Adaboost algorithm (S14). Here, since the Ada Boost algorithm is a known technique, a detailed description thereof will be omitted. When the image in the sub-window is not a face image by performing the Adaboost algorithm, the image in the sub-window may be regarded as the user's hand image.

On the other hand, if the image is identified as a face image in the sub-window, the sub-window is determined as the first face region subwindow (S15), the first face region subwindowed manifold (manifold) as shown in FIG. ) Is stored for space formation.

The face image of the first face region subwindow may have various poses. Here, when the pose of the face image is changed, the data value of each pixel may also be changed accordingly. Accordingly, the manifold space may be formed by storing the plurality of first face region subwindows having poses of various face images by repeating steps S11 to S15 of FIG. 3.

As illustrated in FIG. 6, the manifold space in the present invention is divided into seven regions consisting of forward, upward, downward, leftward, and rightward directions. Each region includes a plurality of classification regions in which pixel data of a first face region subwindow having similar gaze faces among pixel data of the first face region subwindow is clustered in a linear space formed through principal component analysis (PCA Plane). it means.

Among the seven areas shown in FIG. 6, areas P0 and P4 are areas related to the left / right image of the face, area P2 is an area related to the forward image of the face, area P6 is an area about the upward image of the face, and P5 The area is an area related to the downward image of the face, and the P1 and P3 areas are areas related to the image between the face forward and left / right.

Each of the P0 to P6 regions stores five first facial region subwindows in which the pixel data are close to each other, and averages pixel data of the five stored first facial region subwindows to average the pixel data of each of the P0 to P6 regions. Determine the average pixel data value. In addition, a plurality of users may perform steps S11 to S17 to form a manifold space corresponding to each user.

When the manifold space is formed (S17), steps S11 to S14 are performed again to determine a second face region subwindow of the current user to be substantially recognized (S19). Thereafter, the pixel data of the first face region subwindow and the pixel data of the second face region subwindow of each of the areas P0 to P6 forming the manifold space are compared with each other (S20).

Here, the step S20 is to go through two inference process.

[Equation 1]

(K ^* : user of the second face region subwindow, I: second face region subwindow, M _k : manifold space of the k th user)

The user of the second face region subwindow determined through Equation 1 is estimated. That is, it is inferred which manifold space is closest to the user of the predetermined second face region subwindow. Here, d ² (I, M _k ) means the square of the distance (distance of the vector) between the second face region subwindow I and the manifold space Mk of the k-th user.

[Equation 2]

(I: second face area subwindow, Mk: k-th manifold space, Pki: i-th area of k-th manifold space)

Inferring the corresponding manifold space of the user of the second face region subwindow, Bayes' theorem indicates which region is closest to the second face region subwindow in the manifold space Mk having i regions. Infer using [Equation 2] developed using

That is, p (P _ki | I) provides a probability corresponding to the distance between the second face region subwindow I, which is an input image, and the i th region P _ki of the k-th manifold space, And inferring the k-th manifold space having the shortest distance into the manifold space through Equation 2, and the probability value corresponding to the i value having the largest probability among the P _ki regions included in the inferred k-th manifold space The P _ki region is inferred as the region closest to the second face region subwindow as the input image.

For example, as shown in FIG. 6, if the pixel data of the second face region subwindow coincides with the pixel data of the I portion in the manifold space, it can be seen that the I portion is the position closest to the P1 region. Accordingly, pixel data of the second face region subwindow is approximated to pixel data of the first face region subwindow corresponding to the P1 region.

Therefore, the first face region subwindow corresponding to the P1 region is extracted (S21). For example, even if the pixel data of the second face region subwindow is the same as the pixel data of the first face region subwindow included in the P1 region, the pixel data of the first face region subwindow may be extracted.

In addition, assuming that the sub window has a total of 361 pixels of 19 in width and 19 in length, 361 pixels of data are compared between the first face area sub window and the second face area sub window, but to shorten the comparison time. Only the pixel data having the greatest feature that can distinguish the face image can be compared with each other. Here, the pixel data having the largest feature that can distinguish the face image may be approximately 19 pixel data clustered in the center of the sub-window.

In operation S22, it may be recognized that the face pose of the second face region subwindow is the left pose from the face pose of the first face region subwindow corresponding to the extracted P1 region.

Meanwhile, the occlusion of the face may occur due to a given environment (a background difference around the face, a change in illumination, a part of the face is covered by another object, etc.). .

For example, if an obstruction occurs in an image in the second face region subwindow, the pixel data of the first face region subwindow and the pixel data of the second face region subwindow may malfunction.

In order to solve this problem, in the recognition of the facial region and the estimation of the pose, as shown in FIG. 4, when there is a change in pixels between the same first facial region subwindow 10 after the steps S11 to S16 (S16). -1), the first mask data 20 of the pixel whose color is other than the skin color and the second mask data 10 of the pixel whose color is the skin color can be stored (S16-2).

For example, as illustrated in FIG. 7, there may be a change of a pixel such as sunglasses in the first face region sub window 10 by the same user.

The change of pixels such as sunglasses is performed by recognizing the change of pixels in the first face region subwindow 10a when the sunglasses are worn and the first face region subwindow 10b when the sunglasses are not worn. The first mask data 20, which is the pixel data of the sunglasses, and the second mask data 30 of the pixel data of the index skin color around the eyes when the sunglasses are not worn.

If the pixel corresponding to the first mask data 20 exists in the second face region subwindow 40 determined by performing steps S17 to S19 (S19-1). In operation S19, the second face region subwindow 50 from which the masking phenomenon is removed may be extracted by replacing a pixel corresponding to the first mask data 20 with a pixel corresponding to the second mask data 30 (S19). -2). Thereafter, by performing the above steps S20 to S22, the face pose of the second face region subwindow from which the obstruction is removed is recognized, thereby pixel data of the first face region subwindow and pixel data of the second face region subwindow. It is possible to prevent malfunction when comparing with each other.

As shown in FIG. 7, the occlusion phenomenon is one example of sunglasses, but may be provided in various types such as eyeglasses, masks, band-aids, hats, and the like. Of course, it can cope with the mask phenomenon.

On the other hand, when it is determined that the image of the sub-window is not a face image by the Ada Boost algorithm by the step S14 of Figures 3 and 4, the image of the sub-window corresponding to the threshold of [Table 1] by the step S13 Since the pixels are clustered, it can be regarded as the user's hand image.

In this case, when it is determined as the user's hand image in step S14, the direction indicated by the user's hand image, the number of extended fingers, etc. may be estimated and applied to the control of the robot 300 for viewing image content accordingly. Korean Patent Application No. 2007-0075776, filed by the inventor of the present invention and previously filed, has been proposed through a control method of a comic book appreciation robot and a comic appreciation robot using the same.

Through the above-described method, the face recognition method and the control method of the image content viewing robot 300 according to the present invention extract the face region from the input image and estimate the pose of the face region from the extracted face region.

Referring back to FIG. 1, if the pose of the face region is estimated in step S101, it is determined whether the pose of the estimated face region gazes at the front face (S102). At this time, when it is determined that the pose of the face region gazes at the front, a face shape (hereinafter referred to as “shape”) and a face texture are extracted from the extracted face region (S103).

Here, the face shape includes outline information of the face image such as an outline of the face image, eyes, eyebrows, and lips. In other words, the face shape includes information about an edge region in which the color of the face image changes. The face texture includes color information included in the face shape, that is, information about a pixel value, and includes information on the face image inside the face shape.

Thereafter, the extracted face shape and the face texture are warped to a predetermined average shape to generate an average face shape and an average face texture (S103). Here, the warping process is to guarantee the accuracy of the human face in comparison with the extraction of the expression from the face image, since the width or length of the human face may vary. Since the difference in the width or length of the face may vary depending on the distance away from the camera 320, the face is corrected through a warping process.

Then, the average face shape and the average face texture generated through the warping process are retinexed (S105). Retinex processing compensates for changes in facial texture, ie, pixel values, for the face image due to differences in lighting conditions when the image is taken.

Then, the first facial parameter is generated for the facial expression of the current facial image based on the Retinex-treated average face shape and average face texture (S106). Here, in the present invention, the first face parameter is generated by applying an AAM (Active Appearance Model) to the Retinex-treated average face shape and average face texture.

In addition, when the AAM (Active Appearance Model) is applied, the Principal Component Analysis (PCA) technique may be applied to the average face shape and the average face texture. That is, the information combining the average face shape and the average face texture becomes a vector having a long length, and the calculation may be complicated when learning the first face parameter by applying an AAM (Active Appearance Model). For this reason, a Principal Component Analysis (PCA) technique is applied to reduce information in a vector form combining an average face shape and an average face texture.

On the other hand, when capturing the image of the facial expression image for the corresponding facial expression is input in the form of a video, step S101 to S106 of Figure 1 is performed in the frame unit of the image, the image frame for the facial expression in step S107 The first facial parameter is learned through the process of repeating steps S101 to S106 by checking whether it is finished (S108).

When the learning of the first facial parameter for one facial expression is completed (S108), it is checked whether the other facial expressions are learned (S109), and when the other facial expressions are learned, the corresponding facial expression is processed through the above-described process. Through the method of learning the first facial parameter for each of the first facial parameter for each facial expression, for example, facial expressions for pleasure, sadness, anger, fear.

Through the above process, the learning of the first facial parameter for each of the facial expressions of one user is completed, and the first facial parameter may be learned through the same process for the facial expression of another user. The learned first face parameter is registered and stored in the robot 300 for viewing image content.

Hereinafter, a process in which an actual user views the image content through the image content viewing robot 300 while the first face parameter is registered and stored in the image content viewing robot 300 will be described with reference to FIG. 2.

First, when the user stands in front of a display unit 310 such as a monitor of the robot 300 for viewing image content as shown in FIG. 8, and views the image content, the camera 320 provided in the image content robot 300 is viewed. An image of the face of the user is photographed (S200).

Then, the face region is extracted from the image input by the camera 320 and the pose of the extracted face region is estimated (S201). Here, the extraction process and the estimation of the pose of the face region from the image are the same as in step S101 of FIG. 1 (see FIGS. 3 and 4), and thus a detailed description thereof will be omitted.

Then, it is determined whether the extracted face image is staring at the front face (S202). If it is determined that the face image is not staring at the front face, different preset controls are performed for each pose (S210). Here, if the control according to each pose is described as an example, when the pose of the face image is estimated to stare at the lower left side of the display 310, the object in the image content, for example, the cartoon when the image content is a cartoon The character within may move to the lower left on the screen of the display 310.

On the other hand, if it is determined in step S202 that the face image gazes at the front, a face shape and a face texture are extracted from the face image (S203). Here, the extraction of the face shape and the face texture shown in FIG. 2 is the same as the above-described step S103 of FIG. 1, and a detailed description thereof will be omitted.

Then, by warping the face shape and the face texture, the average face shape and the average face texture are generated (S204), and the generated average face shape and the average face texture are retinex ( Retinex) to process (S205). Here, the warping and retinex processing methods in steps S204 and S205 correspond to the warping and retinex processing methods in steps S104 and S105 of FIG. 1 described above. The detailed description thereof will be omitted.

In operation S205, the second face of the facial expression extracted from the person standing in front of the robot 300 for viewing the current image content based on the average face shape and the average face texture processed by Retinex. A parameter is generated (S206).

Here, the second face parameter according to the present invention may be generated by applying an AAM (Active Appearance Model) to the Retinex-treated average face shape and average face texture. When the AAM (Active Appearance Model) is applied, the Principal Component Analysis (PCA) technique may be applied to the average face shape and the average face texture as in the learning of the first face parameter.

Then, the similarity between the second facial parameter and the first parameters pre-learned and stored for each facial expression is calculated (S207), so that the facial expression of the first facial parameter having the greatest similarity with the second facial parameter is obtained as the second face. It is estimated by the facial expression for the parameter (S208).

Here, the calculation of the similarity between the first face parameter and the second face parameter may be calculated by comparing the distribution of the plurality of first face parameters learned for one expression with the distribution of the second face parameters.

Here, the Gaussian distribution may be applied to the distributions of the first face parameter and the second face parameter. That is, the first Gaussian distribution for the first face parameter may be calculated, the second Gaussian distribution for the second face parameter may be calculated, and the similarity may be calculated by the distance difference between the first Gaussian distribution and the second Gaussian distribution. At this time, it is determined that the first face parameter having the smallest distance difference has the largest similarity with the second face parameter.

When the facial expression of the current user is estimated in the above process, the preset control according to the estimated facial expression is performed on the image content. Here, the control of the image content according to the estimated facial expression may be performed by changing at least one color of an object and a background included in the image content according to the estimated facial expression.

Referring to FIG. 9, for example, when the image content is a cartoon, the background of a character, which is an object, may be changed according to an estimated facial expression. 9 illustrates an example in which the first facial parameter is learned about four types of facial expressions such as pleasure, sadness, anger, and fear. When the facial expression is assumed to be pleasure, the background is changed to yellow or sadness. In the case of changing the background to a blue color, or in the case of anger or fear, the control of the video content may be performed by lowering the brightness of the background.

In addition, FIG. 10 (b) shows an example in which the color of a character is changed in the case of an object of image content, that is, a cartoon, according to the estimated facial expression. The control of the image content according to the estimated facial expression according to the present invention may be performed by changing the shape of the object of the image content, as shown in FIG.

9 and 10 illustrate an example of controlling image content according to a recognized facial expression, and the present invention is not limited to the examples illustrated in FIGS. 9 and 10.

Although some embodiments of the invention have been shown and described, it will be apparent to those skilled in the art that modifications may be made to the embodiment without departing from the spirit or spirit of the invention. . The scope of the alias will be defined by the appended claims and their equivalents.

1 and 2 are views for explaining a face recognition method and a control method of the robot for viewing image content according to the present invention,

3 to 7 are diagrams for explaining a process of extracting a face region and estimating a pose in a face recognition method and a control method of a robot for viewing image content according to the present invention;

8 is a diagram illustrating an embodiment of a robot for image content application to which a face recognition method and a control method for a robot for image content viewing according to the present invention are applied;

9 and 10 are diagrams illustrating examples of image contents in which a control based on a facial expression estimated through a face recognition method and a control method of a robot for viewing image contents according to the present invention is performed.

Claims (8)

(a) extracting a face region from an input image to estimate a pose of the extracted face region; (b) extracting a face shape and a face texture of the extracted face area when the pose of the extracted face area is estimated to stare at the front; (c) warping the extracted face shape and face texture to a predetermined average shape to generate an average face shape and an average face texture; (d) Retinex processing the average face shape and the average face texture; (e) performing steps (a) to (d) on a plurality of facial expressions, respectively, and performing the Retinex-treated average face shape and the Retinex treatment on each of the facial expressions; Learning first facial parameters for each facial expression based on the averaged facial textures; (f) Generating a second face parameter based on a Retinex-treated average face shape and average face texture generated by performing steps (a) to (d) on the input image. Making a step; (g) estimating a facial expression for the second facial parameter as one of the first facial parameters having the greatest similarity with the second facial parameter. The method of claim 1, The learning of the first face parameter and the generation of the second face parameter are learned and generated by applying an AAM (Active Appearance Model) to the Retinex-treated average face shape and the average face texture. Facial expression recognition method characterized in that. The method of claim 2, And a Principal Component Analysis (PCA) technique is applied to the average face shape and the average face texture when the AAM is applied. The method of claim 2, Step (g) (g1) calculating a first Gaussian distribution for the first facial parameter; (g2) calculating a second Gaussian distribution for the second facial parameter; (g3) calculating a distance difference between the first Gaussian distribution and the second Gaussian distribution as the similarity. The method according to any one of claims 1 to 4, In step (a), (a1) determining pixels corresponding to human skin through colors in the input image; (a2) determining whether a ratio of the pixels meets a threshold value in a sub-window of a predetermined size of the image; (a3) determining whether an image is a face image in the subwindow through an Adaboost algorithm when the pixels clustered in the subwindow meet a threshold; (a4) if the determination result is a face image in the sub-window, setting the sub-window as the first face region sub-window; (a5) The manifold may be classified and stored according to the face poses by classifying and storing the plurality of first face area sub-windows having various face poses which are determined by repeating steps (a1) to (a4). Forming a space; (a6) performing a step (a1) to (a3) to set a corresponding subwindow as a second face region subwindow when it is a face image; (a7) comparing the pixel data of the first face region sub-window with the pixel data of the first face region sub-window stored in the manifold space and comparing the pixel data of the first face region sub-window to the one closest to the pixel data of the second face region sub-window. Estimating a pose of the region. Displaying image content on a display unit; Photographing a person located in front of the display unit; Estimating a facial expression of a photographed person by applying the facial expression recognition method according to claim 5 to the photographed image; And changing at least one color of an object and a background included in the image content according to the estimated facial expression. The method of claim 6, And controlling the shape of the object according to the estimated facial expression. The method of claim 6, The video content control method of the robot for viewing video content, characterized in that it comprises a cartoon content.

Images