KR101751417B1 - Apparatus and Method of User Posture Recognition - Google Patents

Abstract

The present invention relates to an apparatus and a method for recognizing a posture or a posture of a user.
The user orientation recognition apparatus according to the present invention includes a motion region division unit for generating a difference image from a plurality of frame images and dividing a motion region in which motion is present in the frame image based on the generated difference image; A skin candidate region dividing unit for dividing a skin candidate region of the user in the frame image; A skin region detecting unit for detecting, as a skin region, a region corresponding to the skin candidate region and the motion region in the frame image; And a user attitude recognition unit for analyzing the skin region in the frame image to recognize the attitude or gesture of the user.

Description

[0001] Apparatus and Method for User Posture Recognition [

The present invention relates to an apparatus and a method for recognizing a posture or a posture of a user.

Various technologies exist as a technique of recognizing a user's posture or posture. For example, a method of recognizing the hand posture among the POSO recognition methods extracts the SIFT feature from the input image and distinguishes the hand motion by comparing with the SIFT features appearing in the predefined hand motion. However, the POSGee recognition method using the SIFT feature has a limitation in that the background must be monotonous because the recognition rate increases when similar feature points appear in the background area of the user.

Also, unlike the feature-based method, there is a method of recognizing a user's posture and posture based on skin color division. The skin color segmentation based method divides the skin color using a wide range of threshold values for the color channels of the input image. This method is an easy approach to skin color segmentation, but objects with skin color existing in the background are likely to be mistakenly detected by the user.

(Non-Patent Document 0001) C. Wang, and K. Wang, "Hand Posture Recognition Using Adaboost with SIFT for Human Robot Interaction, " Proc. of Int. Conf. on Advanced Robotics, Vol. 370, pp. 317-329, 2008.

(Non-Patent Document 0002) A. Just, Y. Rodriguez, and S. Marcel, "Hand Posture Classification and Recognition Using the Modified Census Transform, " of IEEE Int. Conf. on Automatic Face and Gesture Recognition, pp. 351-356, Apr. 2006.

The present invention is to provide a user attitude recognition method and an apparatus therefor that can more accurately recognize a user's attitude or posture.

According to an aspect of the present invention, there is provided a user gesture recognition apparatus for generating a difference image from a plurality of frame images, and dividing a motion region in which motion is present in the frame image, based on the generated difference image A motion region dividing section; A skin candidate region dividing unit for dividing a skin candidate region of the user in the frame image; A skin region detecting unit for detecting, as a skin region, a region corresponding to the skin candidate region and the motion region in the frame image; And a user attitude recognition unit for analyzing the skin region in the frame image to recognize the attitude or gesture of the user.

In one embodiment, the motion region dividing unit may include: a difference image generating unit that generates the difference image according to a difference of image signal values in each pixel between the frame images; And a motion region detection unit for detecting and dividing the motion region based on a magnitude of a video signal value of the difference image.

In one embodiment, the motion area detector may detect the motion area according to a comparison result obtained by comparing a video signal value of the difference image with a predetermined threshold value.

In one embodiment, the motion region detector may set the threshold according to an average of magnitudes of differences between the plurality of frame images and the reference frame.

In one embodiment, the difference image generation unit generates at least two difference images between at least three frame images, and the motion area detection unit compares the image signal value of the difference image with the threshold value for each pixel And a motion generation determination value for each pixel is set according to a result of the comparison, and the motion region is detected based on the motion generation determination value set for each of the plurality of difference images.

In one embodiment, the motion area detection unit may calculate an average value of the motion occurrence determination values set for each of the plurality of difference images, and detect an area having the calculated average value at least a predetermined size as the motion area can do.

In one embodiment, the skin candidate region dividing unit may extract the skin color of the user in the frame image, and may divide the skin candidate region in the frame image based on the extracted skin color.

In one embodiment, the skin candidate region dividing unit may include: a face detecting unit that detects the face of the user in the frame image; A skin color extracting unit for extracting skin color of the user from the detected face; And a skin color region dividing unit that divides a skin color region corresponding to the extracted skin color in the frame image to set the skin color region as the skin candidate region.

In one embodiment, the face detector may detect the user's face using a Haar-like feature or a face detection algorithm based on a LBP (Local Binary Pattern) feature.

In one embodiment, the skin color extracting unit performs image segmentation in an image block corresponding to the face detected in the frame image, and extracts, based on the image signal of the area divided into the face of the user according to the image segmentation, The color can be determined.

In one embodiment, the skin color region dividing unit performs image segmentation on the frame image based on the skin color, and divides the region divided into regions corresponding to the skin color into the skin color region as a result of the image segmentation You can decide.

In one embodiment, the image segmentation is a segmentation using a mean-shift technique.

In one embodiment, the skin candidate region dividing unit may divide an area having a video signal value within the skin color threshold in the frame image into the skin candidate region, based on a predetermined skin color threshold.

In one embodiment, the skin region detecting unit may detect an area in which the distance between the skin candidate region and the motion region is within a predetermined distance, as the skin region.

In one embodiment, the skin region detecting unit may detect a region overlaid between the skin candidate region and the motion region as the skin region.

In one embodiment, the user posture recognizing unit may recognize the posture of the user according to the location of the skin region in the frame image.

In one embodiment, the user posture recognizing unit can recognize the gesture of the user by analyzing the image signal of the skin region.

According to an aspect of the present invention, there is provided a user gesture recognition method for generating a difference image from a plurality of frame images and dividing a motion area in which motion exists in the frame image based on the generated difference image A motion region segmentation step A skin candidate region dividing step of dividing a skin candidate region of the user in the frame image; A skin region detection step of detecting, as a skin region, a region corresponding to the skin candidate region and the motion region in the frame image; And a user attitude recognition step of recognizing the user's attitude or gesture by analyzing the skin area in the frame image.

In one embodiment, the motion region segmentation step may include: a difference image generation step of generating the difference image according to a difference of image signal values in each pixel between the frame images; And a motion region detecting step of detecting and dividing the motion region based on a magnitude of a video signal value of the difference image.

In one embodiment, the skin candidate region segmentation step may include: a face detection step of detecting the face of the user in the frame image; Extracting a skin color of the user from the detected face; And a skin color region dividing step of dividing a skin color region corresponding to the extracted skin color in the frame image to set the skin color region as the skin candidate region.

The apparatus and method for recognizing a user's posture according to the present invention more accurately divides a skin region in consideration of a movement of a user in recognizing a user's posture using a divided skin region, The user's posture can be recognized.

1 is a block diagram of a user orientation recognition apparatus according to an embodiment of the present invention.
FIG. 2 is a detailed block diagram of the motion area division unit 100. FIG.
FIG. 3 is a reference diagram for explaining the operation of the motion area division unit 100. FIG.
FIG. 4 is a detailed block diagram of the skin candidate region dividing unit 200. FIG.
FIG. 5 is a reference diagram for explaining the operation of the motion region division unit 100, the skin candidate region division unit 200, and the skin region detection unit 300.
FIG. 6 is a reference diagram for explaining the operation of the user orientation recognition unit 400. FIG.
7 is a flowchart of a method of recognizing a user's attitude according to another embodiment of the present invention.
8 is a detailed flowchart of the motion area segmentation step S100.
9 is a detailed flowchart of the skin candidate region dividing step (S200).
FIG. 10 is a flowchart of a viewer voting method in a broadcast system according to another embodiment of the present invention.

Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings. In the drawings, the same reference numerals are used to designate the same or similar components throughout the drawings. In the following description of the present invention, a detailed description of known functions and configurations incorporated herein will be omitted when it may make the subject matter of the present invention rather unclear. In addition, the preferred embodiments of the present invention will be described below, but it is needless to say that the technical idea of the present invention is not limited thereto and can be variously modified by those skilled in the art.

In the present invention, in order to solve the problem of mistaken expression when objects having a similar color to the user's skin are distributed in the background, the background does not move in the frame image photographed within a predetermined time, The present invention provides a method of dividing a user's motion area in a difference image between frame images and fixing a skin area corresponding to the motion area as a final skin area.

1 is a block diagram of a user orientation recognition apparatus according to an embodiment of the present invention.

The user orientation recognition apparatus according to an embodiment of the present invention may include a motion region division unit 100, a skin candidate region division unit 200, a skin region detection unit 300, and a user orientation recognition unit 400 .

Here, the user attitude recognizing apparatus according to the present invention may be implemented as a computer program having a program module that performs a part or all of the functions of a part or all of the constituent elements selectively combined and combined in one or a plurality of hardware . In addition, each component may be implemented as a single independent hardware or included in each hardware as needed. The user orientation recognition apparatus according to the present invention may be implemented as a software program and operated on a processor or a signal processing module, or may be implemented in hardware to be included in various processors, chips, semiconductors, devices, and the like. Also, the user orientation recognition apparatus according to the present invention may be included in various embedded systems or devices and operated in the form of hardware or software modules. Preferably, the user orientation recognition device according to the present invention may be implemented in a TV device, a notebook computer, a mobile phone, or a server device connected to the network.

The motion region division unit 100 generates a difference image from a plurality of frame images, and divides the motion region in which the motion exists in the frame image, based on the generated difference image.

The skin candidate region dividing unit 200 divides a user's skin candidate region in the frame image.

The skin region detecting unit 300 detects a region corresponding to the skin candidate region and the motion region in the frame image as a skin region.

The user's posture recognizing unit 400 analyzes the skin area in the frame image and recognizes the user's posture or gesture.

Hereinafter, the operation of the motion area division unit 100 will be described in more detail.

The motion region division unit 100 generates a difference image from a plurality of frame images, and divides the motion region in which the motion exists in the frame image, based on the generated difference image. To this end, the user orientation recognition apparatus according to the present invention can photograph a plurality of frame images using a camera and acquire them, or receive frame images taken from the outside.

FIG. 2 is a detailed block diagram of the motion area division unit 100. FIG.

The motion region division unit 100 may include a difference image generation unit 110 and a motion region detection unit 120.

FIG. 3 is a reference diagram for explaining the operation of the motion area division unit 100. FIG.

As described below, a difference image is generated for a plurality of frame images including the frame image shown in FIG. 3A, and the motion region M can be detected and divided as shown in FIG. 3B.

First, the difference image generation unit 110 generates the difference image according to a difference between image signals in each pixel between the frame images. Here, the difference image may be an image generated according to a difference between frame images corresponding to different times. For example, the signal values of the respective image coordinates of the difference image may be set according to the difference values of the image signals between the frame images at different times in the same image coordinates. For example, the video signal value of the difference image may be set to be in accordance with the difference value of the video signal between the frame images calculated as above. Therefore, when the user moves, the image signal value of the moving image in the difference image becomes larger than a predetermined reference value, whereas in the case of the moving image, the image signal value of the difference image becomes relatively small or close to zero. Therefore, the area where the user's motion occurs can be recognized using the difference image generated as described above.

Here, the difference image generation unit 110 may generate at least two or more difference images among at least three or more frame images. That is, a plurality of difference images can be generated between a plurality of frame images. For example, if there are frame images such as F_0, F_2, F_3, .... F_N according to the time, a difference image is generated between adjacent frame images to generate DIFF_1, DIFF_2, ... DIFF_N difference images .

The motion region detection unit 120 detects and divides the motion region based on the magnitude of the video signal value of the difference image. In one embodiment, the motion region detection unit 120 can detect, as a motion region, an area in which the magnitude of the video signal in the difference image is larger than a predetermined threshold value.

Here, since the difference image has a video signal value of 0 or more in all cases in which the image signal value of the pixel is different between the frame images, the difference image can be generated by being sensitive to minute changes. Also, when the image signal value changes in the entire image due to a change in illumination or other environment or a change in the image acquisition sensor, a difference image having a relatively large value is generated in the entire image, and a motion region It may be difficult to detect it. Accordingly, the motion region detector 120 according to the present invention has the effect of more accurately detecting the motion region through the method of detecting the motion region using the calculated threshold as will be described in detail below.

Here, the motion region detector 120 may detect the motion region according to a comparison result obtained by comparing the video signal value of the difference image with a predetermined threshold value. At this time, the motion region detecting unit 120 according to the present invention calculates a threshold considering the variance value of the video signal, and can more accurately detect the motion region using the threshold.

Here, in order to obtain a threshold value, the motion region detection unit 120 may set the threshold value according to an average of magnitudes of differences between a plurality of the frame images and the reference frame. Wherein the reference frame may be selected from among a plurality of the frames. For example, the reference frame may be selected as a first frame, a last frame, or a middle frame among a plurality of frames. Alternatively, the reference frame may be generated as an average of a plurality of frames, or may be a frame generated by filtering a plurality of frames.

In one embodiment, the threshold may be calculated as: " (1) "

Here, a is the threshold value, N is the number of frames to obtain a difference image with the reference frame, p _i is the i-th frame among the plurality of frames, and p is the reference frame. In this case, when the reference frame is selected as one of the plurality of frames, p _i may be frames other than the reference frame, and the index i may be allocated accordingly.

For example, in the case of detecting a motion area from N + 1 frames, if the reference frame p is set to the 0th frame, pi can be 1 to N frames. It is also possible to set another frame as a reference frame other than the 0 < th > frame as needed. And the frame other than the reference frame may be pi.

Here, the motion region detection unit 120 compares the image signal value of the difference image with the threshold value for each pixel, sets a motion generation determination value for each pixel according to the comparison result, The motion area can be detected based on the occurrence judgment value. At this time, the motion region detection unit 120 may calculate an average value of the motion occurrence determination values set for each of the plurality of difference images, and may detect an area having the calculated average value of at least a predetermined size as the motion region have. Here, the predetermined size for comparing with the average value may be set as needed.

For example, the per-pixel motion generation determination value can be calculated by the following equation (2).

Where α is the threshold, and Diff _t (i, j) is the t-th difference image, (i, j) are the coordinates of the image, MD _t (i, j) is a motion generated is determined corresponding to the t-th difference image Value.

The motion area M can be detected according to an average value of motion generation judgment values calculated as shown in the following equation (3).

Next, the operation of the skin candidate region division unit 200 will be described in more detail.

The skin candidate region dividing unit 200 divides a user's skin candidate region in the frame image. Here, the skin candidate region corresponds to the skin of the user and is primarily detected and divided into regions. Here, the skin candidate region that is primarily divided is used to detect a more accurate skin region together with the motion region divided in the motion region dividing unit 100 in the skin region detecting unit 300, which will be described in more detail below.

In one embodiment, the skin candidate region dividing unit 200 may divide an area having an image signal value within the skin color threshold in the frame image into the skin candidate region, based on a predetermined skin color threshold. However, in the case of using the predetermined skin color threshold value, there is a limit to increase the possibility of erroneously dividing the skin candidate region in the image because the skin color is different according to the race and genetic characteristics of the user.

For this, the skin candidate region dividing unit 200 according to the present invention detects a user in a frame image, extracts skin color from the detected user, and divides the skin candidate region based on the detected skin color, instead of using a predetermined fixed skin color Can be used. The skin candidate region dividing unit 200 may extract the skin color of the user from the frame image and divide the skin candidate region in the frame image based on the extracted skin color. Here, in order to more accurately extract the user's skin color, the skin candidate region dividing unit 200 can detect the user's face in the image and extract the color of the detected face into skin color. The face of the user is likely to be covered by other types of clothes depending on the type of clothes worn by the user. However, in general, when the face is extracted and the color is used as described above, The skin color of the user existing in the image can be extracted more accurately.

FIG. 4 is a detailed block diagram of the skin candidate region dividing unit 200. FIG.

The skin candidate region dividing unit 200 may include a face detecting unit 210, a skin color extracting unit 220, and a skin color region dividing unit 230.

The face detecting unit 210 detects the face of the user in the frame image.

In one embodiment, the face detector 210 may detect the user's face using a face detection algorithm based on Haar-like feature or LBP (Local Binary Pattern) feature. For example, the face detection unit 210 detects' P. Viola, and M. Jones, "Rapid Object Detection using a Boosted Cascade of Simple," Proc. of IEEE Conf. face can be detected from the frame image using the Haar-like feature using the technique described in 'On Computer Vision and Pattern Recognition, 2001.' Ahonen, A. Hadid, and M. Pietikainen, "Face Recognition with Local Binary Patterns: Application to Face Recognition," IEEE Trans. on Pattern Analysis & Machine Intelligence, Vol. 28, No. 12, pp. 2037-2041, Dec. 2006. ', it is possible to detect faces based on the LBP feature.

In one embodiment, the face detection unit 210 may detect faces using both the Haar-like feature-based method and the LBP feature-based method to more reliably detect faces in various illumination environments. For example, an intersection or a union of face regions detected in both methods can be detected as a face. The face detection method using the Haar-like feature generally has high accuracy and fast detection speed, but the detection performance is low in a dark environment such as a backlight indoor environment. In this environment, the face detection method using the LBP feature robust to the illumination change can detect the face more stably. In the present invention, both the Haar-like feature-based method and the face detection method using the LBP feature can be used to perform stable face detection in various illumination environments.

In addition, the face detecting unit 210 can detect a face in a frame image using various algorithms used in the field of image recognition. Here, the face detecting unit 210 may detect a face by analyzing a frame image and using predetermined feature information. Here, the face detecting unit 210 extracts various existing types of features from the image to detect the face, and can perform face detection using the extracted features. For example, the face detection unit 210 may extract and use various features such as edge characteristics, corner characteristics, LoG (Laplacian of Gaussian), and DoG (Difference of Gaussian). Here, the face detection unit 210 may include a face detection technology using various existing feature description methods including Scale-invariant feature transform (SIFT), Speed Up Robust Features (SURF), and Histogram of Oriented Gradients Can be used. Alternatively, the face detecting unit 210 may detect and recognize a face by comparing the template image and the frame image. In addition, the face detecting unit 210 can detect a face in the image using a classifier previously learned based on the feature or template extracted as described above. The classifier used here may be an SVM machine or a classifier using various other conventional classification functions. The abnormal face detection unit 210 may detect a face using various known face detection techniques in addition to the method exemplified above, and the operation of the face detection unit 210 is not limited to the above-described techniques .

The skin color extracting unit 220 extracts the skin color of the user from the detected face.

Here, the skin color extracting unit 220 may extract a region corresponding to a color range within a predetermined threshold value preset in the detected face as a facial region, and extract the image signal of the extracted region as skin color. Here, the skin color extracting unit 220 can extract the skin color by filtering the image signal of the extracted region. For example, the skin color extracting unit 220 may extract the skin color by median filtering the image signal of the extracted region, or may extract the average value of the image signal into skin color. Alternatively, the skin color extracting unit 220 may calculate an average using the values obtained by removing the outlier using an average or variance in an image signal, or extract a skin color (e.g., an intermediate value) by extracting a skin color value.

Here, the skin color extracting unit 220 performs image segmentation in the image block corresponding to the face detected in the frame image, and extracts the skin color according to the image signal of the area divided into the face of the user according to the image segmentation, Can be determined.

Next, the skin color region dividing unit 230 divides a skin color region corresponding to the extracted skin color in the frame image, and sets the skin color region as the skin candidate region. Herein, the skin color region dividing unit 230 may sort the pixels having the skin color in the frame image into the skin color regions. In one embodiment, the skin color division unit 230 may select pixels having a video signal value within a predetermined color range based on the skin color in a frame image, and may divide the pixels into skin color regions. Here, the video signal may be a signal value in various color spaces such as RGB, YCbCr, and HSV, which is the same in other parts of the present invention.

At this time, an image segmentation method can be used to more accurately divide the skin color region. In one embodiment, the skin color region dividing unit 230 performs image segmentation on the frame image based on the skin color, and divides the region divided into regions corresponding to the skin color as a result of the image segmentation, It can be determined as a color area. At this time, the image may be segmented using a color-based image segmentation method, and a region corresponding to the skin color among the divided regions may be set as a skin color region. Here, the region corresponding to the skin color may be a region where the difference between the image signal value of the divided region and the skin color is within a predetermined distance.

Here, the skin color region dividing unit 230 performs image segmentation based on the skin color based on the Region Growing technique, and as a result, can determine the divided region corresponding to the skin color as the skin color region .

In one embodiment, image segmentation can be performed by segmentation using the mean-shift technique. For example, the skin color region dividing unit 230 may be referred to as' W. Tao, H. Jin, and Y. Zhang, "Color image segmentation based on mean shift and normalized cuts," IEEE Trans. on Systems, Man and Cybernetics, Vol. 37, No. 5, pp. 1382-1389, Oct. 2007. This paper proposes an image segmentation method based on the technique described in < RTI ID = 0.0 > 2007. < / RTI >

Next, the operation of the skin region detection unit 300 will be described in more detail.

The skin region detecting unit 300 detects a region corresponding to the skin candidate region and the motion region in the frame image as a skin region.

When a skin candidate region 200 divided in the skin candidate region 200 is set as a skin region and a user's posture is recognized using the skin candidate region 200, a background region having a color similar to the face color is erroneously detected as a skin candidate region It may happen that the user's attitude is mistakenly recognized. In the present invention, in order to improve the user's attitude recognition error due to such erroneous detection, only the region corresponding to the motion region divided by the motion region division unit 100 among the skin candidate regions divided according to the face color And the remaining area is determined to be a background area and is removed.

Usually, the background in the image is likely to be stopped, while the user is likely to move. Also, at the moment when the user perceives himself / herself to be stopped, there is a high possibility that the user may move finely due to respiration, heartbeat, and muscle movement. Therefore, according to the motion or the fine movement of the user, the portion corresponding to the user's body moves in a plurality of frame images captured for a predetermined time. In the user orientation recognition apparatus according to the present invention, the motion region dividing unit 100 detects the motion region using the difference image as a motion region and divides the motion region into motion regions, and the skin candidate region dividing unit 200 divides There is an effect that the skin region corresponding to the actual skin of the user can be detected more accurately by detecting the skin region by using the motion candidate region and the skin candidate region which are divided into the region corresponding to the skin.

Here, the skin region detecting unit 300 can detect a region within a predetermined distance from each region or each region as a skin region according to the correspondence relationship between the skin candidate region and the motion region. In one embodiment, the skin region detection unit 300 may detect an area having a distance between the skin candidate region and the motion region within a predetermined distance as the skin region. Here, the predetermined distance is a pixel distance that can be set as needed.

In one embodiment, the skin region detection unit 300 may detect an area overlaid between the skin candidate region and the motion region as the skin region. For example, the skin region detecting unit 300 can perform a logical product operation between the skin candidate region and the motion region to detect the skin region by selecting the skin region. Here, it is needless to say that various other methods can be used as a technique for selecting overlapping regions between the skin candidate region and the motion region.

FIG. 5 is a reference diagram for explaining the operation of the motion region division unit 100, the skin candidate region division unit 200, and the skin region detection unit 300.

When a frame image and a series of frame images as shown in FIG. 5A are given, the motion region M can be divided as shown in FIG. 5B, and the skin candidate region CS can be divided as shown in FIG. 5C. By using the motion region M of FIG. 5B and the skin candidate region CS of FIG. 5C, a more accurate skin region S can be detected as shown in FIG. 5D.

Next, the operation of the user orientation recognition unit 400 will be described in more detail.

The user's posture recognizing unit 400 analyzes the skin area in the frame image and recognizes the user's posture or gesture.

Here, the user's posture recognizing unit 400 can recognize the posture of the user according to the position of the skin region in the frame image. For example, the user attitude recognition unit 400 can recognize the user's attitude according to the position of the skin region based on the detected face position. Here, the user's posture can be defined and stored as skin region location information, and the user's posture recognizing unit 400 compares the recognized skin region location with predefined skin region location information, It is possible to recognize what the posture of the user is.

FIG. 6 is a reference diagram for explaining the operation of the user orientation recognition unit 400. FIG.

For example, when the user's posture is predefined as a left arm, a right arm, and a arm lifting posture, each posture can be defined as information on whether a skin area exists in a specific direction and position from the face position of the user. For example, as shown in Fig. 6, when three zones are previously classified and the ratio of the skin zone to the one zone based on the face position of the user is high, the right hand zone and the skin zone is located in the two zones 6A). When the ratio of the positions of the skin regions in the first and second compartments is similar to a predetermined reference, the user's posture is predefined according to the skin region position arrangement information with both hands (Fig. 6B) can do. The user posture recognizing unit 400 can recognize the user's posture by comparing the skin region information detected by the skin region detecting unit 300 with the predefined information as described above.

Here, the user posture recognizing unit 400 may recognize the gesture of the user by analyzing the image signal of the skin region. Here, the gesture recognition method can use various gesture recognition methods using the existing user's body. Also, the user orientation recognition unit 400 may recognize the operation of the user according to the movement of the skin region between the plurality of frame images.

7 is a flowchart of a method of recognizing a user's attitude according to another embodiment of the present invention.

The method of recognizing a user attitude according to another embodiment of the present invention includes a motion region segmentation step S100, a skin candidate region segmentation step S200, a skin region detection step S300, and a user attitude recognition step S400 . The user orientation recognition method according to the present invention can operate in the same manner as the user orientation recognition device according to the present invention described in detail with reference to FIGS. 1 to 6. A description thereof will be omitted and a brief description will be given mainly of the main operation.

The motion region segmentation step S100 generates a difference image from a plurality of frame images, and divides the motion region in which the motion exists in the frame image based on the generated difference image.

A skin candidate region dividing step (S200) divides the skin candidate region of the user in the frame image.

The skin region detection step S300 detects a skin region corresponding to the skin candidate region and the motion region in the frame image.

The user's posture recognition step (S400) analyzes the skin area in the frame image to recognize the user's posture or gesture.

8 is a detailed flowchart of the motion area segmentation step S100.

The motion region segmentation step S100 may include a difference image generation step S110 and a motion region detection step S120.

The difference image generation step S110 may generate the difference image according to a difference of image signal values in each pixel between the frame images.

The motion region detection step (S120) can detect and divide the motion region based on the magnitude of the video signal value of the difference image.

9 is a detailed flowchart of the skin candidate region dividing step (S200).

The skin candidate region dividing step S200 may include a face detecting step S210, a skin color extracting step S220, and a skin color region dividing step S230.

The face detection step (S210) detects the face of the user in the frame image.

The skin color extraction step S220 extracts the skin color of the user from the detected face.

The skin color region dividing step S230 divides the skin color region corresponding to the extracted skin color in the frame image and sets the skin color region as the skin candidate region.

Another embodiment of the present invention may be a viewer voting method in a broadcasting system.

FIG. 10 is a flowchart of a viewer voting method in a broadcast system according to another embodiment of the present invention.

A viewer voting method in a broadcasting system according to another embodiment of the present invention includes a server receiving an image photographed using a terminal used by a plurality of viewers (S1000), a server analyzing the photographed image, A step of recognizing the user's attitude (S2000), a step of determining a vote number according to a result of the server recognizing the attitude of each user, and integrating the result (S3000).

In step S1000, the server can receive frame images captured by a terminal (TV, notebook, PC) owned by each user. Herein, the user may view a broadcast on the terminal and take a predefined posture in which one of a plurality of choices is selected according to contents to be explained to the user in broadcasting.

The step S2000 may be performed according to the motion region segmentation step S100, the skin candidate region segmentation step S200, the skin region detection step S300, and the user attitude recognition step S400 described in detail with reference to FIG. , And each step can operate in the same manner as described above. By performing the steps S100 to S400, the user's posture can be recognized for each image transmitted from each user's terminal in step S2000.

Next, in step S3000, it is possible to determine which one of the selected options is selected by the user according to the recognized result, and to consolidate the votes determined for each user.

It is to be understood that the present invention is not limited to these embodiments, and all elements constituting the embodiment of the present invention described above are described as being combined or operated in one operation. That is, within the scope of the present invention, all of the components may be selectively coupled to one or more of them.

In addition, although all of the components may be implemented as one independent hardware, some or all of the components may be selectively combined to perform a part or all of the functions in one or a plurality of hardware. As shown in FIG. In addition, such a computer program may be stored in a computer readable medium such as a USB memory, a CD disk, a flash memory, etc., and read and executed by a computer to implement an embodiment of the present invention. As the recording medium of the computer program, a magnetic recording medium, an optical recording medium, a carrier wave medium, and the like can be included.

Furthermore, all terms including technical or scientific terms have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs, unless otherwise defined in the Detailed Description. Commonly used terms, such as predefined terms, should be interpreted to be consistent with the contextual meanings of the related art, and are not to be construed as ideal or overly formal, unless expressly defined to the contrary.

It will be apparent to those skilled in the art that various modifications, substitutions and substitutions are possible, without departing from the scope and spirit of the invention as disclosed in the accompanying claims. will be. Therefore, the embodiments disclosed in the present invention and the accompanying drawings are intended to illustrate and not to limit the technical spirit of the present invention, and the scope of the technical idea of the present invention is not limited by these embodiments and the accompanying drawings . The scope of protection of the present invention should be construed according to the following claims, and all technical ideas within the scope of equivalents should be construed as falling within the scope of the present invention.

100: Motion area division unit
200: skin candidate region partitioning unit
300: skin area partitioning part
400:
110: difference image generating unit
120:
210:
220: skin color extracting unit
230: skin color area division part
S100: Motion area segmentation step
S200: skin candidate region dividing step
S300: skin area segmentation step
S400: user attitude recognition step
S110: Difference image generation step
S120: Motion area detection step
S210: face detection step
S220: skin color extraction step
S230: skin color gamut splitting step

Claims (20)

A motion region division unit for generating a difference image from a plurality of frame images and dividing a motion region in which motion is present in the frame image based on the generated difference image;
A skin candidate region dividing unit for dividing a skin candidate region of the user in the frame image;
A skin region detecting unit for detecting, as a skin region, a region corresponding to the skin candidate region and the motion region in the frame image; And
And a user orientation recognition unit for analyzing the skin region in the frame image to recognize the user's attitude or gesture,
Wherein the motion-
A difference image generation unit for generating the difference image according to a difference of image signal values in each pixel between the frame images; And
And a motion region detection unit for detecting and dividing the motion region based on a magnitude of a video signal value of the difference image. delete The method according to claim 1,
Wherein the motion region detection unit detects the motion region according to a comparison result obtained by comparing a video signal value of the difference image with a predetermined threshold value. The method of claim 3,
Wherein the motion region detection unit sets the threshold value according to an average of magnitudes of differences between the plurality of frame images and the reference frame. The method of claim 3,
Wherein the difference image generation unit generates at least two difference images between at least three frame images,
Wherein the motion region detection unit compares the image signal value of the difference image with the threshold value for each pixel, sets a motion generation determination value for each pixel according to the comparison result, and generates the motion generation determination value And detects the motion area on the basis of the motion information. 6. The method of claim 5,
Wherein the motion area detection unit calculates an average value of the motion occurrence determination values set for each of the plurality of difference images and detects an area having the calculated average value at least a predetermined size as the motion area User orientation recognition device. The method according to claim 1,
Wherein the skin candidate region dividing unit extracts the skin color of the user from the frame image and divides the skin candidate region from the frame image based on the extracted skin color. The apparatus according to claim 1,
A face detector for detecting the face of the user in the frame image;
A skin color extracting unit for extracting skin color of the user from the detected face; And
And a skin color region dividing unit for dividing a skin color region corresponding to the extracted skin color in the frame image to set the skin color region as the skin candidate region. 9. The method of claim 8,
Wherein the face detection unit detects the user's face using a face detection algorithm based on Haar-like feature or LBP (Local Binary Pattern) feature. 9. The method of claim 8,
Wherein the skin color extracting unit performs image segmentation in an image block corresponding to the face detected in the frame image and determines the skin color according to an image signal of an area divided into the face of the user according to the image segmentation The user orientation recognition apparatus comprising: 9. The method of claim 8,
Wherein the skin color region dividing unit performs image segmentation on the frame image based on the skin color and determines a region divided into regions corresponding to the skin color as the skin color region as a result of the image segmentation, A user orientation recognition device. The method according to claim 10 or 11,
Wherein the image segmentation is a segmentation using a mean-shift technique. The method according to claim 1,
Wherein the skin candidate region dividing unit divides an area having an image signal value within the skin color threshold value in the frame image into the skin candidate area based on a preset skin color threshold value. The method according to claim 1,
Wherein the skin region detecting unit detects an area in which the distance between the skin candidate region and the motion region is within a predetermined distance as the skin region. The method according to claim 1,
Wherein the skin region detection unit detects an area overlaid between the skin candidate region and the motion region as the skin region. The method according to claim 1,
Wherein the user's posture recognizing unit recognizes the user's posture according to the location of the skin region in the frame image. The method according to claim 1,
Wherein the user posture recognizing unit recognizes a gesture of the user by analyzing a video signal of the skin region. A motion region dividing step of generating a difference image from a plurality of frame images and dividing a motion region in which motion is present in the frame image based on the generated difference image;
A skin candidate region dividing step of dividing a skin candidate region of the user in the frame image;
A skin region detection step of detecting, as a skin region, a region corresponding to the skin candidate region and the motion region in the frame image; And
And a user attitude recognition step of analyzing the skin area in the frame image to recognize the user's attitude or gesture,
Wherein the motion region segmentation step comprises:
A difference image generation step of generating the difference image according to a difference of image signal values in each pixel between the frame images; And
And a motion region detecting step of detecting and dividing the motion region based on a magnitude of a video signal value of the difference image. delete 19. The method of claim 18, wherein the skin candidate region segmentation step comprises:
A face detecting step of detecting the face of the user in the frame image;
Extracting a skin color of the user from the detected face; And
And a skin color region dividing step of dividing a skin color region corresponding to the extracted skin color in the frame image to set the skin color region as the skin candidate region.

Images