KR101447958B1 - Method and apparatus for recognizing body point - Google Patents

Method and apparatus for recognizing body point Download PDF

Info

Publication number
KR101447958B1
KR101447958B1 KR1020140051607A KR20140051607A KR101447958B1 KR 101447958 B1 KR101447958 B1 KR 101447958B1 KR 1020140051607 A KR1020140051607 A KR 1020140051607A KR 20140051607 A KR20140051607 A KR 20140051607A KR 101447958 B1 KR101447958 B1 KR 101447958B1
Authority
KR
South Korea
Prior art keywords
person
body part
image
value
point
Prior art date
Application number
KR1020140051607A
Other languages
Korean (ko)
Inventor
백재성
Original Assignee
(주)디지엔터테인먼트
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by (주)디지엔터테인먼트 filed Critical (주)디지엔터테인먼트
Priority to KR1020140051607A priority Critical patent/KR101447958B1/en
Priority to PCT/KR2014/007530 priority patent/WO2015167081A1/en
Application granted granted Critical
Publication of KR101447958B1 publication Critical patent/KR101447958B1/en

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00Image analysis

Abstract

The present invention relates to a method and an apparatus for detecting body parts. The method includes searching a person in an obtained image and getting an operation central value by applying a distance transformation matrix on the searched person image; and getting an actual central value by applying the body ratio of an actual person to the operation central value. Thereby, behaviors can be accurately and quickly recognized.

Description

TECHNICAL FIELD The present invention relates to a method and apparatus for detecting a body part,

The present invention relates to a method and apparatus for detecting a body part, and more particularly, to a method and apparatus for detecting a body part, The present invention relates to a body part detecting method and apparatus for accurately detecting a body part without recognizing a classifier and conventional body information data so that a behavior can be accurately and quickly recognized.

Recent developments in technology have made it possible for information processing devices such as computers to recognize the behavior beyond simply recording human actions. Such behavior recognition technology has recently been used in a game machine to move a character's movement from a keyboard, a joystick, Or the motion of the portable terminal is recognized by recognizing a human hand motion or the like and controls the robot to perform an operation corresponding to the motion of the robot, , And various application fields thereof have been diversified.

As a method for recognizing such a behavior, a sensor such as a GPS, an acceleration sensor, a gyroscope, a geomagnetic sensor, or the like is attached to or placed in a human body as disclosed in Patent Documents 10-2006-0122536 and 10-2005-0073313, However, since a separate sensor must be attached to a human body, a separate configuration for fixing the sensor and the person so as not to be separated from each other, or a user's attention is required Therefore, there is a problem in that the distance between the sensor and the operation unit for processing the signal received from the sensor is limited, thereby limiting the behavior and the place of the person.

Accordingly, in recent years, behavior recognition technologies that do not use such a sensor have been used. For example, a person is searched for a motion in an image obtained by using an image acquisition unit such as a camera, 10-1203121, 10-2012-0089948, 10-1281461, and 10-1307642.

Among them, the patent document 10-2012-0108728 discloses a method for imaging a moving object by a stereo camera to generate a stereo image; Calculating depth map data for each pixel in the stereo image; Extracting the moving object from the stereo image; Recognizing a pointer to be a motion recognition object in an object region of the image through image processing on the image; Performing a step of calculating depth map data and recognizing a pointer for each image frame continuously generated in the generating step to track a change in position of the pointer on a three-dimensional space; And calculating and outputting information on a three-dimensional moving direction of the pointer using the changed three-dimensional spatial position information of the tracked pointer. The three-dimensional motion recognition method using a stereo camera And an apparatus for implementing the same are disclosed.

In addition, the patent document 10-1203121 discloses a method of setting a basic position at a predetermined distance in front of a stereo camera; Capturing a moving object with the stereo camera to generate a stereo image; Calculating depth map data for each pixel in the stereo image; Extracting a region of the moving object from the stereo image; Determining whether the extracted object region belongs to the pixel region of the set basic position and whether the depth of the object belongs to the depth range of the basic position and whether the object is in the basic position; Recognizing a pointer to be a motion recognition object in an object region of the image through image processing on the image when the object is at a default position; Performing a step of calculating depth map data and recognizing a pointer for each image frame continuously generated in the generating step to track a change in position of the pointer on a three-dimensional space; And calculating and outputting information on a three-dimensional moving direction of the pointer using the changed three-dimensional spatial position information of the tracked pointer, wherein the step of setting the basic position comprises: Receiving a position setting command, photographing an object for setting a basic position with the stereo camera to generate a stereo image, and calculating depth map data; Extracting the setting object from the stereo image; And setting a pixel region including the extracted region of the setting object as a pixel region of the basic position and setting a depth range of the basic position based on a depth value of the extracted setting object region, The method of recognizing a three-dimensional motion using a stereo camera and an apparatus therefor are disclosed.

However, the above-mentioned patent publications 10-2012-0108728 and 10-1203121 recognize a three-dimensional gesture of a user by using a stereo camera using two general cameras, And the control of the two cameras is difficult.

In addition, Japanese Patent Application Laid-Open No. 10-2012-0089948 discloses a method for obtaining an MHI (Motion History Image), which is a gray image that expresses the degree of motion change in space and time, Stage 1; A gradient image extracting unit extracting a gradient image for each of x and y coordinates from the obtained MHI; A third step of extracting shape information by applying a shape context method to each of the slope images extracted by the shape information extracting unit; And a fourth step of classifying and learning the extracted type information by a SVM (Support Vector Machine) classifier. The method and system for recognizing an operation of a real-time motion recognition system using MHI morphology information are disclosed, Since it is not finding the characteristic parts such as various joints of the human body by detecting the motion by the outline of how the person moves based on the general outline of the person, it recognizes the precise movement of various parts of the body such as robot or game, It is difficult to apply it to the field of precision control.

In addition, the Japanese Patent Application No. 10-1281461 discloses an image acquiring step of acquiring an image including a body part of a user who wants to touch a specific area by an image acquiring unit; A skin region detection step of detecting a skin region in the image such that a skin region detection unit distinguishes a part of the user's body from a background region from the image; A hand region detection step of detecting a hand region by performing a labeling on each pixel in the skin region by a hand region detection unit; A noise filtering step in which a noise filtering unit removes noise in the image based on a morphology algorithm; A center-of-gravity detecting step of the center-of-gravity detecting unit detecting a center-of-gravity point in the hand region; A feature point detection step of detecting at least one feature point that intersects with the finger node in the hand region and the at least one concentric circle, wherein the feature point detection unit forms at least one concentric circle from the center of gravity of the hand region; And determining whether the touch determination unit calculates a distance between the center of gravity of the hand region and the minutiae and determining whether the user touches the touch area based on the calculation result; The present invention relates to a multi-touch input method using image analysis and a system for achieving the same. The system detects a skin region in a captured image and then detects a hand. Since the detection is performed using skin, It is difficult to detect the entire region, and there is a limit in detecting the joint region of the person to be searched. Thus, there is a problem in that it is impossible to control based on the entire human body.

In addition, the above-mentioned Patent No. 10-1307642 discloses a photographing apparatus for photographing a user by a registration of a prior application by the applicant of the present application; A body part information detection unit for reading out x, y, and z values from the 3D data of the human body photographed through the photographing unit to extract a human body part; A user detection unit for detecting a user's image and a user's hand area from a human body part extracted through the body part information detection unit; A content providing unit for providing content to be used together with the user image detected through the user detecting unit; And a control unit for combining the detected user image and the content provided through the content providing unit, wherein the control unit allows the user to select the content provided through the content providing unit, Wherein the 3D digital photographing device reads the x, y, and z values from the 3D data of the human body obtained through the photographing unit to extract a human body part A technology for detecting the entire body of a human body has been proposed as a technique for detecting a user's image from the extracted human body part and synthesizing the detected user image and contents. However, no specific method for detecting various parts of the human body has been proposed.

As described above, in the body part detection method using the conventional camera, there are problems in that various parts of the human body are required to be detected in a long time because of a large amount of calculation, and it is difficult to accurately detect various parts of the human body.

Recently, behavior awareness is used to control various electronic products. Therefore, many techniques for behavior recognition are being developed. For example, there are a distance transformation matrix for finding the center point of the object to be found, and an Omega Shape comparison technique for finding people .

Among them, the distance conversion matrix is a matrix having the closest distance from the current pixel to a pixel having a value of '0', with the largest coordinate being the center.

As an example of finding a center point using such a distance transformation matrix, a process of finding the center of a human hand is described. First, only a human hand portion is detected using a skin color, and then a center transformation matrix is applied to find a center point. (Green dot) and the palm area (blue circle).

However, since the conventional distance conversion matrix does not apply the ratio of the actual human body in the process of finding the center of the object to be searched, the center point is precisely found in the entire shape of the person rather than in the hand, There was a problem that it was difficult to find a part.

The Omega shape is a technique for recognizing an object having an omega shape in the acquired image, which is named after the shape of the contour line connecting the human head and shoulder resembles a character OMEGA [ There is an advantage in that the person can be detected even when the face is not visible, but it is not possible to detect various parts of the human body of the person who is searched only by the technique used to find the person.

As a patent related to an algorithm for finding a conventional person, Japanese Unexamined Patent Application Publication No. 10-2013-0052393 discloses an apparatus and method for detecting an object using a three-dimensional image, which uses a classifier and facilitates object separation using a depth camera, Patent Document 10-2014-0040527 As a method and apparatus for detecting body skeleton and body part information from an image, the body detection method is based on detection of a conventional local maxima (skeletal information).

The present invention requires a large number of hardware (cameras), which is a problem of methods for finding a person from a video obtained through the above-described conventional camera, and is difficult and complicated to control, has a large amount of calculation, Joints and the like can be accurately found, and it is possible to efficiently use techniques such as a distance conversion matrix and Omega shape comparison techniques, which are used for recognizing known person's behavior or searching for a known person To reduce the amount of computation and improve the recognition speed and accuracy.

In addition, by applying known techniques for behavior recognition, accurate recognition of each part of a human body and accurate behavior recognition can be performed by applying human body's actual body shape information to an arithmetic process, And other industries.

In addition, as the depth camera has recently been spread, depth information on human perception has been added, thereby providing a method of recognizing all of a face and a body without a classifier for recognizing a shape of a person, To extract body parts without existing body data.

According to another aspect of the present invention, there is provided a method for searching a human image, the method comprising: finding a person in the obtained image; And finding a new actual center point.

In addition, the above-mentioned body part detection method further includes a step of forming a distance map at an actual center point and labeling and extracting the center of the distance area to find a point for each body part.

On the other hand, the image acquisition uses a three-dimensional camera. In the process of finding a person, a point is sampled based on a pure depth value in an image obtained from a three-dimensional camera, a vector is created at the extracted point, and an omega shape ), It is judged that there is a person when the same data as the omega shape is found.

If a person is found in the previous frame during searching for a person as described above, the current frame and the depth value are compared with each other to track the person area, and the background is removed in addition to the person.

In addition, another feature of the present invention is to provide an image processing apparatus, including: an image acquisition unit 10 for acquiring an image; an image acquisition unit 10 for acquiring a human image from the image acquired from the image acquisition unit 10, And a body part information detection part (20) for searching for a point.

The body part information detection unit 20 includes a person detection unit 22 for detecting a person in the image data acquired from the image acquisition unit 10, a person tracking unit 22 for tracking only the person image detected by the person detection unit 22 24), a background separator (26) for separating the background except for the person traced by the person tracking unit (24), a calculation center value is obtained by applying a distance transformation matrix to the person image found by separating the background A distance map generator 28 for finding actual center points by applying body information therefrom, a distance map is formed at an actual center point found from the distance map generator, and the body parts (arms, legs, and head) are detected And a body separating unit 29 for outputting data corresponding thereto.

According to the present invention configured as described above, the human being is recognized through comparison with the Omega shape, the background is removed, the central predicted point is found by using the distance transformation matrix in the person who is searched, The user finds the actual center point and finds each point of the body, that is, the hands, feet, elbows, knees, and the like based on the found actual center point, so that the body part of the person can be accurately and quickly found.

Since the position of the detected body part is accurate, accurate behavior recognition becomes possible. This behavior recognition technology can be used to control other electronic devices, that is, to control the movement of the robot, or to accurately control the actions of avatars and characters Or to various industrial fields used in computer graphics.

Further, since the above-described precise control becomes possible, the above-described behavior recognition technology can be applied in the fields of medical and electronic industries requiring precise control, and the above-described behavior recognition technology can be applied So that the safe and accurate operation can be performed.

1 is a block diagram showing a body part detecting apparatus according to the present invention;
FIG. 2 is a block diagram showing the configuration of the body part information detection unit of FIG. 1;
3 is a view showing an example of depth image acquisition according to the present invention
4 is a view showing an example of object separation according to the present invention;
5 is a diagram showing an example of pointing extraction and vectorization according to the present invention;
6 is a view showing an omega shape according to the present invention
Figure 7 is a graph
8 is a view showing an example of human tracking according to the present invention;
9 is a view showing an example of an actual center value detection example
10 is a view showing a new distance map along an actual center point;
11 is a view showing a process of outputting a body part from an actual center point according to the present invention;
12 is a view showing an example of body part detection according to the present invention
13 is a flowchart showing a body part detection method according to the present invention
14 is a diagram showing an example of line detection through labeling at an actual center point
15 is a block diagram showing a control device through body part detection according to the present invention;

Hereinafter, embodiments of the present invention will be described with reference to the accompanying drawings.

A body part detecting apparatus according to the present invention will be described with reference to Fig.

1 is a block diagram showing a body part detecting apparatus according to the present invention.

As shown in FIG. 1, the body part detecting apparatus according to the present invention includes an image obtaining unit 10 for obtaining an image, an image obtaining unit 10 for detecting a person in the image obtained from the image obtaining unit 10, And a body part information detector 20 for analyzing the body part information.

A known camera capable of detecting depths such as a three-dimensional camera, a stereo camera, and RGB-D is used as the image obtaining unit 10, and the body part information detecting unit 20 detects the body part information, A separate arithmetic unit including an arithmetic unit, an input unit (keyboard, touch pad, etc.), a display unit (a display such as a monitor), a memory unit in which an arithmetic program is written, An operation unit provided in the camera 10 and a memory unit in which an operation program is recorded may be used.

The body part information detection unit 20 includes a person detection unit 22 for detecting a person in the image data obtained from the image acquisition unit 10, a person tracking unit 24 for tracking a person image found by the person detection unit 22, ), A background separator (26) for separating the background excluding the person traced by the person tracking unit (24), a center value calculation unit A distance map generator 28 for searching for an actual center point by searching for the actual center point from the distance map generator 28 and searching for an actual center point from the distance map generator 28, And a body separating unit 29 for detecting the data and outputting data corresponding thereto.

3, the human detection unit 22 acquires the depth image of FIG. 3 from the image acquisition unit 10 as shown in FIG. 2 (S110), and performs an object separation process S120 from the depth image Dimensional object as shown in FIG. 5, separates each separated object into individual objects through sampling (S130), extracts the pointing as shown in FIG. 5 and performs vectorization (S140). Then, as shown in FIG. 6, A shape similar to the shape is found (S150), and it is confirmed whether or not the person is correct.

If no person is found in this process, the upper process is repeated until the shape of the person is continuously detected (S110 to S150)

7, the distance map generating unit 28 searches for the center of the calculation using the distance conversion matrix (S320) using the image of the person separated from the background (FIG. 8) (S310) , The head position of the person is searched (S330), the x value is fixed from the calculation center value, and the center value (actual center value) is readjusted (S340) as shown in Fig. A new distance map is generated using the center value as shown in FIG. 10 (S350).

Using the new distance map thus generated, the body separating unit 29 labels the points at an arbitrary distance from the actual center point as shown in FIG. 11 (S410), and calculates the head value (S420).

Then, the arm is detected using the labeling data and the head value data in step S430, the leg is detected in step S440, and all the body parts (elbows, knees, feet, hands, etc.) If the leg is not detected, only the upper body region (hand, elbow, etc.) is detected (S450) and output.

An embodiment of the body part detecting method of the present invention using such an apparatus will be described below.

A body part detecting method according to the present invention is characterized in that a body part information detecting part finds a person from image data obtained from a camera and then finds a calculation center value by applying a distance conversion matrix to the image of a person to be searched, Detecting a new actual center point by applying a body proportion of a person, forming a distance map at the actual center point, labeling the center of the distance area to detect a line of each body part, And searching for a point that is a part of each of the points.

This will be described in more detail below with reference to FIG.

13 is a flowchart for performing the body part detection method according to the present invention.

In the body part detection method according to the present invention, the body part information detection unit reads the pure depth value from the image obtained from the camera, that is, the distance value from the camera to a specific object (person) (S401)

At this time, the 3D camera captures a moving object (e.g., a dynamic object) such as an object, a human body, an animal body, etc., and generates depth value data for the image of the captured dynamic object , And transmits the generated depth value data to the body part information detection unit.

Here, three-dimensional (3D) is used to describe (display) a geometric solid or space including X, Y, and Z axis values representing depth, width, and height. Expression), and may include a series of all three-dimensional shapes that represent a three-dimensional effect or a three-dimensional effect.

In one embodiment, the three-dimensional camera can generate the depth value data by predicting the distance from the three-dimensional camera to the dynamic object according to the intensity of the color of the dynamic object image photographed through the three-dimensional camera. At this time, the three-dimensional camera may store the distance corresponding to the color intensity of the dynamic object image photographed through the three-dimensional camera with depth value data in advance.

In another embodiment, a three-dimensional camera may be configured to display the principle that the intensity of color (i.e., saturation (color sharpness)) decreases as the distance from the three-dimensional camera to the dynamic object increases, (Or brightness) of an image of a dynamic object photographed by a three-dimensional camera is excluded, and it is possible to generate the depth value data according to the saturation of an image of a dynamic object photographed only through a three-dimensional camera have. At this time, the 3D camera may set and store depth value data corresponding to the saturation of the dynamic object image photographed through the 3D camera in advance.

The human detection unit 22 constituting the body part information detection unit samples the obtained image from top to bottom (S402), extracts a fixed object (desk, etc.) and a dynamic object, that is, a human point, (S403).

Sampling the image obtained from the step S402 from top to bottom is a process of extracting the position of the fixed object and the dynamic object, that is, the point by point, one by one in the pixel on the y axis while searching pixels from the beginning to the end of the x axis.

Then, in step S403, the extracted points are vectorized using the depth value, and it is determined whether or not an omega shape having an outline shape connecting the head and shoulders of a person is detected to recognize whether a person exists. S404)

If the point corresponding to the Omega shape is not detected in step S404, it is determined that there is no person. Then, the process returns to step S402 to repeat steps S403 to S404 to sample a new image frame from top to bottom, Points are extracted, vectorized and compared to Omega shapes.

If a point corresponding to the omega shape is detected in step S404, human tracking is performed (S405), and the human data and the image data corresponding to the background are separated through segmentation to delete the background data , And transmits it to the corresponding center anticipation point detecting unit 23 so that only the separated human data is calculated in the following distance conversion matrix, thereby greatly reducing the amount of calculation (S406)

At this time, if a person is found in the previous frame, the current frame and the depth value are compared with each other to track the human area.

Thereafter, the distance map generator 28 generates a distance map by a distance transformation matrix (S407), and searches for a center point (hereinafter referred to as an operation center value) (S408)

In order to recognize a person using such a distance transformation matrix, it is necessary to find each part of the human body (for example, a hand, a foot, an elbow, a knee, and the like) (Above the standard value), and a distance conversion matrix is used to search for the above-mentioned chest area.

However, since the calculation center value found using the distance conversion matrix is not centered on the actual ratio of the human body, the center point of the human body, that is, the center of the chest and the center predicted point found may not coincide exactly, A new center point (hereinafter referred to as an actual center point) is searched by applying the body percentage of the actual person to the center value of the true operation (S409)

That is, since the computed center value found through the above-described distance conversion matrix is not exactly located in the human chest region, the x-axis value is fixed from the calculated center value position as shown in Fig. 9, After checking the size (key), the actual center point (the chest area in the embodiment of the present invention) is calculated by using an arbitrary constant based on the confirmed key. Usually, the chest is 3 / 10 position, the actual center point is shifted by 0.3 from the head to the bottom in the human image, which is obtained by multiplying the found key by 0.3, which is an arbitrary constant, as the center point.

That is, the center point is changed by moving the y-axis value only to the position reflecting the body size while keeping the x-axis value at the previously searched center predicted point.

In this case, the constant 0.3 considering the actual body size may be recorded as a fixed constant in the program for behavior recognition so that 0.3 is always used for each behavior recognition. However, for example, a human body condition such as a child, an adult, Since the ratio is different among people, it is also possible to use an arbitrary constant other than '0.3', for example, '0.4' or the like.

The body part information detection unit may be configured to display the body part information such that the user inputs a constant for finding the actual center point of the person who is searched during the execution of the behavior recognition program, You can also use other constants at that time by displaying a constant input menu that allows the user to enter a constant through the input.

For example, a menu is displayed to select whether the person to be recognized is an adult or a child. The constants for adults and children are fixed in advance, and the input data, that is, data corresponding to an adult or a child is determined, The actual center point can be found by applying the fixed constant (for example, a fixed constant corresponding to an adult when inputting adult data, a fixed constant corresponding to an eye when inputting eye data, and the like).

In addition, as another example, the body proportion of a person is divided into eight figures, seven figures, and so on, with the head and the keys as a center, so that the user preliminarily selects and inputs a body proportion (eight figurehead, seven figurehead, Calculates the length of the head of the person searched based on the inputted body ratio, that is, the height of the head and the length of the head, calculates the height of the person sought in correspondence with the body proportion, Key to find the actual center point.

The technique of calculating the key of a person on the basis of the head length and finding the actual center point according to the head length is not limited to a case where a person is identified from the end of the head to the toe in the input image, It is useful to calculate the key by computing the remaining part of the image. At this time, the user can input the body ratio (8th, 7th, etc.) directly, but it can be fixed in advance when the program is manufactured considering the average body shape.

As another example in which the constant is not used as a fixed constant, a menu that allows the user to select and input a constant is output, and the body part information detection unit applies the constant inputted through the input unit to the key of the person To find the actual center point.

If the actual center point is found by using these constants and the calculation center value, the center point is moved from the calculation center value to the actual center point, and then a new distance map is formed by flooding at the moved position (S410) (For example, 5 cm or 10 cm) and extracting the center of the distance area, a line corresponding to the human figure indicated by the white dot in Fig. 14 is formed (S411)

In other words, the center of the line connecting the two boundaries is searched for by finding a circle having the radius (the length of the upper and lower parts of the body or the length of the body) corresponding to the human body with reference to the actual center point, The center of the human body region is searched for by labeling such that the radius is larger by a predetermined distance than the radius corresponding to the body previously found and is larger than the radius corresponding to the body by 5 cm or 10 cm, for example.

When the expected radius of the body is known in advance, labeling for forming the line may be performed after the body is searched from the body as described above. However, if the radius of the body is not known in advance, Labeling is performed every 10 cm to detect the line including the body as above.

As shown in Fig. 14, the points searched through these labeling forms five lines corresponding to the human shape, that is, one line connecting the head and the trunk with respect to the center point, two lines of both arms, and two lines of the legs .

By using the line thus detected, various points of the human body to be searched are found such as hands, elbows, knees, feet, and head.

For example, in the case of a human's arm, the starting point of the arm, that is, the end point of the arm (for example, Fingertips, wrists, etc.), the center of the elbow can be found by multiplying the partial line of the arm by a constant of 0.5.

Another example is to use a real center point and a line to locate a person as a Da Vinci model standing with arms and legs open, and then set a constant value of 0.6 for a 6/10 position from the center point, To find an elbow, and in this way, a point for each human body part is found (S412)

On the other hand, when the starting point of the detected line is based on the actual center point of the detected line, that is, when the starting point of the corresponding line starts on the actual center point, it is recognized as an arm. If the starting point of the line starts at the bottom of the actual center point, it is recognized as a leg or trunk. If the starting point is the right or left of the actual center point or within the same or an error range, do.

On the other hand, when the detected line is bent in the middle of detecting the body point as described above, the break is detected and the elbow or knee is determined from the actual center point according to the position of the corresponding line. When the starting point starts on the center point, it is recognized as an arm. When the starting point is located on the center point, it is recognized as an arm, and the right arm and the left arm are recognized according to the right and left.

Each point of the body found in this way, such as shoulder, elbow, hand, head, etc., is indicated by the blue dot in FIG.

When each part of the body is detected as described above, the behavior is recognized by using it and control is performed accordingly. Such a control apparatus will be described with reference to FIG.

The control device includes a behavior analyzing unit for recognizing a motion locus of the body point detected by the image acquiring unit 10, the body part information detecting unit 20, and the body part information detecting unit 20 A control unit 40 for receiving the behavior recognition data of the person detected by the behavior recognition unit 30 and recognizing the behavior and transmitting a corresponding control signal to the behavior recognition unit 30; And a driving unit 50 for driving various electronic devices such as robot operation control, game control, and new video output. The control unit 40 uses a separate computer from the body part information detection unit 20, A separate computer or an operation unit of a camera constituting the body part information detection unit may be used. The driving unit 50 also performs computer graphics processing, game control, or robot control.

The behavior recognition unit 30 accurately grasps the behavior trajectory of each part of the body. The behavior trajectory of the human body, i.e., the behavior recognition data thus grasped is hardware operation of a device such as a camera, a game machine, A game machine, a robot, and the like, which is transmitted to a control unit 40 of the electronic device for controlling the electronic device, and is transmitted to a driving unit 50 for driving hardware and software of the electronic device, I will look at it.

First, an example of the operation of the camera and the editing of the image obtained therefrom will be described below.

Dimensional camera is fixedly installed, and a user sets a 3D camera by selecting a desired background or contents while viewing a display unit provided in the 3D camera. In this process, the motion control of the three-dimensional camera is set so that the control through the behavior recognition (hereinafter, the behavior recognition mode) can be used.

Therefore, the 3D camera shoots the image through the lens, and the input image is sent to the behavior analyzer to search for the person, find the center predicted point, apply the constant corresponding to the actual body ratio, and change the center point to the actual center point Then, the user finds a point for each body part through labeling and the like, and recognizes the user's behavior by using the point thus found.

The behavior of the user identified in the behavior recognition process is transmitted to the control means of the three-dimensional camera, which is a control unit, and compared with the gesture for three-dimensional camera operation control previously stored in the memory. If the gesture is the same, the driving unit outputs an operation control signal to perform a predetermined operation for each gesture.

For example, when the user rotates the arm in a circular shape by extending the arm forward with a gesture for photographing a moving image on the camera, the controller of the camera controls the gesture of the user, which is the subject sent from the behavior analysis unit, , The moving picture is stopped when the gesture corresponding to the motion picture stopping is transmitted from the behavior analyzing unit.

In this case, the user can select a background or a content by using the gesture. In the behavior analysis process, the user can separate the background and the background, Therefore, the control means receives it and combines the user, who is the subject, with another selected background or content, and outputs it as a new image. This editing process can also be performed as a gesture.

In addition to the driving control using the gesture, the three-dimensional camera may be combined with a conventional method using input means (touch panel, cap pad, etc.) provided in the three-dimensional camera.

In this case, when the camera, which is the image acquiring unit, captures the user and recognizes the behavior in the body part information detection unit in the above-described manner, the control unit of the game apparatus, which is the control unit, The behavior recognized motion of the user performing the game is transmitted to the image editing unit which is the driving unit and is combined with the outputted game character so that the game character performs the action of the user in real time as it is and outputs it to the display.

In addition, the operation of the above-mentioned game machine, that is, the operation control such as game start and end, can also be performed through behavior recognition.

As another example, after photographing with a three-dimensional camera, which is an image acquiring unit, taking a photographing background as a blue screen, a person is searched for from the image acquired by the body part information detecting unit, and the action is transmitted to a control unit The control unit transmits the behavior recognition data transmitted to the image editing unit, which is a driving unit, and combines with the background image to output a new image.

In addition, the recognized behavior recognition data of the person is transmitted to the control unit of the robot, and the control unit of the robot controls the motion of the robot by the driving unit such as various motors constituting the robot so as to synchronize the movement of the robot with the motion recognition data. Signal, so that the robot moves in the same way as a human being. As another example, the behavior recognition data may be used as various motion control commands in addition to the motion of the robot.

10:
20: Body part information detector
22:
24:
26: Background separator
28: Distance map generator
29:
30:
40:
50:

Claims (6)

After the body part information detection unit finds a person in the acquired image, a distance conversion matrix is applied to the found person image to calculate a center value (hereinafter referred to as a calculation center value) corresponding to a human chest area, However,
The body part information detection unit searches the head position of a person and changes only the y value to the position obtained by multiplying the height of the person searched by an arbitrary constant by the x value of the found operation center value, However,
Wherein the arbitrary constant is a distance ratio of a y value from a head to a chest area based on an actual person's height.
The body part detecting method according to claim 1, wherein the body part detecting method further comprises a step of finding a point for each body part by forming and labeling a distance map again at an actual center point of the body part information detecting part.
The method according to claim 1, wherein the image obtained is an image obtained through a three-dimensional camera, and the person is searched by extracting a point while the body part information detection unit samples the image obtained from the three-dimensional camera based on the pure depth value , And a vector is created at the extracted point to compare the omega shape with a person to find a body part.
The body part detection method according to claim 3, wherein the body part information detection unit tracks a human area by comparing a current frame with a depth value on the basis of the detected area, and removes the background other than the person.
The body part detection method according to claim 1, wherein the arbitrary constant is a preset fixed constant or a constant inputted through input means in a process of finding a real center point of a person.
A body part information detector 20 for detecting a person in the image obtained from the image obtaining part 10 and analyzing the image to recognize a behavior,
The body part information detection unit 20 includes a human detection unit 22 for detecting a person from the image data acquired from the image acquisition unit 10 and searching for the head position of the person, An operation center value detector 23 for finding a center value (hereinafter, referred to as an operation center value) corresponding to a human chest area as a center point for finding each part of the human body by applying a distance transformation matrix to the human body detection part 22, An actual center point detection unit 24 for finding a new actual center point by changing only the y value while keeping the x value of the calculation center value at a position obtained by multiplying the elongation of the person searched by an arbitrary constant, A distance map is formed and labeled at the actual center point to extract the center of the distance region and the center of the body portion including at least one of palindrome, leg line, head line, A line detector 26 for detecting a line, and a body point detector 28 for detecting a body point to be searched in a line for each body part detected through the line detector 26,
Wherein the arbitrary constant is a distance ratio of a y value from a head to a chest area based on an actual person's height.
KR1020140051607A 2014-04-29 2014-04-29 Method and apparatus for recognizing body point KR101447958B1 (en)

Priority Applications (2)

Application Number Priority Date Filing Date Title
KR1020140051607A KR101447958B1 (en) 2014-04-29 2014-04-29 Method and apparatus for recognizing body point
PCT/KR2014/007530 WO2015167081A1 (en) 2014-04-29 2014-08-13 Method and device for detecting human body part

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
KR1020140051607A KR101447958B1 (en) 2014-04-29 2014-04-29 Method and apparatus for recognizing body point

Publications (1)

Publication Number Publication Date
KR101447958B1 true KR101447958B1 (en) 2014-10-14

Family

ID=51996783

Family Applications (1)

Application Number Title Priority Date Filing Date
KR1020140051607A KR101447958B1 (en) 2014-04-29 2014-04-29 Method and apparatus for recognizing body point

Country Status (2)

Country Link
KR (1) KR101447958B1 (en)
WO (1) WO2015167081A1 (en)

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR101807813B1 (en) * 2016-03-31 2017-12-12 (주)디지엔터테인먼트 Motion Recognition Service Offering System and Method thereof
KR101839106B1 (en) 2016-01-21 2018-03-26 주식회사 세자 Body analysis system
KR20190071341A (en) * 2017-12-14 2019-06-24 주식회사 아이미마인 Device and method for making body model

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR20050063401A (en) * 2003-12-22 2005-06-28 한국전자통신연구원 Method for detecting root of body
KR20120070316A (en) * 2010-12-21 2012-06-29 한국전자통신연구원 An apparatus and a method for detecting upper body
KR101215948B1 (en) * 2012-04-02 2012-12-27 주식회사 뉴인테크 Image information masking method of monitoring system based on face recognition and body information

Family Cites Families (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP1558015B1 (en) * 2002-08-30 2009-10-07 Sony Corporation Image extraction device, image extraction method, image processing device, image processing method, and imaging device

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR20050063401A (en) * 2003-12-22 2005-06-28 한국전자통신연구원 Method for detecting root of body
KR20120070316A (en) * 2010-12-21 2012-06-29 한국전자통신연구원 An apparatus and a method for detecting upper body
KR101215948B1 (en) * 2012-04-02 2012-12-27 주식회사 뉴인테크 Image information masking method of monitoring system based on face recognition and body information

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
김진우 외3, "원도우 제어를 위한 시각적 비접촉 사용자 인터페이스", 한국정보과학회, 정보과학회논문지:소프트웨어 및 응용 36(6), 2009.6., 471-478. *
김진우 외3, "원도우 제어를 위한 시각적 비접촉 사용자 인터페이스", 한국정보과학회, 정보과학회논문지:소프트웨어 및 응용 36(6), 2009.6., 471-478.*

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR101839106B1 (en) 2016-01-21 2018-03-26 주식회사 세자 Body analysis system
KR101807813B1 (en) * 2016-03-31 2017-12-12 (주)디지엔터테인먼트 Motion Recognition Service Offering System and Method thereof
KR20190071341A (en) * 2017-12-14 2019-06-24 주식회사 아이미마인 Device and method for making body model
KR101995411B1 (en) 2017-12-14 2019-07-02 주식회사 아이미마인 Device and method for making body model

Also Published As

Publication number Publication date
WO2015167081A1 (en) 2015-11-05

Similar Documents

Publication Publication Date Title
US8963963B2 (en) Video-based image control system
US9734393B2 (en) Gesture-based control system
US8879787B2 (en) Information processing device and information processing method
JP6771996B2 (en) Systems and methods for real-time interactive operation of the user interface
Kulshreshth et al. Poster: Real-time markerless kinect based finger tracking and hand gesture recognition for HCI
KR20160147450A (en) The bare hand interaction apparatus and method for augmented rearity using rgb-d images
US9836130B2 (en) Operation input device, operation input method, and program
KR20120068253A (en) Method and apparatus for providing response of user interface
US20160232708A1 (en) Intuitive interaction apparatus and method
US20120119991A1 (en) 3d gesture control method and apparatus
KR101447958B1 (en) Method and apparatus for recognizing body point
WO2020145224A1 (en) Video processing device, video processing method and video processing program
KR102052449B1 (en) System for virtual mouse and method therefor
CN110910426A (en) Action process and action trend identification method, storage medium and electronic device
JP2002366958A (en) Method and device for recognizing image
JP2000149025A (en) Gesture recognizing device and method thereof
KR20160141023A (en) The method of dynamic and static gesture recognition using depth camera and interface of immersive media contents
KR101861096B1 (en) Method and apparatus for controlling information displayed on screen by recognizing hand gesture of user
Abdallah et al. An overview of gesture recognition
JP2017033556A (en) Image processing method and electronic apparatus
KR101525011B1 (en) tangible virtual reality display control device based on NUI, and method thereof
Siam et al. Human computer interaction using marker based hand gesture recognition
Jain et al. Human computer interaction–Hand gesture recognition
Grzejszczak et al. Tracking of dynamic gesture fingertips position in video sequence
Huerta et al. Hand Gesture Recognition With a Novel Particle Filter

Legal Events

Date Code Title Description
E701 Decision to grant or registration of patent right
GRNT Written decision to grant
FPAY Annual fee payment

Payment date: 20181001

Year of fee payment: 5

FPAY Annual fee payment

Payment date: 20190902

Year of fee payment: 6