KR101554677B1 - System for sensoring body behavior - Google Patents

Abstract

The present invention relates to a system for recognizing a physical behavior using an RGB-depth (RGBD) camera, including: an RGBD camera unit for detecting information on each joint of a human body and a skeleton image by extracting three-dimensional depth data of each imaging subject; a joint analysis unit for extracting a movement path of the joint and the skeleton image from the information on each joint of the human body and the skeleton image detected from the RGBD camera unit; and a main server including an activity determination unit for determining a behavior type of the correspondent imaging subject by comparing a stored behavior example DB in which a physical behavior type is presorted by movement path information on the joint and the skeleton image analyzed in the joint analysis unit, and a movement property of each joint of the human body and a skeleton movement. According to the present invention, provided is the system for recognizing a physical behavior which can instantly and accurately realize information on a current behavior type of each person at an imaging place.

Description

[0001] SYSTEM FOR SENSORING BODY BEHAVIOR [0002]

BACKGROUND OF THE INVENTION 1. Field of the Invention [0001] The present invention relates to a body behavior recognition system, and more particularly, to a body behavior recognition system that allows information on a current type of a person to be instantly and accurately grasped and provided at a photographing place.

The present invention relates to a body behavior awareness system.

In the past, only a system has been provided in which a person in charge can visually confirm an image photographed using a CCTV installed in a public place such as a road, an airport or a subway, and observe specific actions or observing behaviors.

However, with the above-mentioned existing system, it is impossible to observe all kinds of behaviors that may be related to specific terrorism or related to suicide or assault, and it is not possible to observe it immediately, there is a problem.

In addition, the existing system has a problem that it requires a lot of manpower, time, and cost even in the case of post-survey.

Although the technique as a background of the present invention is disclosed in Korean Patent Registration No. 0336440, a solution to the above-mentioned problems is not presented.

SUMMARY OF THE INVENTION It is an object of the present invention, which is devised to solve the above-mentioned problems, to provide an image processing apparatus, comprising: an RGBD (RGB-Depth) camera section for detecting joint and skeleton image information of each body; The present invention is intended to provide a body behavior awareness system that enables information on the kind of each person's current behavior to be instantly and accurately grasped and provided at a shooting place by including a server.

In addition, the independent object determination unit for determining the joint and skeleton image information for the independent body constituting the independent body and the movement path of the joint and the skeleton of the independent body determined by the independent object determination unit are analyzed, and the independent inter- The present invention also provides a system for recognizing a body behavior, which can instantaneously and precisely grasp the interactions between two or more persons at a scene where they are photographed.

According to an aspect of the present invention, an RGBD (Depth) camera section for extracting three-dimensional depth data of each object to be imaged and detecting joint and skeleton image information of each body, ; A joint analyzer for analyzing and extracting a movement path of a joint and a skeleton image from the joint and skeleton image information of each of the bodies detected by the RGBD camera unit; a joint analyzer for analyzing the movement path information of the joint and the skeleton image analyzed by the joint analyzer, A main server including an activity judging unit for judging a behavior type of a corresponding object to be photographed by comparing a behavior behavior DB of the type of physical behavior according to the movement characteristics of each joint and skeletal movement; Wherein the joint analyzing unit is configured to determine joints and skeleton image information of each of the bodies detected by the RGBD camera unit, joints and skeletons for an independent body constituting an independent body from an interlocking relationship with predetermined joints and skeletons, And an inter-object interaction determination unit for analyzing the movement paths of the joints and the skeletons of each of the independent bodies determined by the independent object determination unit and determining whether the inter-body interactions are independent, , And the motion path of the joint and the skeleton image is individually analyzed and extracted for each of the independent bodies determined by the independent object determination unit, and the behavior example DB stores the movement characteristics of joints and skeleton movements of two or more independent bodies The independent inter-bodily interactions are further classified and stored , The activity determining unit may determine that the interactions between the bodies are independent of each other in the inter-object interaction determining unit, the moving path information of the analyzed joints and the skeleton images of each independent body determined to be interlocked, The behavioral example DB further includes a mutual behavior type according to a change in the face vector coordinate of each of two or more independent bodies based on a reference point of a predetermined three dimensional coordinate system And the joint analyzer further analyzes and extracts facial information of the head from the joint and skeleton image information of each of the bodies by the independent object determiner, and the inter- If it is determined that the interlocking operation is performed, the reference point of the predetermined three- The activity determination unit may further extract information on change in vector coordinates of faces of each of the determined independent bodies, and when the determination unit determines that the inter-subject interactions are independent in the inter-object interaction determination unit, And comparing the change information of the vector coordinates of each face with the behavior example DB to further determine the kind of mutual action corresponding thereto.

At this time, the joint analyzing unit extracts three-dimensional coordinate values of a moving path of joint and skeleton image information of each of the bodies detected by the RGBD camera unit based on a reference point of a predetermined three-dimensional coordinate system, And extracts the movement route information based on three-dimensional coordinate values of joint and skeleton image information of each body.

delete

delete

As described above, according to the present invention, it is possible to provide a body behavior perception system in which information on the type of the current behavior of each person at the shooting location can be immediately and accurately grasped and provided.

In addition, according to the present invention, it is possible to provide a body behavior perception system capable of instantly and precisely grasping and providing even interoperable behaviors by two or more persons at a scene of photographing.

FIG. 1 is a block diagram illustrating an overall configuration of a body motion perception system according to a preferred embodiment of the present invention. Referring to FIG.
FIG. 2 is a block diagram of an entire block including an object determination unit that allows a user to grasp interworking behavior for two or more persons.
FIG. 3 (a) is an exemplary view of a pre-processing image captured by the RGBD camera unit, FIG. 3 (b) is an exemplary view of image information according to depth data for two objects, Figure 3 (c) is an illustration of each joint and skeleton image information for two persons.
FIG. 4A is an example of a standard for the joint and skeleton classification of the body, FIG. 4B is an example of two joints and skeleton image information extracted according to FIG. 4A, .
5 is an exemplary diagram of a block configuration constituting an overall system for the present invention.
6 is an exemplary classification table for body joint information.
FIGS. 7 to 11 are diagrams for illustrating examples of five types of mutual actions according to changes in face vector coordinates in a two-person relationship; FIG.

BRIEF DESCRIPTION OF THE DRAWINGS The advantages and features of the present invention, and the manner of achieving them, will be apparent from and elucidated with reference to the embodiments described hereinafter in conjunction with the accompanying drawings.

The present invention may, however, be embodied in many different forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the invention to those skilled in the art. To fully disclose the scope of the invention to those skilled in the art, and the invention is only defined by the scope of the claims. Like reference numerals refer to like elements throughout the specification.

DETAILED DESCRIPTION OF THE INVENTION Hereinafter, the present invention will be described with reference to the accompanying drawings.

FIG. 1 is a block diagram illustrating an overall configuration of a body motion perception system according to a preferred embodiment of the present invention. FIG. 2 is a block diagram of an overall block configuration including an object determination unit .

FIG. 3 (a) is an exemplary view of a pre-processing image captured by the RGBD camera unit, FIG. 3 (b) is an exemplary view of image information according to depth data for two objects, Figure 3 (c) is an illustration of each joint and skeleton image information for two persons.

FIG. 4A is an example of a standard for the joint and skeleton classification of the body, FIG. 4B is an example of two joints and skeleton image information extracted according to FIG. 4A, .

FIG. 5 is an exemplary view of a block configuration that constitutes the overall system of the present invention, and FIG. 6 is an exemplary classification table of body joint information.

FIGS. 7 to 11 are diagrams for illustrating an example of five kinds of mutual actions according to changes in face vector coordinates in a two-person relationship; FIG.

The body behavior recognition system according to the preferred embodiment of the present invention includes an RGBD (Depth-of-RGB) camera unit 100 for extracting three-dimensional depth data of each object to be imaged and detecting joint and skeleton image information of each body, A joint analysis unit 210 for analyzing and extracting a movement path of a joint and a skeleton image from the joint and skeleton image information of each of the bodies detected by the RGBD camera unit 100, The type of behavior is classified according to movement path information of joints and skeleton images and movement characteristics of joints and skeleton movements of the body, And an activity determination unit 220. The main server 200 includes an activity determination unit 220,

That is, the body motion recognition system according to the present invention includes the RGBD camera unit 100 and the main server 200.

The RGBD camera unit 100 is an RGBD (RED, GREEN, BLUE, DEPTH) camera that can measure the joint and skeleton information of a body from an image having depth data obtained by a 3D imaging camera having RGBD sensors to be.

The RGBD camera unit 100 includes a photographing unit 110, a background extraction unit 120, a subtraction unit 130, and a joint division extraction unit 140.

The photographing unit 110 is a means for photographing a background space and a person existing in a subway or a shopping mall, such as a general CCTV camera using RGBD sensors. The background extracting unit 120 extracts an image corresponding to the background It is a means to extract and remove.

The skeleton extractor 130 and the joint split extractor 140 are means for extracting joints and skeleton information of a body from an image having the background removed and composed of only a human body and having depth data.

That is, after the background image is removed from the photographed images, the joint and skeleton image information including only the body joint and the skeleton image is detected by the RGBD camera unit 100. That is, the RGBD camera unit 100 detects joint and skeleton information constituting the body by a predetermined image processor from joints and skeleton images of the body having the depth data.

Here, the above-described joint and skeleton image information detection is the same as the conventional joint and skeleton image detection process by the RGBD camera, so a detailed description thereof will be omitted.

The main server 200 includes a joint analysis unit 210, an activity determination unit 220, and a behavior example DB 230.

The main server 200 receives joint and skeleton image information from the RGBD camera unit 100 and analyzes movement of joints and skeletons based on images only, that is, movement paths of joints and skeleton images, And comparing the behavior of the body previously stored in the joint 230 and the movement path of the analyzed joint and skeleton images to determine the behavior characteristics of the person who is photographed.

More specifically, the joint analyzer 210 analyzes and extracts the movement path of the joint and the skeleton image from the joint and skeleton image information of each body detected by the RGBD camera, It is a means of analysis.

The behavior example DB 230 is a DB that stores the types of physical behavior classified according to the body joint and the movement path of the skeleton.

That is, in the behavior example DB 230, data on the movement path of the joint and the skeleton according to the body behavior are stored for each of the body actions.

The activity determining unit 220 determines the activity based on the movement path information of the joint and the skeleton image analyzed by the joint analysis unit and the data of the behavior example DB 230, The action types DBD230 and DBDD230 are compared with each other and a behavior type of the person photographed by the RGBD camera unit 100 from the movement path of the joint and skeleton images photographed and detected by the RGBD camera unit 100 .

At this time, the joint analyzer 210 determines the three-dimensional coordinate values of the movement path of the joint and skeleton image information of each body detected by the RGBD camera unit 100, based on the reference point of the predetermined three-dimensional coordinate system And extracts the movement route information based on the three-dimensional coordinate values of the joint and skeleton image information of each of the extracted bodies.

That is, each of the joint and skeleton images detected by the RGBD camera unit 100 is analyzed as coordinates on a predetermined three-dimensional coordinate system, and the movement path of each joint and the skeleton is analyzed.

The joint analysis unit 210 further includes an independent object determination unit 214 and an object interaction determination unit 215.

The independent object determination unit 214 determines an independent object based on joint distance and skeleton between the joint and skeleton image information of each body detected by the RGBD camera unit 100, Joint and skeleton image information.

That is, when the joint and skeleton image information photographed by the RGBD camera unit 100 includes joint and skeleton image information for two or more persons, the independent object determination unit 214 determines that the body and / Separate the joint and skeleton image information separately to analyze the movement path of the joint and skeleton image separately for each independent body.

The inter-object interaction determination unit 215 analyzes the movement paths of joints and skeletons of each of the independent bodies determined by the independent object determination unit 214, and determines whether or not the inter-body interactions are independent.

That is, the inter-object interaction determination unit 215 analyzes each movement path from two or more independent body-based joint and skeleton image information in the joint and skeleton image information detected by the RGBD camera unit 100, Whether they interact with each other, such as greeting each other, hugging each other, exchanging goods, or the like.

At this time, the behavioral example database 230 further stores the independent inter-bodily interactions according to the movement characteristics of joints and skeletal movements of two or more independent bodies.

At this time, the joint pivot unit 210 will individually analyze and extract the movement path of the joint and the skeleton image for each independent body determined by the independent object determination unit.

If it is determined that the inter-object interactions are independent in the inter-object interaction determination unit 215, the activity determination unit 220 determines that the inter-object interactions between the analyzed joints and the skeleton image movement path Information is compared with the action example DB 230 to further determine the corresponding interlocking behavior type.

At this time, it is preferable that the behavior example DB 230 further stores the mutual action types according to the change of the face vector coordinates of two or more independent bodies based on the reference point of the predetermined three-dimensional coordinate system.

That is, in the action example DB 230, change data of face vector coordinates of two or more persons for each mutual interlock, such as two or more persons hailing each other, hugging each other, hugging, It is stored by type of interlocking.

At this time, the joint analyzer 210 further analyzes and extracts facial information of the head from the joint and skeleton image information of each body by the independent object determiner 214, and the inter- 215, the change information of the vector coordinates of the face of each of the determined independent bodies is further extracted based on the reference point of the predetermined three-dimensional coordinate system.

In addition, when the inter-object interaction determining unit 215 determines that the inter-body interworking is independent, the activity determining unit 220 determines that the inter-object inter- The action example DB 230 is compared with each other and the corresponding mutual action type is further determined.

That is, the body motion perception system according to the present invention determines the type of behavior of each body from joints and skeleton movement paths of each body from the joint and skeleton image information of each body detected by the RGBD camera unit 100, If two or more bodies perform mutually interdependent actions, the type of mutual interdependence can be grasped so that they can be instantly judged by a person's behavior in an airport, subway or public place, .

It will be understood by those skilled in the art that the present invention may be embodied in other specific forms without departing from the spirit or essential characteristics thereof. It is therefore to be understood that the above-described embodiments are illustrative in all aspects and not restrictive. The scope of the present invention is defined by the appended claims rather than the foregoing detailed description, and all changes or modifications derived from the meaning and scope of the claims and the equivalents thereof are included in the scope of the present invention Should be interpreted.

100: RGBD camera unit 110:
120: background extraction unit 130: skeleton extraction unit
140: joint split extraction unit 200: main server
210: joint analysis unit 211: joint motion analysis unit
212: joint behavior detection unit 213: object reading unit
214: Independent object deciding unit 215: Inter-object deciding unit)
220: activity determining unit 230: action example DB

Claims (4)

An RGBD (Depth-of-RGB) camera unit for extracting three-dimensional depth data of each object to be imaged and detecting joint and skeleton image information of each body;
A joint analyzer for analyzing and extracting a movement path of a joint and a skeleton image from the joint and skeleton image information of each of the bodies detected by the RGBD camera unit; a joint analyzer for analyzing the movement path information of the joint and the skeleton image analyzed by the joint analyzer, A main server including an activity judging unit for judging a behavior type of a corresponding object to be photographed by comparing a behavior behavior DB of the type of physical behavior according to movement characteristics of each joint and skeletal movement, Lt; / RTI >
Wherein the joint analyzer comprises:
An independent object judging unit for judging joint and skeleton image information for an independent body constituting an independent body from a joint distance and an interlocking relation between predetermined joints and skeletons from the joint and skeleton image information of each body detected by the RGBD camera unit, Wow,
Further comprising an inter-object interaction determiner for analyzing the movement paths of joints and skeletons of each of the independent bodies determined by the independent object determiner and determining whether the inter-
Extracting and analyzing movement paths of the joint and the skeleton image separately for each independent body determined by the independent object determination unit,
In the action example DB,
The independent inter-bodily interactions are further classified and stored according to the movement characteristics of joints and skeletal movements of two or more independent bodies,
The activity judging unit judges,
When the inter-object interaction judging unit judges that inter-body interactions are independent, it compares the analyzed joints and the movement route information of the skeleton image of each independent body judged to be interlinked with the behavior example DB, And further judges the type thereof,
In the action example DB,
The mutual action types according to the changes of the face vector coordinates of two or more independent bodies are further classified and stored based on the reference point of the predetermined three dimensional coordinate system,
Wherein the joint analyzer comprises:
The independent object determination unit further analyzes and extracts facial information of the head from the joint and skeleton image information of each of the bodies,
Wherein when the inter-object interaction determining unit determines that inter-body interactions are independent, it further extracts change information of the vector coordinates of each of the determined independent bodies based on the reference point of the predetermined three-dimensional coordinate system,
The activity judging unit judges,
When the inter-object interaction determining unit determines that the inter-body interworking is independent, the information on the change in the vector coordinates of the respective faces of the independent bodies judged to be mutually interrelated is compared with the behavioral example DB, Wherein the body motion recognition system uses an RGBD camera.
The method according to claim 1,
Wherein the joint analyzer comprises:
Extracting three-dimensional coordinate values of the movement path of the joint and skeleton image information of each of the bodies detected by the RGBD camera unit on the basis of the reference point of the predetermined three-dimensional coordinate system,
And extracting the movement path information based on three-dimensional coordinate values of joint and skeleton image information of each of the extracted bodies.
delete delete

Images