KR101756916B1 - Device and method for action recognition - Google Patents

Device and method for action recognition Download PDF

Info

Publication number
KR101756916B1
KR101756916B1 KR1020160013087A KR20160013087A KR101756916B1 KR 101756916 B1 KR101756916 B1 KR 101756916B1 KR 1020160013087 A KR1020160013087 A KR 1020160013087A KR 20160013087 A KR20160013087 A KR 20160013087A KR 101756916 B1 KR101756916 B1 KR 101756916B1
Authority
KR
South Korea
Prior art keywords
image
target
information
visible light
recognizing
Prior art date
Application number
KR1020160013087A
Other languages
Korean (ko)
Inventor
박강령
간바야르
울럭벡
이지훈
전은솜
김종현
김영곤
최종석
Original Assignee
동국대학교 산학협력단
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 동국대학교 산학협력단 filed Critical 동국대학교 산학협력단
Priority to KR1020160013087A priority Critical patent/KR101756916B1/en
Application granted granted Critical
Publication of KR101756916B1 publication Critical patent/KR101756916B1/en

Links

Images

Classifications

    • G06K9/00335—
    • G06K9/00342—
    • G06K9/00348—
    • G06K9/20—
    • G06K9/2054—

Landscapes

  • Image Analysis (AREA)

Abstract

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an apparatus and method for recognizing an action, and more particularly to an apparatus and method for detecting an object using an object detection and tracking information. The present invention is capable of recognizing the behavior of the object stably in various lighting and weather environments and real-time behavior recognition because it does not use complicated features.

Description

[0001] DEVICE AND METHOD FOR ACTION RECOGNITION [0002]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an apparatus and method for recognizing an action, and more particularly to an apparatus and method for detecting an object using an object detection and tracking information.

Behavior recognition technology, which automatically understands human behavior in video, has been used for intelligent video surveillance and human - object interaction, and research is proceeding with much interest. Most existing methods of recognizing human behavior in video use motion information because motion information provides important information for classifying the behavior. However, the process of extracting motion information requires a lot of computation and it is difficult to recognize human actions in real time because of the large number of dimensions of extracted features.

 The background art of the present invention is disclosed in Korean Patent Publication No. 2008-0051013 (published on Jun. 10, 2008).

The present invention provides an apparatus and method for recognizing an action of an object using detection and tracking information of an object in a visible ray and a thermal image.

The objects of the present invention are not limited to the above-mentioned objects, and other objects not mentioned can be clearly understood from the following description.

According to an aspect of the present invention, an action recognition apparatus is provided.

A behavior recognition apparatus according to an embodiment of the present invention includes a visible light target region detection unit for generating a visible light background image and detecting a visible light target candidate region using a difference image of a visible light background image and an input visible light ray image, A thermal image target region detection unit for generating a thermal image background image and detecting a thermal image target candidate region by using a difference image between the generated thermal image background image and the input thermal image, A target region combining unit for combining regions to generate combined target region information, a target tracking unit for generating target tracking information by tracking the target based on the combining target region information, and a target recognition unit for recognizing the target target using the target tracking information And an action recognition unit.

According to another aspect of the present invention, there is provided a method for recognizing an action and a computer program for executing the method.

A behavior recognition method and a computer program for implementing the method of the present invention generate a visible light background image and detect a visible light target candidate area using a difference image of a visible light background image and an input visible light ray image A step of generating a thermal image background image, detecting a thermal image candidate region by using a difference image between the generated thermal image background image and the input thermal image, detecting the detected visible light region and the thermal image region Generating combining target area information, generating target tracking information by tracking the target based on the combining target area information, and recognizing an action of the target using the target tracking information.

The present invention is capable of recognizing the behavior of the object stably in various lighting and weather environments and real-time behavior recognition because it does not use complicated features.

1 is a view for explaining an action recognition system according to an embodiment of the present invention;
2 is a view for explaining an action recognition apparatus according to an embodiment of the present invention.
3 is a view for explaining an object behavior recognizing unit according to an embodiment of the present invention;
4 is a view for explaining an action recognition method according to an embodiment of the present invention;
5 is a view for explaining a single action recognition method among the action recognition methods according to an embodiment of the present invention.
6 to 8 are diagrams for explaining a hand waving, a kicking, and a fist striking method in an action recognition method according to an embodiment of the present invention.
9 is a diagram for explaining a mutual action recognition method between a plurality of objects according to an embodiment of the present invention;

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings so that those skilled in the art can easily carry out the present invention. The present invention may, however, be embodied in many different forms and should not be construed as limited to the embodiments set forth herein. Also, when a part is referred to as "including " an element, it is to be understood that it may include other elements as well, without departing from the other elements unless specifically stated otherwise.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Reference will now be made in detail to embodiments of the present invention, examples of which are illustrated in the accompanying drawings.

1 is a view for explaining an action recognition system according to an embodiment of the present invention.

Referring to FIG. 1, an action recognition system includes a visible light image input unit 100, a thermal image input unit 200, and an action recognition device 300.

The visible light image input unit 100 inputs an image of a visible light region. The visible light image input unit 100 may be, for example, a visible light camera.

The thermal image input unit 200 inputs a thermal image. Here, the thermal image can represent the thermal energy sensed by the thermal imaging camera as a brightness value. The thermal image can be expressed, for example, as a brightness value from 0 to 255 in 8 bits. The thermal image input unit 200 may combine the visible light image input unit 100 and the optical axis in parallel so that the combined errors of the input visible light image and the thermal image can be reduced.

The behavior recognition apparatus 300 detects and tracks an object in a visible ray image and a thermal image to recognize an object's behavior. The behavior recognizing apparatus 300 will be described later in more detail with reference to FIG.

2 is a view for explaining an action recognition apparatus according to an embodiment of the present invention.

2, the behavior recognition apparatus 300 includes a visible light region detection unit 310, a thermal image region detection unit 320, an object region combination unit 330, a target tracking unit 340, (350).

The visible light target region detecting unit 310 generates a visible light background image and detects a visible light target candidate region using a difference image between the visible light background image and the input visible light image. The visible light target region detecting unit 310 removes the shadow region from the visible light target candidate region. The visible light target region detection unit 310 removes noise from the visible region candidate region from which the shadow region is removed, and corrects the visible region candidate region to finally detect the visible region.

The thermal image target area detecting unit 320 generates a thermal image background image, and detects a thermal image target candidate area by using a difference image between the generated thermal image background image and the input thermal image. The thermal image target area detection unit 320 removes noise from the thermal image target candidate area, and corrects the thermal image target candidate area to finally detect the thermal image target area.

The target area combining unit 330 combines the visible light target area and the thermal image target area to generate the combined object area information. The target region combining unit 330 may combine the target region detected in each image by selectively using the input time of the input image or the quality of each image or by changing the weights. Since the amount of light is insufficient in a dark environment such as night, for example, it is difficult to obtain a visible light image for detecting an object, so that the object area combining unit 330 can confirm the quality of the input visible light ray or utilize information at night time So as to increase the weight of the thermal image. At this time, the combined object region can be expressed in visible light and thermographic image.

The target tracking unit 340 generates target tracking information by tracking the target based on the combining target area information. The object tracking unit 340 may track an object using, for example, at least one of a Kalman filter, a particle filter, and a CamShift.

 The target action recognition unit 350 recognizes the action of the target using the target tracking information. In order to recognize the action of the object, the target action recognition unit 350 should accumulate at least N (where N is a natural number) the tracked target image and the target region information. The target action recognition unit 350 can recognize an action on the object when N pieces of target image and object region information are accumulated. The target behavior recognizing unit 350 will be described in more detail with reference to FIG.

The target mutual behavior recognition unit 360 determines the presence or absence of mutual actions among a plurality of objects when there is a result of recognizing the target action for a plurality of objects. The target mutual action recognition unit 360 determines that the action of the target has a mutual action when there is a walk or run among the plurality of target action recognition results and if the distance change information between the two targets is larger than the predetermined distance change threshold In this case, we may decide to move away from each other and decide to approve a small case.

FIG. 3 is a diagram for explaining an object behavior recognizing unit according to an embodiment of the present invention.

3, the target behavior recognizing unit 350 includes an aspect information generator 351, a width size change comparator 353, a center point position change comparator 353, an aspect ratio comparator 355, And a hand / foot position comparison unit 357.

The horizontal and vertical information generation unit 351 generates horizontal and vertical length information of the target area. The aspect information generator 351 may generate rectangular box information using the length information and the length information.

The horizontal size change comparator 353 accumulates the difference between the horizontal size of M input regions (where M is a natural number) and the (M + 1) -th horizontal size based on the horizontal sizes of the input N object regions, . Here, M is an index indicating the horizontal size from 1 to N-1. The horizontal size variation comparing unit 353 accumulates the total of N-1 horizontal size variation information and compares the information with the predetermined horizontal size variation threshold T_W. When the cumulative width-length change information of the inputted N pieces of the target area information is larger than a predetermined threshold value TW, the transverse size variation comparing unit 353 compares the behavior of the target in the next step with hand shaking, kicking, punching, It can be judged to be one of the unknown. When the cumulative transverse length change information is not large, the transverse size variation comparing unit 353 may determine that the target behavior is one of standing, sitting, lying down, walking, and running in the next step.

The center point position change comparator 353 recognizes the run based on the center point velocity in the X axis direction among the inputted N pieces of target area information. The center point position change comparator 353 calculates the center point velocity in the X axis direction based on the displacements of the first center coordinate X axis coordinate position and the Nth center point X axis coordinate position on the basis of the N center point X axis coordinate positions If the velocity is greater than the predefined center-point X-axis coordinate position change threshold T_SX, it is judged to be a run. Otherwise, it is determined to be walking or unknown in the next step.

The center point position change comparator 353 accumulates the change information of the total N-1 center point Y-axis coordinate positions and compares the change information with the predetermined center point Y-axis coordinate position change threshold T_MY to determine that the change information of the center point Y-axis coordinate position is larger than T_MY In the next step, it is judged to be one of walking or running. Otherwise, it is judged to be one of standing, sitting, and lying down.

The aspect ratio comparing unit 355 recognizes the run based on the average value of the ratio of the length to the length of the inputted N pieces of the target area information. The average value of the ratio of length to width is calculated from the total of N width-to-length ratios and is judged to be run when the width is smaller than the predefined first aspect ratio threshold T_R1. do. In addition, the aspect ratio comparing unit 355 can determine that the average value of the ratio of the length to height ratio of the inputted N pieces of the target area information is smaller than the predefined second aspect ratio threshold value T_R2, In the next step, you can decide to sit or lie down. Based on the average value of the ratio of the length to the length of the input N area information, it recognizes the sitting and lying. In addition, the aspect ratio comparing unit 355 determines that the average value of the aspect ratio is smaller than the third aspect ratio threshold T_R3, which is defined in advance, and determines that the user is lying down.

The hand and foot position comparing unit 357 determines the hand or foot position in the input target image.

The hand and foot position comparing unit 357 determines that the foot X-axis coordinate position is larger than a predefined threshold T_K in some P images among the N inputted target images.

The hand and foot position comparing unit 357 calculates the X-axis coordinate position change information of the hand in some P images of the N inputted target images to recognize hand shaking. The hand and foot position comparing unit 357 can determine hand shake when the change information of the X axis coordinate positions of the total P-1 hands is accumulated and compared with the predetermined transverse size change threshold T_WX. The hand and foot position comparison unit 357 accumulates the change information of the X axis coordinate positions of the P-1 hands in total and compares the information with the predetermined threshold value T_P. When the change information of the X axis coordinate position of the hand is larger than T_P, .

4 is a diagram for explaining an action recognition method according to an embodiment of the present invention.

Referring to FIG. 4, in step S410, the behavior recognition apparatus 300 detects a visible ray object in a visible ray image obtained from a visible ray camera. The behavior recognition apparatus 300 detects a visible light target area using a difference image between input images or a difference image between a background image and an input image. In addition, the behavior recognition apparatus 300 can detect a visible light target area through background modeling such as GMM (Gaussian Mixture Models) or MOG (Model of Gaussian).

In step S420, the behavior recognition device 300 detects a thermal image object area in the thermal image obtained by the thermal imaging camera.

In step S430, the behavior recognition apparatus 300 combines the detected visible ray target area and the thermal image target area to generate the combined object area information. The behavior recognition apparatus 300 can check the input time of the input image or the quality of each image, and selectively use the detected region in each image or combine the different regions with different weights. Since the amount of light is insufficient in a dark environment such as night, for example, it is difficult to acquire a visible light image for detecting an object, so that the behavior recognition apparatus 300 can check the quality of the input visible light ray or utilize information at night time It can be combined in a form of increasing the weight of the thermal image. At this time, the combined object region can be expressed in visible light and thermographic image.

Steps S410 to S430 may be performed in units of a specific image interval or in a case where target tracking fails in the next step S440.

In step S440, the behavior recognition apparatus 300 generates the object tracking information by tracking the object based on the combining object area information. Here, the behavior recognition apparatus 300 may track an object using at least one of a Kalman filter, a particle filter, and a CamShift.

In step S450, the behavior recognition apparatus 300 recognizes the behavior of the object based on the tracked object information. In order to recognize the behavior of the object, the behavior recognition apparatus 300 must accumulate at least N (where N is a natural number) the tracked object image and object region information. The behavior recognition apparatus 300 can recognize an action recognition of a single object and a mutual action of a plurality of objects when N pieces of image and region information about the object are accumulated. When the number of objects is plural, the behavior recognition apparatus (300) simultaneously obtains a single behavior recognition result according to each object and a mutual action recognition result between the two objects.

5 is a diagram for explaining a single action recognition method among the action recognition methods according to an embodiment of the present invention.

Referring to FIG. 5, in step S510, the behavior recognition device 300 compares the horizontal size change information among the inputted N pieces of subject area information with a predetermined horizontal size change threshold T_W. Here, the horizontal size change information is calculated by accumulating the difference between the horizontal size of M (where M is a natural number) and the M + 1th horizontal size based on the input N horizontal sizes. Here, M is an index indicating the horizontal size from 1 to N-1.

In step S520, the behavior recognition apparatus 300 determines that the horizontal size change information among the inputted N pieces of the target area information is larger than the predetermined horizontal size change threshold T_W and the actions such as hand waving, kicking, and punching are classified . When an action such as hand waving, kicking, or punching is classified, the action recognizing device 300 recognizes actions such as hand waving, kicking, and punching based on the N pieces of target image information input after step S610 do.

In step S530, the behavior recognition apparatus 300 determines whether the center point X axis coordinate position change information of the N pieces of the target area information input without classifying the actions such as hand waving, kicking, and punching is classified based on the predefined center point When the X axis coordinate position change is larger than the threshold value T_SX, the target behavior is recognized as a jump. The behavior recognition apparatus 300 calculates the center-point velocity in the X-axis direction based on the displacements of the first center-point X-axis coordinate position and the N-th center-point X-axis coordinate position based on the N center-point X-

In step S540, the behavior recognition apparatus 300 determines that the center point velocity in the X-axis direction among the inputted N pieces of target area information is not larger than the predefined center-point X-axis coordinate position variation threshold T_SX, If the average value of the aspect ratios is smaller than the predefined first aspect ratio threshold value T_R1, the target behavior is recognized as walking. The behavior recognition apparatus 300 calculates an average value of the aspect ratio from the total of N width-to-height ratio. The behavior recognition apparatus 300 recognizes that the behavior of the object is unknown if the average value of the aspect ratio is not smaller than the predefined first aspect ratio threshold T_R1.

In step S550, the behavior recognition apparatus 300 determines that the horizontal size change information among the inputted N pieces of the target area information is not larger than the predetermined horizontal size change threshold T_W and the change information of the center point Y axis coordinate position is the center point Y axis coordinate position change threshold If it is greater than T_MY, it is judged to be one of walking or running in the next step. Otherwise, it is judged to be one of standing, sitting, and lying down in the next step. The behavior recognition apparatus 300 calculates the position change information in the Y-axis direction by accumulating the difference between the M-th center point Y-axis coordinate position and the (M + 1) -th center point Y-axis coordinate position.

In step S560, the behavior recognition apparatus 300 determines that the change information of the Y-axis coordinate position of the center point Y is larger than the center point Y-axis coordinate position change threshold T_MY and the velocity of the center point Y is greater than the center point Y velocity change threshold T_SY If it is large, it recognizes the action of the object as a run. The behavior recognition apparatus 300 determines that the target behavior is to be walked when the velocity of the center point Y among the N pieces of the target area information input is not greater than the center point Y velocity change threshold T_SY.

In step S570, the behavior recognition apparatus 300 determines that the change information of the Y-axis coordinate position of the center point among the N pieces of the target area information input is not larger than the center point Y-axis coordinate position variation threshold T_MY and the average value of the ratio of the width- Is smaller than the second aspect ratio threshold value T_R2.

In step S580, the behavior recognition apparatus 300 determines that the target behavior is to be seated if the average value of the ratio of the length to height ratio of the N pieces of the target area information is less than the third predetermined aspect ratio threshold T_R3, In this case, the act of the subject is judged to lie down.

6 to 8 are views for explaining a hand waving, a kicking, and a fist striking method among the methods of recognizing an action according to an embodiment of the present invention.

Referring to FIG. 6, in step S610, the behavior recognition apparatus 300 determines a hand or foot position based on a target image.

Referring to FIG. 7, the behavior recognition apparatus 300 defines a horizontal distance D, a first distance L1, and a second distance L2 to distinguish hands and feet from N tracked subject images. The behavior recognition apparatus 300 calculates the distance to the maximum X-axis coordinate position of the object (white) area along the Y-axis coordinate position on the basis of the minimum value of X in the image as the lateral distance D. In another embodiment of the present invention, the behavior recognition apparatus 300 may calculate the horizontal length (D) by using the distance from the maximum value of X in the image to the coordinate position of the minimum X axis of the object area along the Y axis coordinate position .

The behavior recognition apparatus 300 calculates the first distance L1 and the second distance L2 based on the graph of the horizontal distance D and the height Y. [ The behavior recognition apparatus 300 recognizes X, Y axis coordinate positions (M) corresponding to the maximum value of the horizontal distance (D) values on the graph for the horizontal distance (D) The distance between the straight line connecting the Y axis coordinate positions (F) and the line between two points is calculated to define the boundary point as the boundary point, and X (X) corresponding to the maximum value of the boundary point and the horizontal distance The distance to the Y axis coordinate position M is defined as the first distance L1 and the distance to the X and Y axis coordinate position F corresponding to the last value of Y is defined as the second distance L2 .

When the ratio of the first distance L1 to the second distance L2 is larger than the predetermined distance ratio threshold, the behavior recognition apparatus 300 determines the X, Y axis coordinate position (M) of the X-axis and Y-axis coordinate positions (M) corresponding to the maximum value of the lateral distance (D) values when the distance is smaller than the predetermined distance ratio threshold value. 8 is a diagram illustrating a first distance L1 and a second distance L2 along an object's behavior according to an embodiment of the present invention.

In step S620, when the foot position is found in step S610, the behavior recognition apparatus 300 determines that the X-axis coordinate position of the foot in the tracked target image is larger than the predefined foot X-axis coordinate position threshold T_K .

In step S630, the behavior recognition apparatus 300 determines that the x-axis coordinate position of the foot in the tracked target image is not larger than the predefined foot x-axis coordinate position threshold T_K, and the X-axis coordinate position and the Y- When the position change information is larger than the predetermined hand X axis coordinate position change threshold T_HX and the hand Y axis coordinate position change threshold T_HY, it is judged that the target action is hand waving. The X and Y position change information of the hand is calculated by accumulating the difference between the X and Y positions of the Mth hand and the X and Y axis coordinate positions of the M + 1th hand based on the X and Y positions of the P hand.

In step S640, the behavior recognition apparatus 300 determines that the X axis coordinate position and the Y axis coordinate position change information are not larger than the predetermined hand X axis coordinate position variation threshold T_HX and the hand Y axis coordinate position variation T_HY in the P images, If the cumulative change information of the X-axis coordinate position of the hand is larger than the predetermined second hand X-axis coordinate position change threshold T_P, it is determined that the target action is fist-poked. Otherwise, the run, walk, It can be judged to be an act of one of "none".

9 is a diagram for explaining a mutual action recognition method between a plurality of objects according to an embodiment of the present invention.

In step S910, when the behavior recognition apparatus 300 has a single object recognition result for a plurality of objects, it determines whether there is mutual action between the two objects. The behavior recognition apparatus (300) determines that there is a mutual action when the object is in a walking or running state among a plurality of object recognition results. The behavior recognition apparatus 300 determines that there is no mutual action when the single object recognition result for two objects is not walking or running.

In step S920, the behavior recognition apparatus 300 determines whether a single object recognition result for two objects is walking or running, and if the distance change information between two objects of the inputted N object area information is smaller than the predetermined distance variation threshold T_DS If they are large, they decide to move away from each other and decide to approve small cases. The behavior recognition apparatus 300 calculates the distance change information between two objects by accumulating the M-th distance and the M + 1-th distance difference between the two objects based on the N center positions of the two objects.

The behavior recognition method according to various embodiments of the present invention can be implemented in the form of a program command that can be executed through various means such as a server. Further, the program for executing the behavior recognition method according to the present invention may be installed in a computer means and recorded in a computer-readable medium. The computer readable medium may include program instructions, data files, data structures, and the like, alone or in combination. Program instructions to be recorded on a computer-readable medium may be those specially designed and constructed for the present invention or may be available to those skilled in the computer software arts. Examples of computer-readable media include magnetic media such as hard disks, floppy disks and magnetic tape; optical media such as CD-ROMs and DVDs; magnetic media such as floppy disks; Includes hardware devices specifically configured to store and execute program instructions such as magneto-optical media and ROM, RAM, flash memory, and the like.

The embodiments of the present invention have been described above. It will be understood by those skilled in the art that the foregoing description of the present invention has been presented for illustrative purposes and that those skilled in the art will readily understand that various changes in form and details may be made therein without departing from the spirit and scope of the invention. It will be possible. It is therefore to be understood that the above-described embodiments are illustrative in all aspects and not restrictive. The scope of the present invention is defined by the appended claims rather than the detailed description, and all changes or modifications derived from the meaning and scope of the claims and their equivalents should be construed as being included in the scope of the present invention.

100: visible light image input unit
200: thermal image input unit
300:

Claims (20)

A behavior recognition apparatus comprising:
A visible light target area detecting unit for generating a visible light background image and detecting a visible light target candidate area using a difference image between the visible light background image and the input visible light image;
A thermal image target area detecting unit for generating a thermal image background image and detecting a thermal image target candidate area by using a difference image between the generated thermal image background image and the input thermal image;
An object area combining unit for combining the detected visible light target area and the thermal image target area to generate the combined object area information;
A target tracking unit that generates target tracking information by tracking the target based on the combining target area information;
An object behavior recognition unit for recognizing an object behavior using object tracking information; And
And a target mutual action recognition unit for determining presence or absence of an mutual action among a plurality of objects when there is a target action recognition result for a plurality of objects,
Wherein the target mutual action recognizing unit judges that an action of the target has a mutual action when there is a walk or a run among a plurality of target action recognition results and if the distance change information between the plurality of targets is larger than a predetermined distance change threshold, The device is judged to be farther away, and the device is judged to be approached in a small case. delete delete A behavior recognition apparatus comprising:
A visible light target area detecting unit for generating a visible light background image and detecting a visible light target candidate area using a difference image between the visible light background image and the input visible light image;
A thermal image target area detecting unit for generating a thermal image background image and detecting a thermal image target candidate area by using a difference image between the generated thermal image background image and the input thermal image;
An object area combining unit for combining the detected visible light target area and the thermal image target area to generate the combined object area information;
A target tracking unit that generates target tracking information by tracking the target based on the combining target area information; And
And a target behavior recognition unit for recognizing an action of the target using the target tracking information,
The target behavior recognition unit
A horizontal and vertical information generator for generating horizontal length information and vertical length information of an input target area;
A horizontal size change comparator for recognizing any one of an action of hand waving, kicking, punching, punching, walking, and walking by using information of cumulative horizontal size change of N input regions (where N is a natural number) And
And a center point position change comparing unit for recognizing the run based on the center point velocity in the X axis direction among the N pieces of the object area information inputted or determining cumulative change information of the center point Y axis coordinate position as walking or running Action recognition device. delete 5. The method of claim 4,
The target behavior recognition unit
And an aspect ratio comparing unit for recognizing the inputted N pieces of the target area information as one of a run, a clerk and a sitting by using an average value of the ratio of the length to the length of the object.
5. The method of claim 4,
The target behavior recognition unit
The hand or foot position is determined in the input image, and the kicking direction is recognized using the X-axis coordinate position of the foot in some images of the inputted N target images or the X-axis coordinate position change information of the hand is calculated, And a hand / foot position comparing unit for recognizing the hand / foot position comparing unit.
In the behavior recognition method,
Generating a visible light background image and detecting a visible light target candidate region using a difference image of the visible light background image and the input visible light image;
Generating a thermal background image, detecting a thermal image candidate region using a difference image between the generated thermal background image and the input thermal image;
Combining the detected visible light target area and the thermal image target area to generate combined object area information;
Generating object tracking information by tracking an object based on the combining object area information; And
And recognizing an action of the object using the object tracking information,
The step of recognizing the behavior of the object using the object tracking information
Accumulating at least N (where N is a natural number) tracked object image and object area information;
Comparing the horizontal size change information among the target area information with a predetermined horizontal size change threshold; And
Recognizing the action of the object as a jump if the horizontal size change information is smaller than the horizontal size change threshold and the center point X axis coordinate position change information of the target area information is larger than a predefined center point X axis coordinate position change threshold Including an action recognition method. 9. The method of claim 8,
The step of generating the visible light background image and detecting the visible light target candidate area using the difference image of the visible light background image and the input visible light ray image
A method of recognizing a visible light target area by using a difference image between input images or using a difference image between a background image and an input image, or by performing background modeling of GMM (Gaussian Mixture Models) or MOG (Model of Gaussian).
9. The method of claim 8,
The step of combining the detected visible light target area and the thermal image target area to generate the combined object area information
A method for recognizing an action that combines a visible light target area and a thermal image target area by selectively using an object area detected in each image or by changing a weight, by checking the input time of the input image or the quality of each image.
delete 9. The method of claim 8,
If the center point X axis coordinate position change information is not larger than the predefined center point X axis coordinate position change threshold and the average value of the horizontal length to vertical length ratio in the object area information is smaller than the predefined first aspect ratio threshold, And recognizing that the user is walking. 9. The method of claim 8,
When the horizontal size change information among the inputted N pieces of the object area information is not larger than the predetermined horizontal size change threshold and the change information of the center point Y axis coordinate position is larger than the center point Y axis coordinate position change threshold, And recognizing the action as one of the actions.
14. The method of claim 13,
And recognizing that the target behavior is a walk when the velocity of the center point Y of the N pieces of the target area information is not greater than the center point Y velocity change threshold value.
15. The method of claim 14,
When the change information of the center point Y axis coordinate position among the inputted N pieces of target area information is not larger than the center point Y axis coordinate position change threshold and the average value of the ratio of width to height ratio is smaller than the predefined second aspect ratio threshold, A step of recognizing the action as a clerk.
In the behavior recognition method,
Generating a visible light background image and detecting a visible light target candidate region using a difference image of the visible light background image and the input visible light image;
Generating a thermal background image, detecting a thermal image candidate region using a difference image between the generated thermal background image and the input thermal image;
Combining the detected visible light target area and the thermal image target area to generate combined object area information;
Generating object tracking information by tracking an object based on the combining object area information; And
And recognizing an action of the object using the object tracking information,
The step of recognizing the behavior of the object using the object tracking information
Determining a position of the hand or foot in the target image information; And
And recognizing an action of the subject as a kick-out when the X-axis coordinate position of the foot in the tracked target image is larger than the predefined foot X-axis coordinate position threshold. 17. The method of claim 16,
Axis coordinate position of the foot is not larger than the predefined foot X-axis coordinate position threshold, and the X-axis coordinate and Y-axis coordinate position of the hand change in the images are changed. And recognizing an action of the target as a hand shake when the threshold value is larger than the threshold value.
18. The method of claim 17,
Axis coordinate position and a Y-axis coordinate position change information of the hand are not larger than a predetermined hand X-axis coordinate position variation threshold and a hand Y-axis coordinate position variation threshold, and cumulative change information of an X-axis coordinate position of the hand is not greater than a predetermined second hand And recognizing an action of the object as a punching if it is larger than the X axis coordinate position change threshold.
In the behavior recognition method,
Generating a visible light background image and detecting a visible light target candidate region using a difference image of the visible light background image and the input visible light image;
Generating a thermal background image, detecting a thermal image candidate region using a difference image between the generated thermal background image and the input thermal image;
Combining the detected visible light target area and the thermal image target area to generate combined object area information;
Generating object tracking information by tracking an object based on the combining object area information; And
And recognizing an action of the object using the object tracking information,
The step of recognizing the behavior of the object using the object tracking information
If there is a single target action recognition result for two objects and the distance change information between two objects of the input N area information is larger than the predetermined distance change threshold value, Recognizing an action; And
And recognizing an action of an object when the distance change information between the two objects among the input N pieces of object area information is smaller than a predetermined distance change threshold. A computer program recorded on a computer-readable recording medium for executing the action recognition method according to any one of claims 8 to 10 and 12 to 19.
KR1020160013087A 2016-02-02 2016-02-02 Device and method for action recognition KR101756916B1 (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
KR1020160013087A KR101756916B1 (en) 2016-02-02 2016-02-02 Device and method for action recognition

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
KR1020160013087A KR101756916B1 (en) 2016-02-02 2016-02-02 Device and method for action recognition

Publications (1)

Publication Number Publication Date
KR101756916B1 true KR101756916B1 (en) 2017-07-12

Family

ID=59352941

Family Applications (1)

Application Number Title Priority Date Filing Date
KR1020160013087A KR101756916B1 (en) 2016-02-02 2016-02-02 Device and method for action recognition

Country Status (1)

Country Link
KR (1) KR101756916B1 (en)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR102127946B1 (en) * 2019-02-21 2020-06-30 동국대학교 산학협력단 Device and method for person re-identification
CN112070016A (en) * 2020-09-08 2020-12-11 安徽兰臣信息科技有限公司 Detection method for identifying child behavior and action

Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR101583773B1 (en) * 2014-10-29 2016-01-20 동국대학교 산학협력단 Apparatus and method for object detection

Patent Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR101583773B1 (en) * 2014-10-29 2016-01-20 동국대학교 산학협력단 Apparatus and method for object detection

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR102127946B1 (en) * 2019-02-21 2020-06-30 동국대학교 산학협력단 Device and method for person re-identification
CN112070016A (en) * 2020-09-08 2020-12-11 安徽兰臣信息科技有限公司 Detection method for identifying child behavior and action
CN112070016B (en) * 2020-09-08 2023-12-26 浙江铂视科技有限公司 Detection method for identifying child behavior and action

Similar Documents

Publication Publication Date Title
JP7272417B2 (en) Object tracking system, object tracking device, and object tracking method
US11789545B2 (en) Information processing device and method, program and recording medium for identifying a gesture of a person from captured image data
JP6625220B2 (en) Method and system for detecting the action of an object in a scene
EP1330128B1 (en) Automatic detection and tracking of multiple individuals' faces using multiple cues
JP6419830B2 (en) System, method and apparatus for image retrieval
US7613325B2 (en) Human detection device and human detection method
US9600760B2 (en) System and method for utilizing motion fields to predict evolution in dynamic scenes
JP5077164B2 (en) Tracking device and tracking method
JP5754990B2 (en) Information processing apparatus, information processing method, and program
JP5634517B2 (en) Video image monitoring
JP2009075868A (en) Apparatus, method, and program for detecting object from image
JP2013501289A (en) Image analysis method
JP6649231B2 (en) Search device, search method and program
Manikandan et al. Human object detection and tracking using background subtraction for sports applications
KR101756916B1 (en) Device and method for action recognition
Okamoto et al. Summarization of egocentric moving videos for generating walking route guidance
US20210235012A1 (en) Electronic device and method for controlling electronic device
Zhang et al. Robust multi-view multi-camera face detection inside smart rooms using spatio-temporal dynamic programming
JP2019096062A (en) Object tracking device, object tracking method, and object tracking program
Farazi et al. Real-time visual tracking and identification for a team of homogeneous humanoid robots
KR101771655B1 (en) Device and method for action recognition
JP6040825B2 (en) Object tracking program, object tracking method, and object tracking apparatus
JP5241687B2 (en) Object detection apparatus and object detection program
Zhang et al. Long-term face tracking in the wild using deep learning
JP7240940B2 (en) Object image extraction device, method, and software program

Legal Events

Date Code Title Description
E701 Decision to grant or registration of patent right
GRNT Written decision to grant