JP2013016171A - Method, device, and program for identifying human behavior cycle and recognizing human behavior - Google Patents

Abstract

PROBLEM TO BE SOLVED: To identify a human behavior cycle in a video and identify human behavior.SOLUTION: A method and a device for identifying a human behavior cycle and recognizing human behavior are provided in the present invention. The method for identifying a human behavior cycle in a video includes: processing for dividing human body areas of respective frames in a search range into a plurality of human body segments; processing for identifying local behavior cycles of the human body segments in a current frame; and processing for identifying a human behavior cycle in the current frame on the basis of the local behavior cycles. Human behavior cycles are identified corresponding to different behavior or individuals, and thereby temporal and spatial characteristics are extracted more accurately so that human behavior can be recognized more accurately.

Description

  The present invention relates to computer recognition of human body motion.

  Human body motion recognition technology has been widely applied and is attracting more and more attention. Using this technology, it is possible to quickly detect movements of interest from the video. Human body motion recognition technology can be applied to fields such as video monitors (banks, hospitals, factories, etc.), human computer interactions (scenes such as virtual games), and sports rules.

  Various realization methods for human body motion recognition technology have been proposed, and realization methods based on temporal and spatial features have been actively studied and are effective methods. In this type of method, information on a current frame and a plurality of adjacent frames is shown by temporal and spatial features, and then recognition of human body motion is realized by using temporal and spatial features. In the process of extracting temporal / spatial features, a human body motion cycle having the same length is set for all moving images.

  However, since there are differences in motion and individuals, it is necessary to increase the recognition rate of human motion realized by temporal and spatial characteristics. For this reason, it is desirable to develop technology that can identify suitable human motion cycles for different motions and different individuals so that temporal and spatial features can be extracted and more accurate human motion recognition can be realized. Yes.

  The following presents a simplified summary of the invention in order to provide a basic understanding of some aspects of the invention. It should be understood that this summary is not an exhaustive overview of the invention. Accordingly, the summary is not intended to identify key points or critical parts of the invention, nor is it intended to limit the scope of the invention. The purpose of this summary is to provide some concepts in a simplified form and to provide an introduction to the more detailed description that follows.

  Embodiments are intended to provide a method, an apparatus, and a program for specifying a human motion cycle in a video and specifying a human motion in a video.

  In order to achieve the above object, one aspect of the present invention provides a method for identifying a human motion cycle in a video, and the method divides a human body region of each frame within a search range into a plurality of human body parts. Processing, processing for specifying a local motion cycle of each human body part in the current frame, and processing for specifying a human motion cycle in the current frame based on the local motion cycle.

  According to one aspect of the present invention, a method for recognizing human motion in a video is provided, the method identifying a human motion cycle in the video based on the method, and a time based on the human motion cycle. A process of extracting spatial features and recognizing human motion in the video.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

  The present invention can be better understood by the following description with reference to the drawings. In all the drawings, the same or similar reference numerals indicate the same or similar elements. The drawings are included herein with the following detailed description, formed a part of this specification, and further illustrate preferred embodiments of the invention and illustrate the principles and advantages of the invention. Used for.

3 is a flowchart illustrating a method for identifying a human body motion cycle in a video according to an embodiment of the present invention.

6 is a flowchart illustrating a process of dividing a human body region in a frame according to an embodiment of the present invention into a plurality of human body parts.

4 is a flowchart for specifying a local operation period according to an embodiment of the present invention.

3 is a flowchart illustrating a method for recognizing human motion in a video according to an embodiment of the present invention.

FIG. 2 is a block diagram illustrating an apparatus for identifying a human body motion cycle in a video according to an embodiment of the present invention.

It is a block diagram which shows the human body part division part by one Example of this invention.

It is a block diagram which shows the local operation period specific | specification part by one Example of this invention.

1 is a block diagram illustrating an apparatus for recognizing human body motion in a video according to an embodiment of the present invention. FIG.

FIG. 6 is a block diagram illustrating an exemplary structure of a general purpose personal computer capable of implementing methods and / or apparatus according to embodiments of the invention.

  Embodiments of the present invention will be described below with reference to the drawings. In the interest of clarity and simplicity, not all features of an actual embodiment are described in the specification. However, in the course of developing an actual example, there can be many decisions that are identified in the embodiment so that the specific goals of the developer can be achieved. For example, it is to be understood that when restriction conditions regarding the system and the business are set, these restriction conditions may be changed depending on the embodiment. Also, although development work is usually very complex and time consuming, it should be further understood by those skilled in the art according to the present disclosure that such development work is within the scope of the work normally performed. It is.

  Here, further, in order to prevent the present invention from being obscured by unnecessary details, in the drawings, the device structure and / or processing steps closely related to the embodiments of the present invention are illustrated. It should be noted that only other details are shown and other details not relevant to the present invention are omitted.

  FIG. 1 shows a flowchart of a method for identifying a human motion cycle in a video according to an embodiment of the present invention.

  The method for specifying the human motion cycle shown in FIG. 1 can specify the human motion cycle in the video based on the periodicity of the human motion.

  Specifically, the human body motion generally has periodicity. For this reason, when the motion cycle ends, each part of the human body returns to the posture at the start of motion. Accordingly, it is possible to specify the human body motion cycle in the video based on the motion cycle of each part of the human body in the video.

  As shown in FIG. 1, in step S102, a human body region in a video can be divided to form a plurality of human body parts.

  Specifically, the human body region can be divided for each frame within a predetermined search range. Thereby, a plurality of human body parts can be acquired from each frame.

  Next, in step S104, the local motion period of each human body part in the current frame can be specified.

  Specifically, the local motion period of each part can be specified based on the motion vector of each human body part in the current frame.

  Next, in step S106, the human motion period of the current frame in the video can be identified based on the local motion periods of the plurality of human body parts in the current frame.

  Specifically, the maximum local motion period among the local motion periods of a plurality of human body parts in the current frame can be specified as the human body motion period of the current frame in the video.

  Thus, the human body motion cycle identification method for video based on the human body part using the human body motion periodicity can accurately identify the human body motion cycle in the video based on the periodicity of the human body motion.

  In the method for identifying the human body motion period in the video, different search ranges can be provided according to different application scenes or different needs.

  In an embodiment of the present invention, the search range may be a frame in a time zone having a predetermined first time length after the current frame. In addition, the first time length is a length of time that can sufficiently include one human body motion, and the first time length is set rationally according to experience values or specific application scenes. Can do.

  The above search range is merely exemplary and does not limit the present invention. The search range may be set in other manners.

  In another embodiment of the present invention, the search range may be a frame in a time zone having a predetermined second time length before the current frame. The second time length is a length of time that can sufficiently include one human body motion, and the second time length is set rationally according to experience values or specific application scenes. be able to.

  The second time length may be the same as the first time length or may be different from the first time length.

  In the above method, the process of dividing the human body region in the frame into a plurality of human body parts can be realized by various appropriate technical means.

  FIG. 2 is a flowchart illustrating a process of dividing a human body region in a frame into a plurality of human body parts according to an embodiment of the present invention.

  As shown in FIG. 2, in step S202, the outline of the human body region in the frame can be extracted.

  Any known edge extraction method can be used to extract the contour of the human body region in the frame.

  For example, the canny algorithm can be adopted to extract the outline of the human body region in the frame.

  The canny algorithm is an edge extraction method commonly used in the image processing field. For more information on the canny algorithm, see the document “A Computational Approach To Edge Detection” published by canny, J. (see IEEE Trans. Pattern Analysis and Machine Intelligence, 8: 679- 7). Can do. The entire contents of this document are incorporated herein and the description is omitted for the sake of brevity and clarity.

  Next, in step S204, a connection point on the contour line can be specified based on the extracted contour line of the human body region.

  In addition, the said connection point may connect the two different human body parts on the outline. For example, there are connection points between arms and limbs.

  Specifically, a region where two different human body parts are connected on the contour line generally shows a concave valley shape. For this reason, it is conceivable that the connecting point on the contour line should be located at the lowest point of the concave valley. Therefore, the connection point can be specified by specifying the lowest point of the concave valley.

  For example, a concave valley appears on the contour line at the connection point between the arm and the limb. There is a high possibility that the lowest point of the concave valley is a connection point to be specified (that is, a connection point where an arm and a limb contact each other).

  In one embodiment of the present invention, the connecting point on the human body contour can be specified by calculating the curvature of each pixel point on the contour.

  Next, in step S206, the human body region can be divided into a plurality of human body parts based on the connection points.

  Specifically, after specifying the connection point, the human body can be divided by some human-specific information (for example, the head shape approximates a circle and the limb shape approximates a rectangle). it can. Thereby, a plurality of human body parts can be acquired.

  For details of a specific technique for dividing a human body into a plurality of human body parts based on the connection points on the contour line, see “Digital Image Processing” created by Rafael C. Gonzalez and Richard E. Woods (Electronic Industry Publishing Co., Ltd.). 2002), the entire contents of which are hereby incorporated by reference and omitted for the sake of brevity and clarity.

  As described above, according to the embodiment shown in FIG. 2, the human body region in the frame can be efficiently and accurately divided into a plurality of human body parts.

  The embodiment shown in FIG. 2 is merely exemplary. The present invention is not limited to this. For example, other known human body segmentation methods (for example, a method using a region) may be employed to segment a human body region in a frame into a plurality of human body parts.

  In the above method, after the human body is divided into a plurality of human body parts, the local motion period of each human body part can be further specified.

  FIG. 3 shows a flow chart for identifying a local motion period of a human body part according to one embodiment of the present invention.

  As shown in FIG. 3, in step S302, the motion vector of the human body part can be calculated.

  The motion vector may be a parameter indicating the motion speed and direction of the human body part.

  The motion vector of the human body part can be calculated based on the motion of all or some of the pixels in the human body part.

  In one embodiment of the present invention, the motion vector of the entire human body part can be obtained based on the motion of some pixels in the human body part.

  For example, when the human body part is the head, first, based on the movement of feature pixels (for example, pixels corresponding to the positions of eyes, nose, and mouth), the motion vectors of these feature pixels can be calculated respectively. .

  Specifically, the motion vector of each feature pixel can be acquired from the difference in position between two adjacent frames (for example, the previous frame or the next frame) of the feature pixel.

  Thereafter, the motion vector of the human body part can be obtained by calculating the sum of the motion vectors of the feature pixels of the human body part.

  In the above exemplary description, the motion vector of the human body part is calculated based on some feature pixels. However, the present invention is not limited to this, and other modifications can be used.

  For example, in another embodiment of the present invention, the motion vector of the human body part can be obtained by calculating the motion vector of all the pixels in the human body part and then calculating the sum of all the motion vectors.

  Returning to FIG. 3, next, in step S <b> 304, the motion end frame of the human body motion can be searched within the search range based on the motion vector of the human body portion.

  When the motion end frame of the human body motion can be searched, in step S306, the time length between the current frame and the motion end frame can be specified as the local motion cycle of the human body portion.

  If the motion end frame cannot be searched within the search range, the predetermined third time length can be specified as the local motion cycle of the human body part in step S308.

  The third time length is a length of time that can sufficiently include one complete human body motion, and the third time length can be reasonably determined based on experience values or different application situations. Can be set.

  Thereby, the local motion period for every human body part in the current frame can be specified.

  In the above embodiment, the process of searching for the motion end frame of the human body part within the search range can be realized by the motion vector of the human body part.

  In one embodiment of the present invention, the motion end frame of the human body part can be identified by searching for a frame that approximates the vector sum of the motion vectors of the human body part in each frame within the search range.

  Specifically, for example, when the human motion cycle in the video frame sequence is specified in real time, the motion vector of the human body part in the frame is accumulated in the forward direction in the predetermined search range starting from the current frame. be able to.

  When it is found that the accumulated value of the motion vector is close to zero (for example, when the accumulated value is smaller than a predetermined first threshold value), the frame when the accumulated value of the motion vector becomes smaller than the first threshold value Can be specified as the motion end frame of the human body part.

  The first threshold value is a value close to zero set in advance based on experience values or actual application scenes.

  On the other hand, if it cannot be found until the end that the accumulated value of the motion vectors is smaller than the first threshold within the predetermined search range, it can be determined that the operation end frame cannot be searched within the predetermined search range. .

  As a result, it is possible to realize a process of searching for the motion end frame of the human body part within the search range.

  Although the case where the human body motion cycle in the above-described video frame sequence is specified in real time (when searching in order forward) has been described as an example, the above description is merely illustrative. The present invention is not limited to this.

  For example, in another embodiment of the present invention, for example, when analyzing an existing video and specifying a human body motion cycle in a video frame sequence, the current frame is used as a starting point within a predetermined search range. The motion end frame of the human body part may be specified by sequentially accumulating the motion vectors of the human body part in the frame in the backward direction.

  Accordingly, it is possible to realize a process of searching for the motion end frame of the human body part within the search range.

  According to the method for identifying the human motion period in the video in the above embodiment, the present invention further provides a method for recognizing the human motion in the video.

  FIG. 4 shows a flowchart of a method for recognizing human motion in a video according to an embodiment of the present invention.

  As shown in FIG. 4, in step S402, the human body motion cycle in the video can be specified.

  For example, the human body motion cycle in the video may be specified by adopting the method of any of the above embodiments.

  Next, in step 404, the human body motion can be recognized based on the identified human body motion cycle.

  Specifically, temporal / spatial features can be extracted based on the specified human body motion cycle, and human body motions can be recognized based on the temporal / spatial features.

  In the human motion recognition method described above, a human motion cycle adapted to different motions or different individuals is specified. Thereby, temporal and spatial features can be extracted more accurately, and more accurate human body motion recognition can be realized.

  Corresponding to the above method, embodiments of the present invention similarly provide an apparatus corresponding to the method.

  FIG. 5 shows a block diagram of an apparatus for identifying a human motion cycle in a video according to an embodiment of the present invention.

  As shown in FIG. 5, an apparatus 500 for identifying a human body motion cycle in a video may include a human body part division unit 502, a local motion cycle specifying unit 504, and a human body motion cycle specifying unit 506.

  The human body part segmenting unit 502 can segment human body regions in a video to form a plurality of human body parts.

  Specifically, the human body part classifying unit 502 can classify the human body region for each frame within a predetermined search range. Thereby, a plurality of human body parts may be acquired from each frame.

  The local operation cycle specifying unit 504 can specify the local operation cycle of each human body part in the current frame.

  Specifically, the local operation cycle specifying unit 504 can specify the local operation cycle of each part based on the motion vector of each human body part in the current frame.

  The human body motion cycle specifying unit 506 can specify the human body motion cycle of the current frame in the video based on the local motion cycles of a plurality of human body parts in the current frame.

  Specifically, the human body motion cycle specifying unit 506 can specify the maximum local motion cycle among the local motion cycles of the plurality of human body parts in the current frame as the human body motion cycle of the current frame in the video.

  Accordingly, the human body motion cycle in the video can be accurately identified based on the periodicity of the human body motion by the device that identifies the human body motion cycle in the video based on the human body part using the above human body motion periodicity. .

  In the above-described method for identifying the human body motion period in the video, different search ranges can be set according to different application scenes or different requirements.

  In an embodiment of the present invention, the search range may be a frame in a time zone having a predetermined first time length after the current frame. The first time length may be a time length sufficiently including one human body motion, and the first time length is rationally determined according to an experience value or a specific application scene. Can be set.

  The above-described search range is merely illustrative, and the present invention is not limited to this, and the search range can be set by other methods.

  In another embodiment of the present invention, the search range may be a frame in a time zone having a predetermined second time length from the current frame. The second time length may be a time length sufficiently including one human body motion, and the second time length is reasonably determined according to an experience value or a specific application situation. Can be set.

  Further, the second time length may be the same as the first time length or may be different from the first time length.

  In the above apparatus, the human body part segmenting unit can segment the human body region in the frame into a plurality of human body parts by various appropriate technical means.

  FIG. 6 is a block diagram of a human body partial section according to an embodiment of the present invention.

  As shown in FIG. 6, the human body part classification unit 600 may include a contour extraction unit 602, a connection point extraction unit 604, and a division unit 606.

  The contour extraction unit 602 can extract the contour line of the human body region in the frame by any known edge extraction method.

  For example, the canny algorithm can be adopted to extract the outline of the human body region in the frame.

  The connection point extraction unit 604 can further specify a connection point on the contour line based on the extracted contour line of the human body region.

  Specifically, a region connecting two different human body parts on the contour line generally appears in the shape of a concave valley. Considering that the connecting point on the contour line should be located at the lowest point of the concave valley, the connecting point can be specified by specifying the lowest point of the concave valley.

  In one embodiment of the present invention, a connection point on a human body contour line can be specified by a method of calculating a curvature of each pixel point on the contour line.

  The dividing unit 606 can further divide the human body region into a plurality of human body parts based on the connection points.

  Specifically, after identifying the connection points, human division is performed based on some human-specific information (for example, the shape of the head approximates a circle and the shape of the extremities approximates a rectangle). Thus, a plurality of human body parts can be acquired.

  The above-described embodiment using the human body segmentation unit shown in FIG. 6 is merely illustrative, and the present invention is not limited to this. For example, the human body segmentation unit may include other known human body segmentation methods ( For example, a region-based method or the like may be employed to divide a human body region in a frame into a plurality of human body parts.

  In the above apparatus, after the human body part division unit divides the human body into a plurality of human body parts, the local operation period specifying unit can specify the local operation period of each human body part.

  FIG. 7 shows a block diagram of a local operation period specifying unit according to an embodiment of the present invention.

  As shown in FIG. 7, the local motion cycle specifying unit 700 can include a motion vector calculating unit 702, a motion end frame searching unit 704, and a time length specifying unit 706.

  The motion vector calculation unit 702 can calculate the motion vector of the human body part.

  The motion vector calculation unit 702 can calculate the motion vector of the human body part based on the motion of all or some of the pixels in the human body part.

  In an embodiment of the present invention, the motion vector calculation unit 702 can obtain a motion vector of the entire human body part based on the motion of some pixels in the human body part.

  For example, if the human body part is the head, the motion vector calculation unit first calculates the motion vectors of these feature pixels based on the feature pixels (for example, pixels corresponding to the positions of the eyes, nose, and mouth). can do.

  Specifically, the motion vector calculation unit 702 obtains a motion vector of each feature pixel based on a position difference between two adjacent frames (for example, the previous frame or the next frame) of the feature pixel. Can do.

  Thereafter, the motion vector of the human body part can be obtained by calculating the sum of the motion vectors of the feature pixels of the human body part.

  In the above exemplary description, the motion vector of the human body part is calculated based on some feature pixels. However, the present invention is not limited to this, and other modifications can be adopted.

  For example, in another embodiment of the present invention, the motion vector calculation unit 702 obtains the motion vector of the human body part by calculating the motion vector of all the pixels in the human body part and then calculating the sum of all the motion vectors. You can also

  The motion end frame search unit 704 can search for the motion end frame of the human body motion within the search range based on the motion vector of the human body part calculated by the motion vector calculation unit.

  When the motion end frame search unit 704 can search for the motion end frame of the human body motion, the time length specifying unit 706 uses the time length between the current frame and the motion end frame as the local motion period of the human body part. Can be identified.

  In addition, when the motion end frame search unit 704 cannot search for the motion end frame within the search range, the time length specifying unit 706 specifies the predetermined third time length as the local motion period of the human body part. can do.

  The third time length is a length of time that can sufficiently include one complete human body motion, and the third time length can be reasonably determined based on experience values or different application scenes. Can be set.

  Thereby, the local motion period specifying unit 700 can specify the local motion period of each human body part in the current frame.

  In the above embodiment, the process of searching for the motion end frame of the human body part within the search range in the motion end frame search unit can be realized based on the motion vector of the human body part.

  In one embodiment of the present invention, the motion end frame search unit searches for a motion end frame of the human body part by searching for a frame that approximates a vector sum of motion vectors of the human body part within the search range. Can be identified.

  Specifically, for example, when the human body motion cycle in the video frame sequence is specified in real time, the motion end frame search unit starts from the current frame and moves the human body part in the frame forward in a predetermined search range. The motion vectors are accumulated in order.

  When the motion end frame search unit finds that the cumulative value of the motion vector is close to zero (for example, when the cumulative value is smaller than a predetermined first threshold value), the cumulative value of the motion vector is the first threshold value. The frame when it becomes smaller can be specified as the motion end frame of the human body part.

  The first threshold value may be an experience value or a value close to zero set in advance by an actual application scene.

  On the other hand, if the motion end frame search unit does not find that the cumulative motion vector value is smaller than the first threshold within the predetermined search range until the end, the motion end frame search unit within the predetermined search range. It can be determined that the search was not successful.

  Thereby, the motion end frame search unit can realize a process of searching for a motion end frame of the human body part within the search range.

  In the above description, the operation of the motion end frame search unit has been described by taking as an example the case where the human motion cycle in the video frame sequence is specified in real time (when searching forward in order), but the above description is merely illustrative It is a thing. The present invention is not limited to this.

  For example, in another embodiment of the present invention, for example, when analyzing an existing video and specifying a human body motion cycle in a video frame sequence, the motion end frame search unit starts from the current frame, Within the predetermined search range, the motion vector of the human body part in the frame may be accumulated in order backward to search for the motion end frame of the human body part.

  Accordingly, the motion end frame search unit can also realize a search for the motion end frame of the human body part within the search range.

  According to the apparatus for identifying the human body motion period in the video in the above embodiment, the present invention can further provide an apparatus for recognizing the human body motion in the video.

  FIG. 8 is a block diagram of an apparatus for recognizing human motion in a video according to an embodiment of the present invention.

  As shown in the figure, an apparatus 800 for recognizing human motion in a video according to an embodiment of the present invention may include a human motion cycle specifying unit 802 and a recognition unit 804.

  The human body motion cycle specifying unit 802 can specify the human body motion cycle in the video.

  For example, the human body motion cycle specifying unit 802 includes a device for specifying the human body motion cycle in any of the above-described embodiments, and can specify the human body motion cycle in the video.

  The recognition unit 804 can recognize a human body motion based on the identified human body motion cycle.

  Specifically, the recognizing unit 804 can extract temporal / spatial features based on the specified human body motion cycle, and can recognize human body motions based on the temporal / spatial features.

  The apparatus for recognizing the human body motion can appropriately specify the human body motion cycle for different motions or different individuals. Thereby, temporal and spatial features can be extracted more accurately, and more accurate human body motion recognition can be realized.

  The technical details of the device for identifying the human motion cycle in the video, the device for recognizing the human motion in the video, and the elements included in the device can be understood by referring to the description of the method. In order to clarify the specification, unnecessary explanation is omitted here.

  In addition, the various examples and examples described in the present application are all illustrative. It should be understood that the present invention is not limited thereto. In the present specification, descriptions such as “first”, “second” and the like are merely used for distinction in the description in order to clearly explain the present invention. Therefore, it should not be understood that any matter has a limited meaning.

  Each component module and unit in the apparatus can be realized by software, firmware, hardware, or a combination thereof. Specific means or schemes that can be used to implement are well known to those skilled in the art and will not be described here. When implemented by software or firmware, a program constituting the software may be installed from a storage medium or a network to a computer having a dedicated hardware configuration (for example, the general-purpose computer 900 shown in FIG. 9). The computer can execute each function when various programs are installed.

  In FIG. 9, a central processing unit (CPU) 901 executes various processes based on a program stored in a read-only memory (ROM) 902 or a program loaded from a storage unit 908 to a random access memory (RAM) 903. To do. The RAM 903 also stores data necessary for the CPU 901 to execute various processes as necessary. The CPU 901, ROM 902, and RAM 903 are connected to each other via a bus 904. An input / output interface 905 is also connected to the bus 904.

  Input unit 906 (including keyboard, mouse, etc.), output unit 907 (display, including cathode ray tube (CRT), liquid crystal display (LCD), etc., speaker), storage unit 908 (including hard disk, etc.), and communication unit 909 (including a network interface card such as a LAN card, a modem, etc.) is connected to the input / output interface 905. The communication unit 909 executes communication processing via a network, for example, the Internet. A driver 910 is also connected to the input / output interface 905 as necessary. A removable medium 911 such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, or the like is attached to the driver 910 as necessary. Thereby, the read computer program is installed in the storage unit 908 as necessary.

  When the series of processing is realized by software, a program constituting the software is installed from a network, for example, the Internet, or a storage medium, for example, a removable medium 911.

  Such a storage medium is shown in FIG. A person skilled in the art should understand that a program is stored therein and that the program may be read from the device and transmitted. Examples of the removable medium 911 include a magnetic disk (including a floppy disk (registered trademark)), an optical disk (including a compact disk read-only memory (CD-ROM) and a digital versatile disk (DVD)), and a magneto-optical disk (mini-disk). Disk (MD) (including registered trademark) and semiconductor memory. Alternatively, the storage medium may be a ROM 902. Alternatively, it may be a hard disk included in the storage unit 908, in which a program is stored, and a hard disk provided to the user together with a device including these.

  The present invention further provides a program product in which machine-readable command codes are stored. When the command code is read and executed by a machine, the method according to an embodiment of the present invention can be executed.

  In addition, a storage medium on which a program product storing a command code readable by the machine is mounted is also included in the disclosure of the present invention. Examples of the storage medium include, but are not limited to, floppy disks, optical disks, magneto-optical disks, memory cards, and memory sticks.

  Finally, it should be further explained that the terms “include”, “have”, or any other synonym are used in a non-exclusive sense, and a process, method, article or device that includes a series of elements As well as other elements not explicitly listed. In addition, it includes the unique elements of such a process, method, article or device. Also, unless otherwise limited, an element defined by the term “including one ...” does not exclude the presence of another identical element in a process, method, article, or apparatus that includes the element. .

  As mentioned above, although the Example of this invention was described in detail, referring drawings, it should be understood that the above-mentioned embodiment is only for demonstrating this invention, and does not limit this invention. is there. For those skilled in the art, various modifications and changes can be made to the above embodiment without departing from the spirit and scope of the present invention. Accordingly, the scope of the invention is limited by the claims and their equivalents.

In addition to the above examples, the following additional notes are disclosed.
(Appendix 1)
Dividing the human body region of each frame within the search range into multiple human body parts,
Identify the local motion period of each human body part in the current frame,
Identifying a human motion period in a current frame based on the local motion period;
A method for identifying a human motion cycle in a video including processing.
(Appendix 2)
The human body motion cycle in the video according to claim 1, wherein the search range is a frame in a time zone of a later first time length or a previous second time length from a current frame. How to identify.
(Appendix 3)
The process of dividing the human body region of each frame within the search range into a plurality of human body parts is as follows:
Extract the outline of the human body area in the frame,
Identify the connecting points of human body regions based on contour lines,
Dividing the human body region into a plurality of human body parts based on the connection points;
The method of specifying a human body motion cycle in a video as set forth in appendix 2, characterized by including a process.
(Appendix 4)
The process of identifying the local motion period of each human body part in the current frame is
Calculating a motion vector of the human body part;
Search the motion end frame of the human body part within the search range,
When the motion end frame can be searched, the time length between the motion end frame and the current frame is specified as a local motion period of the human body part.
The method of specifying a human body motion cycle in a video as set forth in appendix 2, characterized by including a process.
(Appendix 5)
In the search range, the process of searching for the motion end frame of the human body part includes a frame in which the sum of motion vectors of the human body part can be made smaller than a first threshold in the search range. 5. The method for specifying a human body motion cycle in a video according to the technical plan 4, which includes a process of searching as an end frame.
(Appendix 6)
The video according to claim 4, wherein the process of calculating a motion vector of the human body part includes a process of calculating a motion vector of the human body part based on a motion of all or a part of pixels in the human body part. To identify the human body movement cycle.
(Appendix 7)
The process of identifying a human body motion period in a current frame based on the local motion period includes a process of identifying a maximum local motion period among the local motion periods of the plurality of human body parts as a human body motion period. A method for specifying a human body motion cycle in the video according to attachment 2.
(Appendix 8)
Identifying a human motion cycle in the video based on the method according to any one of appendices 1 to 7,
A method for recognizing human motion in a video, comprising: extracting temporal and spatial features based on the human motion cycle and recognizing human motion in the video.
(Appendix 9)
A human body part classification unit that divides the human body region of each frame within the search range into a plurality of human body parts;
A local motion cycle specifying unit for specifying a local motion cycle of each human body part in the current frame;
An apparatus for specifying a human body motion cycle in a video, comprising: a human body motion cycle specifying unit for specifying a human body motion cycle in a current frame based on the local motion cycle.
(Appendix 10)
The human body motion period in the video according to claim 9, wherein the search range is a frame in a time zone of a subsequent first time length or a previous second time length from a current frame. Device to identify.
(Appendix 11)
The human body part section is
An outline extraction unit for extracting the outline of the human body region in the frame;
A connection point extraction unit for specifying a connection point of the human body region based on the contour line;
The apparatus for identifying a human body motion cycle in video according to claim 10, further comprising a dividing unit that divides the human body region into a plurality of human body parts based on the connection points.
(Appendix 12)
The local operation cycle specifying unit includes:
A motion vector calculation unit for calculating a motion vector of the human body part;
An operation end frame search unit for searching for an operation end frame of the human body part within the search range;
A time length specifying unit that specifies a time length between the action end frame and the current frame as a local action period of the human body part when the action end frame can be searched; The apparatus which specifies the human body movement period in the video of Additional remark 10 which is.
(Appendix 13)
The motion end frame search unit further searches the motion range as a motion end frame for a frame in which the sum of motion vectors of the human body part can be made smaller than a first threshold within the search range. The apparatus which specifies the human body movement period in the video of Additional remark 12.
(Appendix 14)
The human body motion cycle in the video according to appendix 12, wherein the motion vector calculation unit further calculates a motion vector of the human body part based on a motion of all or a part of pixels in the human body part. Device to do.
(Appendix 15)
The human body motion cycle specifying unit further specifies a maximum local motion cycle among the local motion cycles of the plurality of human body parts, and specifies the maximum local motion cycle as a human body motion cycle. A device for identifying a human motion cycle in the video described in 1.
(Appendix 16)
A human body motion cycle specifying unit for specifying a human body motion cycle in a video comprising the device according to any one of appendices 9 to 15,
An apparatus for recognizing human motion in a video, comprising: a recognition unit that extracts temporal and spatial features based on the human motion cycle and recognizes human motion in the video.
(Appendix 17)
A program that causes a computer to execute the method according to any one of appendices 1 to 8.

Claims (11)

Dividing the human body region of each frame within the search range into multiple human body parts,
Identify the local motion period of each human body part in the current frame,
Identifying a human motion period in a current frame based on the local motion period;
A method for identifying a human motion cycle in a video including processing.   The human body motion in the video according to claim 1, wherein the search range is a frame in a time zone of a later first time length or a previous second time length from a current frame. How to identify the period. The process of dividing the human body region of each frame within the search range into a plurality of human body parts is as follows:
Extract the outline of the human body area in the frame,
Identify the connecting points of human body regions based on contour lines,
Dividing the human body region into a plurality of human body parts based on the connection points;
The method according to claim 2, further comprising: processing. The process of identifying the local motion period of each human body part in the current frame is
Calculating a motion vector of the human body part;
Search the motion end frame of the human body part within the search range,
When the motion end frame can be searched, the time length between the motion end frame and the current frame is specified as a local motion period of the human body part.
The method according to claim 2, further comprising: processing.   In the search range, the process of searching for the motion end frame of the human body part includes a frame in which the sum of motion vectors of the human body part can be made smaller than a first threshold in the search range. 5. The method according to claim 4, further comprising a process of searching as an end frame.   5. The process of calculating a motion vector of the human body part includes a process of calculating a motion vector of the human body part based on a motion of all or a part of pixels in the human body part. A method for identifying human motion cycles in video.   The process of identifying a human body motion period in a current frame based on the local motion period includes a process of identifying a maximum local motion period among the local motion periods of the plurality of human body parts as a human body motion period. The method for specifying a human body motion cycle in a video according to claim 2. Identifying a human motion period in the video based on the method according to any one of claims 1 to 7,
A method for recognizing human motion in a video, comprising: extracting temporal and spatial features based on the human motion cycle and recognizing human motion in the video. A human body part classification unit that divides the human body region of each frame within the search range into a plurality of human body parts;
A local motion cycle specifying unit for specifying a local motion cycle of each human body part in the current frame;
An apparatus for specifying a human body motion cycle in a video, comprising: a human body motion cycle specifying unit for specifying a human body motion cycle in a current frame based on the local motion cycle. A human body motion cycle specifying unit that includes the apparatus according to claim 9 and that specifies a human body motion cycle in a video;
An apparatus for recognizing human motion in video, comprising: a recognition unit that extracts temporal and spatial features based on said human motion cycle and recognizes human motion in video. The program which makes a computer perform the method of any one of Claims 1 thru | or 8.

Images