CN102831380A - Body action identification method and system based on depth image induction - Google Patents

Body action identification method and system based on depth image induction Download PDF

Info

Publication number
CN102831380A
CN102831380A CN201110160313XA CN201110160313A CN102831380A CN 102831380 A CN102831380 A CN 102831380A CN 201110160313X A CN201110160313X A CN 201110160313XA CN 201110160313 A CN201110160313 A CN 201110160313A CN 102831380 A CN102831380 A CN 102831380A
Authority
CN
China
Prior art keywords
user
body
limb
depth image
environment
Prior art date
Application number
CN201110160313XA
Other languages
Chinese (zh)
Inventor
陈大炜
Original Assignee
康佳集团股份有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 康佳集团股份有限公司 filed Critical 康佳集团股份有限公司
Priority to CN201110160313XA priority Critical patent/CN102831380A/en
Publication of CN102831380A publication Critical patent/CN102831380A/en

Links

Abstract

The invention relates to a body action identification method and a body action identification system based on depth image induction. The body action identification method comprises the following steps: acquiring the depth image information of a user and an environment where the user stands; extracting the body outline of the user from the background of the depth image information; respectively changing the size of each part in the skeletal framework of the human body to be adapted to the body outline of the user, and acquiring the adapted body skeletal framework of the user; tracking and extracting the data which present the movement of the body of the user in a manner adapted to the body skeletal framework; and identifying the body action of the user according to the data which present the movement of the body of the user. According to the invention, the body action of the user is further identified and tracked by establishing the skeletal system of the user, so that the problem existing in the current action induction identification solution is better solved, the body action identification efficiency is improved, and the user experience of human-computer interaction is improved.

Description

一种基于深度图像感应的肢体动作识别方法及系统 One kind of body motion recognition method and system based on the depth of the image sensor

技术领域 FIELD

[0001] 本发明涉及人机交互技术,具体地说,本发明涉及一种基于深度图像感应的肢体动作识别方法及系统。 [0001] The present invention relates to human-computer interaction, and more particularly, the present invention relates to a body movement recognition method and system based on the depth of the image sensor.

背景技术 Background technique

[0002] 因为鼠标、键盘等传统人机交互设备在用户体验的自然性和友好性方面都存在一定的局限性,人机交互技术成为近年来非常热门的研究领域,进而出现了越来越多的诸如触控控制、声音控制、手势控制、动作感应等各种新型人机交互方式。 [0002] because the mouse, keyboard and other traditional human interface devices there are some limitations in the natural user experience and friendliness, human-computer interaction technology in recent years become a hot research field, and then there have been more and more various new human-computer interaction, such as a touch control, sound control, gesture control, motion sensors and the like. 特别是以任天堂的Wii和索尼的MOVE为代表的动作感应人机交互方式,通过各类传感器设备,实时的完成对肢体动作,更具体的说是上肢动作的识别过程,并转化为游戏主机等宿主设备能够识别的命令,是目前非常流行的一种人机交互方式。 Especially in the Nintendo Wii and Sony's motion-sensing MOVE as the representative of human-computer interaction, through various sensor devices, real-time completion of the physical action, and more specifically to the identification process of the upper limb movement, and converted to game consoles the host device can recognize the command, it is a very popular kind of human-computer interaction.

[0003] 以Wii的动作感应解决方案为例,其核心是位于特制的用户手持的游戏手柄中的MEMS (Micro Electromechanical System,即微电子机械系统)三轴加速度感应芯片,当用户手持手柄做动作时,三轴加速度传感器则能够将用户的手势动作通过该传感器转换为数字信号从而能够被系统识别。 [0003] In the motion-sensing Wii solutions, for example, the core is located in a special game controller held by the user in MEMS (Micro Electromechanical System, i.e. MEMS) three-axis acceleration sensor chip, when the user holds the handle operation do when the triaxial acceleration sensor can be switched by a user's gesture motion sensor into a digital signal which can be recognized by the system. 而MOVE的动作感应解决方案则是基于图像识别的原理,其核心是显示设备上的摄像头对特制的用户手持的游戏手柄上的发光彩球轨迹的识别和跟踪,而且彩球的发光颜色会根据实际环境的光照颜色条件进行自动调节,以保证能被系统高效识别。 The MOVE operation sensing solution is based on the principle of image recognition, which is a core identification and tracking light-emitting ball track on the gamepad camera device held by the user to the special, but will vary depending on the emission color balls light color condition to automatically adjust the actual environment, in order to ensure efficient system can be identified.

[0004] 无论是采用MEMS传感器的方案还是图像识别的方案,都还是需要手持辅助设备,这对于用户体验来说还是有一定的限制。 [0004] Whether using the MEMS sensor or an image recognition program of the embodiment, the auxiliary handheld devices are still required, the user experience is that there are certain restrictions. 比如,用户动作过大时容易将手持辅助设备扔出,因为MEMS传感器手柄本身价格也比较高,跌落受损的经济损失比较大;M0VE方案中如果用户在强光环境中,则用户的动作识别率会大幅下降,甚至无法识别用户动作,严重影响用户体验。 For example, when the user hand-held accessory Dongzuoguoda easily throw, since the MEMS sensor handle itself price is relatively high, falling damaged large economic loss; M0VE embodiment light if a user environment, the user's motion recognition rate will be substantially reduced, if not impossible to identify a user action, seriously affecting the user experience.

[0005]目前以手势识别为代表的,可以让用户不借助任何外部辅助设备进行人机交互的解决方案,基本上都是基于二维图像处理和模式识别技术,对使用环境光照条件等有较苛刻的要求。 [0005] currently gesture recognition as the representative, allowing users without human interaction solutions using any external auxiliary equipment, are essentially based on two-dimensional image processing and pattern recognition technology, the use of ambient lighting conditions, which are more demanding requirements. 从肢体动作识别过程来说,传统的动作识别需要进行动作建模、动作分割、动作分析等多个复杂步骤和过程,特别是对于动态肢体动作来说,不同的用户在进行肢体动作的时候会存在速率差异、轨迹差异等,从而导致动作建模轨迹在时间轴上引起非线性波动,而这种非线性波动的消除非常困难和复杂,所以传统的基于二维图像的肢体动作识别率和识别效率普遍不够高。 When the body movement identification process, the traditional model motion recognition operation is required, the operation is divided, a plurality of steps and a complicated analysis process actions, especially for dynamic body movements, the different body movements of the user will be performed rate difference exists, track differences, resulting trajectory modeling operation on the time axis due to nonlinear wave, and this is very difficult to eliminate fluctuations of nonlinear and complex, the conventional two-dimensional image based on the recognition rate and body movements efficiency generally not high enough.

[0006] 另一方面,真实的用户肢体动作都是在三维环境下做出的,而基于二维图像处理的结果是将用户的三维动作映射为二维动作来进行处理,很难获得真实的三维动作信息,也就在很大程度上限制了可以识别的肢体动作的丰富性,限制了手势识别设备的广泛应用。 [0006] On the other hand, the real user body movements are made under the three-dimensional environment, and based on the results of the two-dimensional image processing is to map the three-dimensional motion of a two-dimensional user action to be processed, it is difficult to get real three-dimensional motion information, it greatly limits the richness of recognizable body movements, limiting the widespread application of gesture recognition devices.

发明内容[0007] 本发明的主要目的在于克服现有技术的不足之处,公开一种基于深度图像感应的肢体动作识别方法及系统,识别和跟踪用户的肢体动作,提高肢体动作识别效率。 SUMMARY OF THE INVENTION [0007] The main object of the present invention to overcome the shortcomings of the prior art, discloses a method and system for body motion recognition based on the depth image sensing, identifying and tracking a user's body movements, body movement to improve the recognition efficiency.

[0008] 本发明公开了一种基于深度图像感应的肢体动作识别系统,包括: [0008] The present invention discloses a body movement based on the depth image sensing recognition system, comprising:

深度图像信息获取单元:用于获取用户及其所在环境的深度图像信息; The image depth information acquisition unit: means for obtaining user location and the depth image information of the environment;

肢体轮廓抽取单元:用于从上述深度图像信息的背景中抽取用户肢体轮廓; Contour extraction unit limb: for extracting a user profile limb from the background depth information in the image;

标准骨骼适配单元:用于分别改变标准人体骨骼框架中各个部分的大小,使其与上述用户肢体轮廓相适应,获得相应于该用户的适配肢体骨骼框架; Standard bone adaptation unit: a size standard for human bones were changed in each frame portion, so as to adapt said user profile limb, the corresponding adaptation to the user's skeletal frame limb;

骨骼运动跟踪单元:用于在深度图像信息中,以适配肢体骨骼框架的格式跟踪、抽取表达用户肢体的运动的数据; Skeletal motion tracking unit: for image information in the depth, to adapt the format of the frame limb skeleton tracking, expression data extracted motion of the user's limb;

肢体动作识别单元:用于根据上述表达用户肢体的运动的数据识别用户的肢体动作。 Action recognition unit body: a body movement based on the data for identifying a user of the above-described expression movement of the limb of the user.

[0009] 本发明公开的肢体动作识别系统中,所述深度图像信息获取单元进一步包括: 深度图像传感单元:用于向用户所在方向发射经过编码的红外结构原始光平面,并接 [0009] The body movement recognition system disclosed in the present invention, the depth image information obtaining unit further comprises: a depth image sensing unit: means for transmitting an encoded infrared light plane structure of the original user's direction, and then

收和感应经由用户及其所在环境反射回来的三维环境对象红外结构光;和 Sensing the reflected and received via the user and the environment where the infrared structured light three-dimensional environment objects; and

深度图像处理单元:用于通过对比上述三维环境对象红外结构光的编码和原始结构光平面的编码,来获取深度图像息。 Depth image processing means: for comparison the three-dimensional environment object coding structure of infrared light and the original structure of the light plane, is acquired by the depth image information.

[0010] 所述肢体轮廓抽取单元是根据连续的深度图像信息中的运动差分分析抽取用户肢体轮廓。 [0010] The limb contour extraction unit extracts user body contour is analyzed in accordance with the continuous movement difference image depth information.

[0011] 所述标准骨骼适配单元通过缩放、旋转、变形的计算方法对所述标准人体骨骼框架进行适配。 The [0011] Standard bone adaptation unit, rotation, deformation calculation of adapting the standard human skeleton frame by scaling.

[0012] 所述标准人体骨骼框架包括按照正常人体结构相互连接的头部、躯干、盆骨、左上臂、左下臂、左手、右上臂、右下臂、右手、左大腿、左小腿、左脚、右大腿、右小腿和右脚。 [0012] The standard human skeletal frame comprising a head according to the normal anatomy of interconnected, torso, pelvis, the left upper arm, the lower left arm, left, right upper, right lower arm, the right hand, left leg, left leg, left , right thigh, right leg and right foot.

[0013] 本发明还公开了一种基于深度图像感应的肢体动作识别方法,包括如下步骤: [0013] The present invention also discloses a method of identifying a body movement based on the depth image sensing, comprising the steps of:

A,获取用户及其所在环境的深度图像信息; A, acquires the user information and the environment where the depth image;

B,从上述深度图像信息的背景中抽取用户肢体轮廓; B, extracts the user profile limb from the background depth information in the image;

C,分别改变标准人体骨骼框架中各个部分的大小,使其与上述用户肢体轮廓相适应,获得相应于该用户的适配肢体骨骼框架; C, respectively, to change the size of a standard human bone portion of each frame, so as to adapt said user profile limb, the corresponding adaptation to the user's skeletal frame limb;

D,在深度图像信息中,以适配肢体骨骼框架的格式跟踪、抽取表达用户肢体的运动的数据; D, in the depth image information in a format adapted to limb skeletal frame tracking expression data extracted motion of the user's limb;

E,根据表达用户肢体的运动的数据识别用户的肢体动作。 E, body movements in accordance with the user identification data expressing motion of the user's limb.

[0014] 本发明公开的肢体动作识别方法,所述步骤C中进一步包括如下步骤: [0014] The body movement recognition method of the present disclosure, the step C further comprising the steps of:

Cl,根据标准人体骨骼框架和步骤B中获取的用户肢体轮廓,保证盆骨的对应位置相 Cl, standard human user profile limb skeletal frame obtained in step B and to ensure that the corresponding phase position of the pelvis

一致; Consistent;

C2,移动、缩放标准人体骨骼框架的躯干骨骼至合适高度,保证头部骨骼的对应位置相 C2, move, resize standard human bone trunk skeletal frame to a suitable height to ensure that the corresponding position of the head relative to the bone

一致; Consistent;

C3,移动、缩放标准人体骨骼框架的下肢骨骼至合适位置,保证脚部的对应位置相一 C3, movement, zoom level lower extremity bones of the human skeleton framework to a suitable location, in a corresponding position relative to the foot

致; Induced;

C4,移动、缩放标准人体骨骼框架的上肢骨骼至合适位置,保证双手对应位置相一致;C5,检查对比适配骨架关键点位置是否与所述用户肢体轮廓相一致,如果否,则回到上述步骤Cl,从盆骨开始重新适配;如果是,则进入下一步。 C4, move, resize standard limb bones of the human skeleton framework to a suitable position corresponding to the position of his hands to ensure consistent; whether C5, adapted to compare the skeleton key check point position coincides with the limb of the user profile, and if not, the process returns to the above-described step Cl, adaptation is restarted from the pelvis; if yes, the process proceeds to the next step. [0015] 所述步骤A中可以采用双目视觉技术、或者飞行时间技术、或者结构光编码技术获取用户及其所在环境的深度图像信息。 [0015] Step A binocular vision techniques can be employed, or the time of flight technique, or structured light coding techniques to obtain the user location and the depth image information environment.

[0016] 所述步骤A采用结构光编码技术获取用户及其所在环境的深度图像信息的方法,进一步包括如下步骤: [0016] Step A using the structured light image coding techniques acquire depth information of the user and the environment where, further comprising the steps of:

Al,向用户所在方向发射经过编码的红外结构原始光平面,并接收和感应经由用户及其所在环境反射回来的三维环境对象红外结构光; Al, transmitting the encoded original infrared light plane structure where the direction of the user, and receiving and sensing reflected via a user environment where a three-dimensional environment and its target back structure infrared light;

A2,通过对比上述三维环境对象红外结构光的编码和原始结构光平面的编码,来获取深度图像信息。 A2, compare the three-dimensional environment object coding structure of infrared light and the original structure of the light plane, to obtain the depth image information.

[0017] 步骤C中还可以包括:C6, 在用户动作时检查肢体轮廓是否与全身骨骼的实际运动保持一致。 [0017] Step C may further include: C6, check whether the contour of the limb consistent with the actual movement of the whole body bone when a user operation.

[0018] 本发明公开的一种基于深度图像感应的肢体动作识别方法及系统,基于深度图像传感器和深度图像处理单元,可以将用户肢体图像高效的从复杂背景中分离出来,能够重建用户肢体的骨骼系统,进一步的识别和跟踪用户的肢体动作,最终完成用户肢体动作识别过程,从而可以较好的解决现有的动作感应识别解决方案中存在的问题,提高肢体动作识别效率,改善人机交互用户体验。 [0018] The present invention is based on body motion recognition method and system for sensing the depth image, an image sensor based on the depth and the depth image processing unit, the user may be efficient image limb separated from complex background disclosed, the user can be reconstructed limb skeletal system, further identify and track the user's body movements, body movements to finalize the user identification process, which can better solve existing motion-sensing solutions to identify the problems and improve body movement recognition efficiency, improve human-computer interaction user experience.

附图说明 BRIEF DESCRIPTION

[0019] 图I为本发明的肢体动作识别系统的一个实施例的电路框图。 [0019] a body movement recognition system of FIG. I is a circuit block diagram of the present invention, an example of embodiment.

[0020] 图2为本发明的肢体动作识别方法的一个实施例的流程图。 Recognition of body movements [0020] FIG 2 is a flowchart of one embodiment of the invention embodiment.

[0021]图3是本发明的肢体动作识别系统采用的肢体骨骼结构示意图。 [0021] FIG. 3 is a schematic diagram limb skeletal structure body movement recognition system employed in the present invention.

[0022] 图4是本发明的标准骨骼适配方法的一个实施例的流程图。 [0022] FIG. 4 is a flowchart of one embodiment of a standard bone adaptation method of the present invention.

具体实施方式 Detailed ways

[0023] 下面结合附图和具体实施方式对本发明作进一步详细说明。 [0023] Hereinafter, the present invention is described in further detail in conjunction with accompanying drawings and specific embodiments.

[0024] 获取图像深度信息的方式有多种,常见的包括双目视觉技术,飞行时间技术,结构光编码技术等,不失一般性地,本发明以结构光编码技术作为获取图像深度信息的一种手段来描述本发明。 [0024] The image depth information acquiring a variety of ways, including common binocular vision, time of flight technique, the structure of an optical encoder technology, without loss of generality, the present invention is structured as an optical coding technique acquires image depth information a means to describe the present invention.

[0025] 如图I所示为本发明的肢体动作识别系统的一个实施例的电气结构框图,本发明的基于深度图像感应的肢体动作识别系统主要构成包括: [0025] FIG electrical block diagram of one embodiment of body movement recognition system I the invention is shown, the depth recognition system based on the body movement of the image sensor of the present invention is mainly composed comprising:

深度图像传感单元:负责向用户所在方向发射经过编码的结构光平面,并接收和感应经由用户所在环境反射回来的红外结构光。 The depth image sensing unit: a light emission structure is responsible for the user plane where the encoded direction, and receiving and sensing the reflected infrared light back structure via the user environment is located.

[0026] 深度图像处理单元:根据结构光编码技术原理,通过对比三维环境对象结构光编码和原始平面结构光编码,来获取结构光传感器可视范围内的场景深度信息。 [0026] The depth image processing unit: the light technical principle coding structure, three-dimensional environment by comparing the object structure and the structure of an optical encoder encodes the original plane of light to obtain depth information of a scene within a visible range of the optical sensor architecture.

[0027] 肢体轮廓抽取单元:负责从环境背景中抽取用户肢体轮廓,不失一般性,这里假设用户在实际使用环境中相对于其他场景部分是运动的,所以可以根据连续深度图像中的运动差分分析抽取出用户肢体轮廓。 [0027] limb contour extraction unit: extracts the user is responsible for the environmental background profile limb, without loss of generality, it is assumed here that the user in actual use relative to other parts of the scene is in motion, the differential motion can be continuous in the depth image according to the environment analysis extract user profile limbs.

[0028] 标准骨骼适配单元:负责将标准人体骨骼框架根据实际抽取的用户肢体轮廓进行自动适配,包括缩放、旋转、变形等过程,从而将标准骨骼框架转换为与当前用户肢体轮廓相适应的适配肢体骨骼框架系统。 [0028] Standard skeleton adapter unit: standard human skeleton framework responsible for automatically adapted according to the actual user profile limb extracted, the process including scaling, rotation, deformation or the like, so as to convert the standard skeletal frame and adapted to current user profile limb adaptation limb skeletal frame systems. [0029] 骨骼运动跟踪单元:负责跟踪适配肢体骨骼框架系统的运动; [0029] motion tracking unit skeleton: is responsible for tracking system adapted to limb skeletal frame motion;

肢体动作识别单元:负责完成肢体动作识别过程。 Action recognition unit body: body movement is responsible for completing the recognition process.

[0030] 图I中的深度图像传感单元和深度图像处理单元,就是采用结构光编码技术作为获取图像深度信息的手段。 [0030] FIG. I is the depth of the image sensing unit and the depth image processing unit, is the use of structured light image coding techniques as a means to acquire depth information. 如果采用双目视觉技术、或者飞行时间技术,采用相应的深度图像传感器单元,再进行相应的深度图像处理技术,也同样可以获得图像深度信息。 If binocular vision, or time of flight technique, using the corresponding depth image sensor unit, then the corresponding depth image processing technology, it is possible to obtain an image depth information.

[0031] 如图2所示为本发明的肢体动作识别方法的一个实施例的流程图,主要步骤包括: [0031] The flowchart of one embodiment of a method of identifying body movements shown in FIG. 2 of the present invention, the main steps comprising:

1、深度图像传感单元获取深度图像数据; 1, the depth of the image sensing unit acquiring a depth image data;

2、深度图像处理单元获取可视范围内的场景深度信息,传给肢体轮廓抽取单元,抽取动态肢体轮廓; 3、标准骨骼适配单元进行标准人体骨骼适配,获取与当前用户肢体轮廓相适应的适配肢体骨骼框架; 2, the image processing unit acquires the depth information of a scene depth within the visible range, passed limb contour extraction means extracts contour dynamic limb; 3, standard bone standard human bone adaptation unit adapted to acquire a current user profile limb adapted adaptation limb skeletal framework;

4、对适配肢体骨骼框架进行骨骼动作跟踪; 4, the adaptation of skeletal framework limb skeleton tracking operation;

5、肢体工作识别单元进行肢体动作识别。 5, the working body identification unit identifying body movements.

[0032] 如图3所示是本发明的肢体动作识别系统采用的标准肢体骨骼结构示意图,包括:头部、躯干、盆骨、左上臂、左下臂、左手、右上臂、右下臂、右手、左大腿、左小腿、左脚、右大腿、右小腿和右脚。 [0032] FIG. 3 is a schematic view of a standard limb skeletal structure body movement recognition system employed in the present invention, includes: head, torso, pelvis, the left upper arm, the lower left arm, left, right upper, right lower arm, the right hand , left thigh, left leg, left foot, right thigh, right leg and right foot.

[0033] 如图4所示是本发明的标准骨骼适配方法的一个实施例的流程图,本发明中的标准骨骼适配步骤包括: [0033] As shown in FIG. 4 is a flowchart of one embodiment of a standard bone adaptation method of the present invention, the present invention comprises the step of adapting standard bone:

I、系统提示用户做出标准适配姿态,即竖直站立,双手平伸,且保证肢体在深度图像传感单元的可视范围以内。 I, the system prompts the user to make a standard adapter posture, i.e. standing upright, Stretch hands, and to ensure that within the visible range of the limb in the depth of the image sensing unit. 也可以用户不需要做出特定适配姿态,而是适配系统自动去适配。 Users do not need to be made can be adapted to specific gesture, but adapting the system to automatically adapt.

[0034] 2、用户按照提示做出标准适配姿态,并保持静止。 [0034] 2, the user makes a gesture in accordance with standard adapter tips, and remains stationary.

[0035] 3、系统从标准人体骨骼框架的盆骨位置开始进行骨骼适配,保证盆骨位置与肢体轮廓对应位置相一致。 [0035] 3, the position of the system from a standard human bone pelvis frame skeleton adapted to start to ensure pelvis position coincides with the contour corresponding to the position of the limb.

[0036] 4、移动、缩放标准人体骨骼框架的躯干骨骼至合适高度,保证头部骨骼与肢体轮廓对应位置相一致。 [0036] 4, move, resize standard human bone trunk skeletal frame to a suitable height to ensure that the head and body bone contour corresponding positions coincide.

[0037] 5、移动、缩放标准人体骨骼框架的下肢骨骼至合适位置,保证脚部与肢体轮廓对 [0037] 5, move, resize standard human bone of the lower limb skeletal frame into position to ensure that the contour of the foot and the limb

应位置相一致。 It should be consistent with the position.

[0038] 6、移动、缩放标准人体骨骼框架的上肢骨骼至合适位置,保证双手与肢体轮廓对 [0038] 6, move, resize standard limb bones of the human skeleton framework to a suitable position to ensure that the profile limb of hands

应位置相一致。 It should be consistent with the position.

[0039] 7、检查获得的适配肢体骨骼框架的全部骨骼是否与实际肢体轮廓匹配。 [0039] 7, the limb bones of all bones adaptation framework on the checked matches the actual physical profile. 如果否,则回到上述第3步,从盆骨开始重新适配。 If not, return to step 3 above, start again from the pelvis adaptation. 是,则进入下一步。 Yes, go to the next step.

[0040] 这里的检查,是指对比适配骨架关键点位置是否与用户肢体轮廓相一致,比如骨骼上的头部关键点是否在用户轮廓的头部位置等。 [0040] Here inspection means adapted to compare whether the position of the skeleton key coincides with the user profile limb, such as the head of the key points on the bones of the head position of the user whether the contour or the like.

[0041 ] 8、系统提示用户开始做任意动作。 [0041] 8, the system prompts the user to start doing any action.

[0042] 9、检查全身骨骼是否与实际肢体轮廓的运动动作保持一致。 [0042] 9, to check whether whole body bone and actual limb movement profile of action is consistent.

[0043] 10、完成骨骼匹配过程。 [0043] 10, the matching process is completed bone.

[0044] 本发明通过基于深度图像传感器和深度图像处理单元,可以将用户肢体图像高效的从复杂背景中分离出来,并能获得场景图像的深度信息,进而能够重建用户肢体的骨骼系统,更进一步的能够识别和跟踪用户的肢体动作,最终完成用户肢体动作识别过程,提高肢体动作识别效率,改善人机交互用户体验。 [0044] The present invention can be efficiently separated based on the depth image sensor and the image processing unit depth image from the limb of the user out of the complex background, and can obtain depth information of the scene image, the user can be reconstructed and further limb skeletal system, and further by able to identify and track the user's body movements, body movements to finalize the user identification process, improve the efficiency of body movement recognition, human-computer interaction to improve the user experience.

[0045] 以上所述仅为本发明的较佳实施例而已,并不用以限制本发明,凡在本发明的精神和原则之内所作的任何修改、等同替换和改进等,均应包含在本发明的保护范围之内。 [0045] The foregoing is only preferred embodiments of the present invention but are not intended to limit the present invention, any modifications within the spirit and principle of the present invention, equivalent substitutions and improvements should be included in the present within the scope of the invention.

Claims (10)

1. 一种基于深度图像感应的肢体动作识别系统,其特征在于,包括: 深度图像信息获取单元:用于获取用户及其所在环境的深度图像信息; 肢体轮廓抽取单元:用于从上述深度图像信息的背景中抽取用户肢体轮廓; 标准骨骼适配单元:用于分别改变标准人体骨骼框架中各个部分的大小,使其与上述用户肢体轮廓相适应,获得相应于该用户的适配肢体骨骼框架; 骨骼运动跟踪单元:用于在深度图像信息中,以适配肢体骨骼框架的格式跟踪、抽取表达用户肢体的运动的数据; 肢体动作识别单元:用于根据上述表达用户肢体的运动的数据识别用户的肢体动作。 A body movement sensor recognition system based on the depth image, wherein, comprising: a depth image information acquisition unit: means for obtaining user information and the environment where the depth image; contour extraction block limb: for the depth image from the background information extracted user profile limb; standard bone adaptation unit: means for changing the size of a standard human bone portion of each frame, respectively, so as to adapt said user profile limb, the corresponding adaptation to the user's skeletal frame limbs ; skeleton motion tracking unit: for image information in the depth, to adapt the format of the frame limb skeleton tracking, extracting data expressing motion of the user's limb; body motion recognition unit: the motion based on the data for identifying the user extremity of expression user's body movements.
2.如权利要求I所述的肢体动作识别系统,其特征在于,所述深度图像信息获取单元进一步包括: 深度图像传感单元:用于向用户所在方向发射经过编码的红外结构原始光平面,并接收和感应经由用户及其所在环境反射回来的三维环境对象红外结构光;和深度图像处理单元:用于通过对比上述三维环境对象红外结构光的编码和原始结构光平面的编码,来获取深度图像息。 2. The body motion recognition system according to claim I, wherein the depth image information obtaining unit further comprises: a depth image sensing unit: for encoded original planar infrared light emitting structure where the direction of the user, and receiving reflected and induced via the user objects and the environment where the infrared structured light three-dimensional environment; and a depth image processing means: means for encoding the comparison by encoding the three-dimensional structure of the infrared light environment object and the original plane of the structured light to obtain depth image information.
3.如权利要求2所述的肢体动作识别系统,其特征在于,所述肢体轮廓抽取单元根据连续的深度图像信息中的运动差分分析抽取用户肢体轮廓。 Body movement recognition system according to claim 2, characterized in that the limb contour extraction unit extracts the analysis according to the user profile limb motion depth difference of successive image information.
4.如权利要求3所述的肢体动作识别系统,其特征在于,所述标准骨骼适配单元通过缩放、旋转、变形的计算方法对所述标准人体骨骼框架进行适配。 4. The body of the motion recognition system as claimed in claim 3, wherein said standard bone adaptation unit by scaling, rotation, deformation calculation of the standard human bone adaptation framework.
5.如权利要求4所述的肢体动作识别系统,其特征在于,所述标准人体骨骼框架包括按照正常人体结构相互连接的头部、躯干、盆骨、左上臂、左下臂、左手、右上臂、右下臂、右手、左大腿、左小腿、左脚、右大腿、右小腿和右脚。 5. The body motion recognition system as claimed in claim 4, wherein the standard human skeletal frame comprising a head according to the normal anatomy of interconnected, torso, pelvis, the left upper arm, the lower left arm, left, right upper arm , lower right arm, right hand, left thigh, left leg, left foot, right thigh, right leg and right foot.
6. 一种基于深度图像感应的肢体动作识别方法,其特征在于,包括如下步骤: A、获取用户及其所在环境的深度图像信息; B、从上述深度图像信息的背景中抽取用户肢体轮廓; C、分别改变标准人体骨骼框架中各个部分的大小,使其与上述用户肢体轮廓相适应,获得相应于该用户的适配肢体骨骼框架; D、在深度图像信息中,以适配肢体骨骼框架的格式跟踪、抽取表达用户肢体的运动的数据; E、根据表达用户肢体的运动的数据识别用户的肢体动作。 A body motion recognition method based on the depth image sensor, characterized by comprising the steps of: A, acquires the user information and the environment where the depth image; B, extracts the user profile limb from the background depth information in the image; C, respectively, to change the size of a standard human bone portion of each frame, so as to adapt said user profile limb, the corresponding adaptation to the user's skeletal frame limb; D, in the depth image information in the frame to fit the limb bones track format, expression data extracted motion of the user's limb; E, body movements in accordance with the user identification data expressing motion of the user's limb.
7.如权利要求6所述的肢体动作识别方法,其特征在于,所述步骤C中进一步包括如下步骤: Cl、根据标准人体骨骼框架和步骤B中获取的用户肢体轮廓,保证盆骨的对应位置相一致; C2、移动、缩放标准人体骨骼框架的躯干骨骼至合适高度,保证头部骨骼的对应位置相一致; C3、移动、缩放标准人体骨骼框架的下肢骨骼至合适位置,保证脚部的对应位置相一致; C4、移动、缩放标准人体骨骼框架的上肢骨骼至合适位置,保证双手对应位置相一致; C5、检查对比适配骨架关键点位置是否与所述用户肢体轮廓相一致,如果否,则回到上述步骤Cl,从盆骨开始重新适配;如果是,则进入下一步。 7. The body of the motion recognition method as claimed in claim 6, wherein the step C further comprising the step of: Cl, standard human user profile limb skeletal frame obtained in step B and to ensure a corresponding pelvis positions coincide; C2, move, resize standard human bone trunk skeletal frame to a suitable height corresponding to the position of the head to ensure consistent bone; a C3, movement, zoom level lower extremity bones of the human skeleton framework to a suitable location, in the foot corresponding positions coincide; C4, move, resize standard limb bones of the human skeleton framework to a suitable position to ensure that the hands corresponding positions coincide; C5, that checks the key position is adapted to the skeleton consistent with the limb of the user profile, and if not , then returns to the step Cl, adapted to re-start from the pelvis; if yes, the process proceeds to the next step.
8.如权利要求6所述的肢体动作识别方法,其特征在于,所述步骤A中采用双目视觉技术、或者飞行时间技术、或者结构光编码技术获取用户及其所在环境的深度图像信息。 Said body motion recognition method as claimed in claim 6, wherein said step A binocular vision, or time of flight technique, or structured light coding techniques to obtain the user location and the depth image information environment.
9.如权利要求8所述的肢体动作识别方法,其特征在于,所述步骤A采用结构光编码技术获取用户及其所在环境的深度图像信息的方法,进一步包括如下步骤: Al、向用户所在方向发射经过编码的红外结构原始光平面,并接收和感应经由用户及其所在环境反射回来的三维环境对象红外结构光; A2、通过对比上述三维环境对象红外结构光的编码和原始结构光平面的编码,来获取深度图像信息。 9. The body of the motion recognition method as claimed in claim 8, wherein said step of using structured light A coding method for obtaining the depth image information user and the environment where, further comprising the step of: Al, where the user after the original direction of the emitted infrared light plane coding structure, and to receive and sense the user via the three-dimensional environment and the environment where the object reflected infrared back light structure; A2, by comparing the light plane and encoding the original structure of the three-dimensional structure of the infrared light environment object coding, to obtain the depth image information.
10.如权利要求7所述的肢体动作识别方法,其特征在于,还包括: C6、在用户动作时检查肢体轮廓是否与全身骨骼的实际运动保持一致。 10. The body motion recognition method according to claim 7, characterized in that, further comprising: C6, check whether the contour of the limb consistent with the actual movement of the whole body bone when a user operation.
CN201110160313XA 2011-06-15 2011-06-15 Body action identification method and system based on depth image induction CN102831380A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201110160313XA CN102831380A (en) 2011-06-15 2011-06-15 Body action identification method and system based on depth image induction

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201110160313XA CN102831380A (en) 2011-06-15 2011-06-15 Body action identification method and system based on depth image induction

Publications (1)

Publication Number Publication Date
CN102831380A true CN102831380A (en) 2012-12-19

Family

ID=47334511

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201110160313XA CN102831380A (en) 2011-06-15 2011-06-15 Body action identification method and system based on depth image induction

Country Status (1)

Country Link
CN (1) CN102831380A (en)

Cited By (12)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103735268A (en) * 2013-09-29 2014-04-23 沈阳东软医疗系统有限公司 Body position detecting method and system
CN103995587A (en) * 2014-05-13 2014-08-20 联想(北京)有限公司 Information control method and electronic equipment
CN104361321A (en) * 2014-11-13 2015-02-18 侯振杰 Methods of judging fall behaviors and body balance for old people
CN104353240A (en) * 2014-11-27 2015-02-18 北京师范大学珠海分校 Running machine system based on Kinect
CN104598867A (en) * 2013-10-30 2015-05-06 中国艺术科技研究所 Automatic evaluation method of human body action and dance scoring system
CN104834913A (en) * 2015-05-14 2015-08-12 中国人民解放军理工大学 Flag signal identification method and apparatus based on depth image
CN105229411A (en) * 2013-04-15 2016-01-06 微软技术许可有限责任公司 Robust stereo depth system
CN105824006A (en) * 2014-12-23 2016-08-03 国家电网公司 Method for eliminating safety hidden danger of substation personnel
CN106022213A (en) * 2016-05-04 2016-10-12 北方工业大学 Human body motion recognition method based on three-dimensional bone information
CN106250867A (en) * 2016-08-12 2016-12-21 南京华捷艾米软件科技有限公司 Realization method of human-body bone tracking system based on depth data
CN106529399A (en) * 2016-09-26 2017-03-22 深圳奥比中光科技有限公司 Human body information acquisition method, device and system
CN106650217A (en) * 2015-10-29 2017-05-10 佳能市场营销日本株式会社 Information processing apparatus and information processing method

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101256672A (en) * 2008-03-21 2008-09-03 北京中星微电子有限公司 Object image depth restruction apparatus based on video camera apparatus as well as projecting apparatus thereof
CN101388114A (en) * 2008-09-03 2009-03-18 北京中星微电子有限公司 Method and system for estimating human body attitudes
CN101765020A (en) * 2008-12-23 2010-06-30 康佳集团股份有限公司 Television capable of sensing stereo image
US20100197400A1 (en) * 2009-01-30 2010-08-05 Microsoft Corporation Visual target tracking

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101256672A (en) * 2008-03-21 2008-09-03 北京中星微电子有限公司 Object image depth restruction apparatus based on video camera apparatus as well as projecting apparatus thereof
CN101388114A (en) * 2008-09-03 2009-03-18 北京中星微电子有限公司 Method and system for estimating human body attitudes
CN101765020A (en) * 2008-12-23 2010-06-30 康佳集团股份有限公司 Television capable of sensing stereo image
US20100197400A1 (en) * 2009-01-30 2010-08-05 Microsoft Corporation Visual target tracking

Cited By (21)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105229411A (en) * 2013-04-15 2016-01-06 微软技术许可有限责任公司 Robust stereo depth system
US9928420B2 (en) 2013-04-15 2018-03-27 Microsoft Technology Licensing, Llc Depth imaging system based on stereo vision and infrared radiation
US10268885B2 (en) 2013-04-15 2019-04-23 Microsoft Technology Licensing, Llc Extracting true color from a color and infrared sensor
CN103735268A (en) * 2013-09-29 2014-04-23 沈阳东软医疗系统有限公司 Body position detecting method and system
CN103735268B (en) * 2013-09-29 2015-11-25 沈阳东软医疗系统有限公司 Method and system for detecting one kind of position
CN104598867B (en) * 2013-10-30 2017-12-01 中国艺术科技研究所 A personal operation of the automatic scoring system assessment and dance
CN104598867A (en) * 2013-10-30 2015-05-06 中国艺术科技研究所 Automatic evaluation method of human body action and dance scoring system
CN103995587A (en) * 2014-05-13 2014-08-20 联想(北京)有限公司 Information control method and electronic equipment
CN103995587B (en) * 2014-05-13 2017-09-29 联想(北京)有限公司 An information control method for an electronic device, and
CN104361321A (en) * 2014-11-13 2015-02-18 侯振杰 Methods of judging fall behaviors and body balance for old people
CN104361321B (en) * 2014-11-13 2018-02-09 侯振杰 A method for judging the behavior of falls in the elderly and body balance method
CN104353240A (en) * 2014-11-27 2015-02-18 北京师范大学珠海分校 Running machine system based on Kinect
CN105824006A (en) * 2014-12-23 2016-08-03 国家电网公司 Method for eliminating safety hidden danger of substation personnel
CN104834913A (en) * 2015-05-14 2015-08-12 中国人民解放军理工大学 Flag signal identification method and apparatus based on depth image
CN104834913B (en) * 2015-05-14 2018-04-03 中国人民解放军理工大学 Based semaphore depth image recognition method and apparatus
CN106650217A (en) * 2015-10-29 2017-05-10 佳能市场营销日本株式会社 Information processing apparatus and information processing method
CN106022213B (en) * 2016-05-04 2019-06-07 北方工业大学 A kind of human motion recognition method based on three-dimensional bone information
CN106022213A (en) * 2016-05-04 2016-10-12 北方工业大学 Human body motion recognition method based on three-dimensional bone information
CN106250867B (en) * 2016-08-12 2017-11-14 南京华捷艾米软件科技有限公司 Implementation of the human skeleton tracking system based on depth data
CN106250867A (en) * 2016-08-12 2016-12-21 南京华捷艾米软件科技有限公司 Realization method of human-body bone tracking system based on depth data
CN106529399A (en) * 2016-09-26 2017-03-22 深圳奥比中光科技有限公司 Human body information acquisition method, device and system

Similar Documents

Publication Publication Date Title
CA2767788C (en) Bringing a visual representation to life via learned input from the user
KR100948095B1 (en) Motion-input device for a computing terminal and method of its operation
US9519989B2 (en) Visual representation expression based on player expression
JP5845184B2 (en) Human tracking system
Suarez et al. Hand gesture recognition with depth images: A review
JP3863809B2 (en) Input system by the image recognition of hand
CA2804902C (en) A method circuit and system for human to machine interfacing by hand gestures
JP5255623B2 (en) Volume recognition method and system
US9383823B2 (en) Combining gestures beyond skeletal
US9468848B2 (en) Assigning gesture dictionaries
US20100277489A1 (en) Determine intended motions
US10331731B2 (en) Method and apparatus for identifying input features for later recognition
US9519828B2 (en) Isolate extraneous motions
US7042440B2 (en) Man machine interfaces and applications
KR101765970B1 (en) Voice-body identity correlation
US20070139370A1 (en) Motion recognition system and method for controlling electronic devices
CN103246351B (en) A user interactive system and method
US20130010071A1 (en) Methods and systems for mapping pointing device on depth map
US8565479B2 (en) Extraction of skeletons from 3D maps
CN102470274B (en) Automatic generation of visual representation
CN101344816B (en) Human-machine interaction method and device based on sight tracing and gesture discriminating
US8140339B2 (en) Method and apparatus for translating hand gestures
US20110228976A1 (en) Proxy training data for human body tracking
US10234941B2 (en) Wearable sensor for tracking articulated body-parts
US20150245010A1 (en) Apparatus and system for interfacing with computers and other electronic devices through gestures by using depth sensing and methods of use

Legal Events

Date Code Title Description
C06 Publication
C10 Entry into substantive examination
RJ01 Rejection of invention patent application after publication