CN111913577A - Three-dimensional space interaction method based on Kinect - Google Patents

Three-dimensional space interaction method based on Kinect Download PDF

Info

Publication number
CN111913577A
CN111913577A CN202010763675.7A CN202010763675A CN111913577A CN 111913577 A CN111913577 A CN 111913577A CN 202010763675 A CN202010763675 A CN 202010763675A CN 111913577 A CN111913577 A CN 111913577A
Authority
CN
China
Prior art keywords
color
kinect
partition image
dimensional space
depth
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202010763675.7A
Other languages
Chinese (zh)
Inventor
程琪
李强生
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Wuhan Muzigong Digital Technology Co ltd
Original Assignee
Wuhan Muzigong Digital Technology Co ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Wuhan Muzigong Digital Technology Co ltd filed Critical Wuhan Muzigong Digital Technology Co ltd
Priority to CN202010763675.7A priority Critical patent/CN111913577A/en
Publication of CN111913577A publication Critical patent/CN111913577A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/011Arrangements for interaction with the human body, e.g. for user immersion in virtual reality

Landscapes

  • Engineering & Computer Science (AREA)
  • General Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Human Computer Interaction (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • User Interface Of Digital Computer (AREA)
  • Processing Or Creating Images (AREA)

Abstract

The invention relates to a three-dimensional space interaction method based on Kinect, which comprises the following steps: creating a color partition image, wherein the color partition image has the same resolution as the Kinect depth map; loading the color partition image and storing coordinates and color information corresponding to each pixel; acquiring the coordinate of the center position of the current target in the Kinect depth image, and acquiring the color information of the current target in the color partition image according to the coordinate; and judging whether the target is positioned in a space for triggering interactive operation or not according to the color information and the depth information of the target in the color partition image. According to the invention, the region is divided by using the color of the picture, the picture can define the pattern by user, and the interaction detection of the irregular region is more accurate and flexible.

Description

Three-dimensional space interaction method based on Kinect
Technical Field
The invention relates to the field of computer image processing, relates to a three-dimensional space interaction technology, and particularly relates to a three-dimensional space interaction method based on Kinect.
Background
NUI is based on the mutual mode of human instinct, and people can utilize modes such as pronunciation, gesture, catch of eye, limbs to realize man-machine interaction, and this kind of mutual mode more approaches human spontaneous behavior mode, can alleviate user's study burden. Somatosensory interaction (somatosensory interaction) is the mainstream trend of NUI, and therefore, becomes the most popular direction in the field of natural interaction research at present.
The somatosensory interaction means that a user directly uses limb actions to interact with peripheral devices or environments without any control equipment, so that the user has a feeling of being personally on the scene. The somatosensory technology generally needs the support of computer technologies such as gesture recognition, bone-clanging tracking, facial expression recognition and the like. Common somatosensory interaction equipment comprises Wii (world information interface) released by Nintendo corporation of Japan, Microsoft Kinect, a somatosensory controller LeapMotion released by Leap of somatosensory controller manufacturing company and the like, and a user can send an instruction through gestures, postures or actions in the process of interacting with machine equipment to realize a natural interaction mode.
The Kinect V2 has three cameras in total, the RGB camera on the left side is used for acquiring a color image with the maximum resolution of 1920 x 1080, and at most 30 frames of images are acquired every second; the remaining two cameras are depth sensors, the left side is an infrared emitter, the right side is an infrared receiver, used to obtain the operator's position: kinect is flanked by microphone arrays to obtain stable Speech information, which can be used for Speech recognition in conjunction with Microsoft Speech platform SDK: the base is internally provided with a motor which can rotate left and right along with the movement of the object.
Disclosure of Invention
Aiming at the technical problems in the prior art, the invention provides a three-dimensional space interaction method based on Kinect, which comprises the following steps: creating a color partition image, wherein the color partition image has the same resolution as the Kinect depth map; storing the coordinate and color information corresponding to each pixel; acquiring the coordinate of the center position of the current target in the Kinect depth image, and acquiring the color information of the current target in the color partition image according to the coordinate; and judging whether the target is positioned in a space for triggering interactive operation or not according to the color information and the depth information of the target in the color partition image.
In some embodiments of the invention, the color-partition image distinguishes regions of different depth information by different colors. Preferably, the color-partition image is customizable. Further, the color partition image is completely overlapped with the Kinect depth map.
In some embodiments of the invention, the color information comprises a name of a color or a numerical representation of a pixel. Specifically, the color name can be expressed in languages of chinese, english, and the like, or can be identified in common RGB and YUV forms.
In some embodiments of the present invention, the coordinate of the center position of the current target in the depth map and the depth information further include threshold filtering the depth map.
In some embodiments of the present invention, the obtaining of the coordinates of the center position of the current target in the Kinect depth map is implemented by OpenCV.
In some embodiments of the invention, the interaction comprises a somatosensory interaction.
The beneficial effect of adopting the further scheme is that: the image color is used for dividing the area, the pattern can be defined by the image, and the interaction detection of the irregular area is more accurate and flexible.
Drawings
FIG. 1 is a diagram of basic steps of a Kinect-based three-dimensional space interaction method according to some embodiments of the present invention;
FIG. 2 is a diagram of basic steps of a Kinect-based three-dimensional space interaction method according to some embodiments of the present invention;
FIG. 3a is a color-zoned image of some embodiments of the present invention; FIG. 3b is a Kinect measured depth map according to some embodiments of the present invention;
FIG. 4 is a diagram illustrating a common gesture interaction.
Detailed Description
The principles and features of this invention are described below in conjunction with the following drawings, which are set forth by way of illustration only and are not intended to limit the scope of the invention.
As shown in fig. 1, a three-dimensional space interaction method based on Kinect includes the following steps: s1, creating a color partition image, wherein the color partition image has the same resolution as a Kinect depth image; s2, loading the color partition image, and storing coordinates and color information corresponding to each pixel; s3, obtaining the coordinate of the center position of the current target in the Kinect depth image, and obtaining the color information of the current target in the color partition image according to the coordinate; and S4, judging whether the target is positioned in a space for triggering interactive operation according to the color information and the depth information of the target in the color partition image.
It should be noted that the RGB camera resolution of the first generation Kinect is 640 × 480(Kinect v1), and the depth camera resolution is 320 × 240; the second generation RGB camera resolution was 1920 x 1080 and the depth camera resolution was 512 x 424(Kinect v 2). The above interaction methods include, but are not limited to, the Kinect generation or the second generation.
In some embodiments of the present invention, the color-partition image distinguishes regions with different depth information by different colors. Preferably, the color-partition image is customizable. In the color partition map in fig. 3, the partition map is generally similar to an ellipse, and the upper, middle and lower regions are divided according to different colors, each region represents different depth information, and the depth information in the same region is the same or in the same numerical value interval. In particular, the color-partition image is completely coincident with the Kinect depth map for debugging convenience or computational convenience.
In some embodiments of the invention, the color information comprises a name of a color or a numerical representation of a pixel. Specifically, the color name may be expressed in chinese, english, or the like, and the value of the pixel may be expressed in the form of RGB, HIS, HSV, CMYK, YUV, or the like, which is common.
As shown in fig. 2 and fig. 3, in some embodiments of the present invention, a three-dimensional space interaction method based on Kinect includes the following specific steps: and step S1, installing a computer and a Kinect software and hardware environment, and confirming that the drive and the operation are normal. And hanging the Kinect downwards for installation, wherein the installation height is 4.5 meters lower than the detection effective range of the Kinect depth camera.
Step S2, the picture in fig. 3a is loaded, and each pixel is stored in an array such as: ColorArray [ (x1, y1, black), (x1, y2, black) … (x512, y424, color) ], each element of the array contains pixel coordinate information x, y and color information of the current pixel.
Step S3, acquiring each frame of Kinect infrared depth map and determining whether an object is detected, and storing information of the depth map into an array such as:
DepthArray=[(x1,y1,4),(x1,y2,4)…(x512,y424,depth)]。
in step S4, when an object (target) enters the Kinect depth camera detection area, such as a white dot in fig. 3b, the central position coordinates of the white dot are obtained by OpenCV processing of the depth map.
Step S5, because the depth map coincides with the color partition map coordinates of fig. 3a, after the coordinates of the white point of fig. 2 are obtained in step S4, the coordinates of the white point in fig. 3a are obtained, and then the color of the pixel at the current coordinate position is obtained from the ColorArray array in step S2, so that the planar area where the current object is located can be obtained according to the color.
In step S6, after the object coordinates are obtained in step S4, the depth information depth of the object is obtained from the depthacray array in step S3, and the spatial region where the current object is located is obtained. Therefore, the interactive operation can be triggered when an object enters a specific three-dimensional space region.
In some embodiments of the invention, the interaction comprises a somatosensory interaction. In particular, a common gesture operation for 3D interaction is shown in fig. 4.
The above description is only for the purpose of illustrating the preferred embodiments of the present invention and is not to be construed as limiting the invention, and any modifications, equivalents, improvements and the like that fall within the spirit and principle of the present invention are intended to be included therein.

Claims (8)

1. A three-dimensional space interaction method based on Kinect is characterized by comprising the following steps:
creating a color partition image, wherein the color partition image has the same resolution as the Kinect depth map;
loading the color partition image and storing coordinates and color information corresponding to each pixel;
acquiring the coordinate of the center position of the current target in the Kinect depth image, and acquiring the color information of the current target in the color partition image according to the coordinate;
and judging whether the target is positioned in a space for triggering interactive operation or not according to the color information and the depth information of the target in the color partition image.
2. The Kinect-based three-dimensional space interaction method as claimed in claim 1, wherein the color partition image distinguishes regions with different depth information by different colors.
3. The Kinect-based three-dimensional space interaction method as recited in any one of claims 1 or 2, wherein the color-partition image is customizable.
4. The Kinect-based three-dimensional space interaction method as recited in claim 3, wherein the color partition image is completely overlapped with the Kinect depth map.
5. The Kinect-based three-dimensional space interaction method as recited in claim 1 or 2, wherein the color information comprises a name of a color or a numerical representation of a pixel.
6. The method of three-dimensional interaction of Kinect as claimed in claim 1 or 2, wherein the coordinate of the center position of the current target in the depth map and the depth information further comprises performing a threshold filtering process on the depth map.
7. The Kinect three-dimensional space interaction method as claimed in claim 1 or 2, wherein the obtaining of the coordinates of the center position of the current target in the Kinect depth map is implemented by OpenCV.
8. The three-dimensional space interaction method of Kinect as claimed in claim 1 or 2, the interaction operation comprising somatosensory interaction.
CN202010763675.7A 2020-07-31 2020-07-31 Three-dimensional space interaction method based on Kinect Pending CN111913577A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202010763675.7A CN111913577A (en) 2020-07-31 2020-07-31 Three-dimensional space interaction method based on Kinect

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202010763675.7A CN111913577A (en) 2020-07-31 2020-07-31 Three-dimensional space interaction method based on Kinect

Publications (1)

Publication Number Publication Date
CN111913577A true CN111913577A (en) 2020-11-10

Family

ID=73287562

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202010763675.7A Pending CN111913577A (en) 2020-07-31 2020-07-31 Three-dimensional space interaction method based on Kinect

Country Status (1)

Country Link
CN (1) CN111913577A (en)

Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103207709A (en) * 2013-04-07 2013-07-17 布法罗机器人科技(苏州)有限公司 Multi-touch system and method
US20150002419A1 (en) * 2013-06-26 2015-01-01 Microsoft Corporation Recognizing interactions with hot zones
CN104360729A (en) * 2014-08-05 2015-02-18 北京农业信息技术研究中心 Multi-interactive method and device based on Kinect and Unity 3D
US20160191879A1 (en) * 2014-12-30 2016-06-30 Stephen Howard System and method for interactive projection
CN109145802A (en) * 2018-08-14 2019-01-04 清华大学 More manpower gesture man-machine interaction methods and device based on Kinect
CN109513157A (en) * 2018-10-16 2019-03-26 广州嘉影软件有限公司 Training exercises exchange method and system based on Kinect somatosensory
CN110045821A (en) * 2019-03-12 2019-07-23 杭州电子科技大学 A kind of augmented reality exchange method of Virtual studio hall

Patent Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103207709A (en) * 2013-04-07 2013-07-17 布法罗机器人科技(苏州)有限公司 Multi-touch system and method
US20150002419A1 (en) * 2013-06-26 2015-01-01 Microsoft Corporation Recognizing interactions with hot zones
CN105518584A (en) * 2013-06-26 2016-04-20 微软技术许可有限责任公司 Recognizing interactions with hot zones
CN104360729A (en) * 2014-08-05 2015-02-18 北京农业信息技术研究中心 Multi-interactive method and device based on Kinect and Unity 3D
US20160191879A1 (en) * 2014-12-30 2016-06-30 Stephen Howard System and method for interactive projection
CN109145802A (en) * 2018-08-14 2019-01-04 清华大学 More manpower gesture man-machine interaction methods and device based on Kinect
CN109513157A (en) * 2018-10-16 2019-03-26 广州嘉影软件有限公司 Training exercises exchange method and system based on Kinect somatosensory
CN110045821A (en) * 2019-03-12 2019-07-23 杭州电子科技大学 A kind of augmented reality exchange method of Virtual studio hall

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
裴以建;韩甲甲;闫哲;薛端;: "Kinect彩色图像在光标移动控制中的应用", 计算机工程, no. 10, pages 241 - 245 *
贾丙佳;李平;: "人机交互过程中数字手势的识别方法", 华侨大学学报(自然科学版), no. 02 *

Similar Documents

Publication Publication Date Title
US11896893B2 (en) Information processing device, control method of information processing device, and program
CN111880657B (en) Control method and device of virtual object, electronic equipment and storage medium
EP2956882B1 (en) Managed biometric identity
KR20200124280A (en) Motion recognition, driving motion analysis method and device, electronic device
CN109996051B (en) Projection area self-adaptive dynamic projection method, device and system
US20150002419A1 (en) Recognizing interactions with hot zones
CN111417989B (en) Program, information processing method, information processing system, head-mounted display device, and information processing device
JP5161435B2 (en) Image processing apparatus, image processing system, computer control method, and program
CN111199583B (en) Virtual content display method and device, terminal equipment and storage medium
KR20170014491A (en) Method and device for recognizing motion
CN108563327B (en) Augmented reality method, device, storage medium and electronic equipment
JP7162079B2 (en) A recording medium for recording a method, system and computer program for remotely controlling a display device via head gestures
US20170228047A1 (en) Information processing device, information processing method and program
US11284020B2 (en) Apparatus and method for displaying graphic elements according to object
CN110489027A (en) Handheld input device and its display position control method and device for indicating icon
US20180032142A1 (en) Information processing apparatus, control method thereof, and storage medium
KR102565444B1 (en) Method and apparatus for identifying object
KR102572675B1 (en) Method and electronic device for adaptively configuring user interface
CN105468249B (en) Intelligent interaction system and its control method
WO2023142555A1 (en) Data processing method and apparatus, computer device, storage medium, and computer program product
JP6452585B2 (en) Information processing apparatus and position information acquisition method
CN111913577A (en) Three-dimensional space interaction method based on Kinect
CN109144598A (en) Electronics mask man-machine interaction method and system based on gesture
CN110489026A (en) A kind of handheld input device and its blanking control method and device for indicating icon
KR101048406B1 (en) Game system and method for recognizing user posture as user command

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination