CN102902355A - Space interaction method of mobile equipment - Google Patents
Space interaction method of mobile equipment Download PDFInfo
- Publication number
- CN102902355A CN102902355A CN2012103201662A CN201210320166A CN102902355A CN 102902355 A CN102902355 A CN 102902355A CN 2012103201662 A CN2012103201662 A CN 2012103201662A CN 201210320166 A CN201210320166 A CN 201210320166A CN 102902355 A CN102902355 A CN 102902355A
- Authority
- CN
- China
- Prior art keywords
- pixel
- hand
- image
- interaction method
- virtual scene
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 86
- 230000003993 interaction Effects 0.000 title claims abstract description 39
- 230000011218 segmentation Effects 0.000 claims abstract description 12
- 230000004044 response Effects 0.000 claims abstract description 8
- 230000008859 change Effects 0.000 claims abstract description 6
- 239000004744 fabric Substances 0.000 claims description 17
- 230000033001 locomotion Effects 0.000 claims description 17
- 238000000605 extraction Methods 0.000 claims description 10
- 239000000284 extract Substances 0.000 claims description 8
- 238000001914 filtration Methods 0.000 claims description 6
- 230000006870 function Effects 0.000 claims description 4
- 230000002452 interceptive effect Effects 0.000 claims description 3
- 230000010365 information processing Effects 0.000 claims description 2
- 239000011159 matrix material Substances 0.000 claims description 2
- 230000007704 transition Effects 0.000 claims 1
- 238000013507 mapping Methods 0.000 abstract 1
- 230000002123 temporal effect Effects 0.000 abstract 1
- 230000008569 process Effects 0.000 description 18
- 238000010586 diagram Methods 0.000 description 11
- 238000001514 detection method Methods 0.000 description 7
- 230000001133 acceleration Effects 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 2
- 238000003384 imaging method Methods 0.000 description 2
- 230000006872 improvement Effects 0.000 description 2
- 238000012545 processing Methods 0.000 description 2
- 241000282326 Felis catus Species 0.000 description 1
- 238000012937 correction Methods 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000009792 diffusion process Methods 0.000 description 1
- 230000009977 dual effect Effects 0.000 description 1
- 239000000463 material Substances 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000001915 proofreading effect Effects 0.000 description 1
- 239000007787 solid Substances 0.000 description 1
- 230000009466 transformation Effects 0.000 description 1
- 238000013519 translation Methods 0.000 description 1
Images
Landscapes
- Image Processing (AREA)
Abstract
The invention discloses a space interaction method of mobile equipment with the human-computer interaction capacity. The method includes the steps of rebuilding a depth map of an image according to the two-dimensional image shot by the mobile equipment in real time; performing region segmentation on the depth map to obtain a human body position region in the depth map; mapping the human body position region in the depth map to a virtual scene; and detecting whether the human body position in the virtual scene collides with other objects in the virtual scene, if the collision happens, determining a body language according to the temporal space change of the human body position in the virtual scene, and the virtual scene making response according to the body. According to the method, accuracy and timeliness of space interaction can be improved.
Description
Technical field
The present invention relates to the fields such as image processing, three-dimensional image reconstruction, man-machine interaction and computer utility, relate in particular to a kind of spatial interaction method of mobile device.
Background technology
The spatial interaction technology of mobile device is to utilize the mobile device (such as mobile phone, IPad etc.) with interactive capability to carry out spatial interaction, so that the dummy object in the mobile device can be made corresponding response according to people's indication.Fig. 1 is the synoptic diagram of the spatial interaction technology of existing mobile device.As shown in Figure 1, show the image of a width of cloth cat in the mobile device, spatial interaction by mobile device, we can make in the place ahead (place that its camera can be taken) of mobile device the gesture of the grasping movement of a vacation of hand, mobile device makes the virtual cat of its demonstration by virtual hand crawl by this gesture of identification.
The spatial interaction method generally includes that depth image is rebuild, hand region is cut apart, Unitary coordinate, collision detection, motion process description and the steps such as gesture identification, virtual scene response, and wherein to rebuild be one of them committed step to depth image.
In computer vision field, depth map is being played the part of very important role, and its quick obtaining seems extremely important.Depth map not only can generate stereo-picture, can also realize that the reconstruction of three-dimensional model and image-based play up.Wherein in real-time reconstructing three-dimensional model, it is particularly important that the accuracy that depth map obtains and real-time seem.
For the diversified method of having obtained of depth map, such as laser imaging radar, laser measuring machine method, structured light method, computer stereo vision method etc.Wherein the computer stereo vision method is a kind of traditional depth map acquisition methods, comprises monocular, binocular and multi-view stereo vision method.In reconstructing three-dimensional model, mostly adopt the method for computer stereo vision at present.Yet this method but exists some shortcomings and treats improvements: for the feature point extraction accuracy, because the texture variations among the different images is different, too smooth zone is difficult to extract accurately unique point; For the real-time of feature point extraction, different extracting method exists very large difference in speed, often when guaranteeing real-time, has lost accuracy, and when guaranteeing accuracy, has lost real-time.Therefore, how to guarantee simultaneously that accuracy and real-time are important topics.
Summary of the invention
The technical matters that (one) will solve
Technical matters to be solved by this invention is the spatial interaction method that proposes a kind of mobile device, to solve existing spatial interaction technology accuracy and the not good problem of real-time.
(2) technical scheme
The present invention proposes a kind of spatial interaction method that is applied to mobile device, and described mobile device refers to have the portable information processing device of interactive capability and image camera function, and this exchange method comprises the steps:
Step 4, detect in the described virtual scene human body whether with this scene in other objects bump, if bump, then forward step 6 to, if do not bump, then forward step 7 to;
Step 5, determine people's body language according to the spatial variations in time of the human body in the described virtual scene;
Step 6, described virtual scene are made response according to described body language;
Step 7, judge whether the image of the human body that described mobile device is taken upgrades, if upgrade, then turns back to step 1, if do not upgrade, then the method finishes.
(3) beneficial effect
The present invention aspect the reconstruction of depth map, efficiently solves accuracy and the real-time of obtaining depth image by the spatial interaction method of the mobile device algorithm when depth map reconstruction, Region Segmentation, the collision detection is carried out brand-new design; Aspect Region Segmentation, can accurately extract hand region in conjunction with connected domain and skin color segmentation method; Aspect collision detection, use the method for ball bounding box, guaranteed mutual real-time.
Description of drawings
Fig. 1 is existing spatial interaction technology synoptic diagram;
Fig. 2 is the process flow diagram of the spatial interaction method of mobile device proposed by the invention;
Fig. 3 is the main flow chart of depth image reconstruction procedures of the present invention;
Fig. 4 is the process flow diagram that generates describer in the depth image reconstruction procedures of the present invention;
Fig. 5 is generating depth map flow chart of steps in the depth image reconstruction procedures of the present invention;
Fig. 6 is hand region segmentation procedure main flow chart of the present invention;
Fig. 7 is the process flow diagram of neighborhood determining step in the hand region segmentation procedure of the present invention;
Fig. 8 is the process flow diagram of colour of skin determining step in the hand region segmentation procedure of the present invention;
Fig. 9 is collision detection flow chart of steps of the present invention;
Figure 10 is the process flow diagram of Unitary coordinate step of the present invention;
Figure 11 is the process flow diagram of motion process description of the present invention and gesture identification step;
Figure 12 is the integer representation synoptic diagram of two-dimensional space four direction in motion process description of the present invention and the gesture identification step.
Embodiment
For making the purpose, technical solutions and advantages of the present invention clearer, below in conjunction with specific embodiment, and with reference to accompanying drawing, the present invention is described in further detail.
Fig. 2 is the process flow diagram of the spatial interaction method of mobile device proposed by the invention, and the spatial interaction method of described mobile device specifically may further comprise the steps:
The method that has the depth map of multiple acquisition realtime graphic in the prior art for example adopts the mobile device with multi-angled shooting function that hand images is taken, and afterwards the X-Y scheme that obtains is carried out depth reconstruction.In one embodiment of the invention, described mobile device with multi-angled shooting function refers to have the movably man-machine interaction instrument of dual camera.This step is carried out real-time deep figure reconstruction with the image of binocular mobile device collection, and so-called binocular refers to be positioned at two image collecting devices with suitable distance on same plane and the same level line.Adopt binocular mobile device real-time image acquisition, by the camera calibration pre-service, the binocular image that collects is carried out polar curve proofread and correct, realize that the row of binocular image is aimed at.Binocular image after proofreading and correct is carried out the steps such as continuity check that quick parallax figure rebuild and carried out the later stage, filtering, at last disparity map is converted to depth map.
According to one embodiment of the present invention, this step can adopt the growth of connected domain and the colour of skin to judge that the mode that combines carries out.
Because in spatial interaction, hand often is positioned at the front scenic spot of image, so the degree of depth that it produces is maximum, so search the corresponding pixel of depth capacity, it is defined as the hand position pixel, then to all directions growth district, if the pixel in this neighborhood of pixel points meets certain rule, then these pixels are considered as the same area.It is all out divided that described rule should guarantee to be positioned in the image pixel of hand region, if for example the absolute value of the gray scale difference value between adjacent two pixels is less than defined threshold, can determine that then these adjacent two pixels are arranged in same connected domain.
Then, can continue to adopt the method for colour of skin judgement to be partitioned into accurate hand region.The method is namely obtained the hand colour element dot information that has split, and just is the colour of skin if certain pixel satisfies given condition, otherwise is the non-colour of skin, and this pixel is rejected.
The reconstruction depth image of the hand region that this step will split from realistic space is mapped in the virtual scene.Determine in virtual scene and the depth image after the ratio between the hand region, the image coordinate of hand region in the depth image is mapped in the virtual scene, i.e. the generating virtual hand.
Step 4, detect in the described virtual scene human body whether with this virtual scene in other objects bump, if bump, then forward step 6 to, if do not bump, then forward step 7 to.
According to one embodiment of the present invention, the some set that obtains virtual hand in the step 3 can be become the triangle gridding set, and adopt the technology of ball bounding box to carry out described detection.
Step 5, determine people's body language according to the spatial variations in time of the human body in the described virtual scene.
According to a kind of embodiment of the present invention, determine the behavior of hand gesture according to rules such as the direction of motion of this hand region in the successive frame of virtual scene, speed, acceleration.In a specific embodiment, can define six gestures: move to left, move to right, on move, move down, push away, draw, wherein push away and draw to refer to respectively do perpendicular to binocular mobile device plane drawing near and from the close-by examples to those far off gesture.
Step 6, described virtual scene are made response according to described body language.
This step for example is that hand is made the gesture that moves to left, and the virtual hand in the virtual scene is also made the gesture that moves to left, and dummy object is made response according to the gesture that moves to left of virtual hand.
Step 7 judges whether the image of the human body that described a plurality of video camera is taken upgrades, if upgrade, then turns back to step 1, if do not have, then this method finishes.
Because therefore the real-time of spatial interaction technology needs judge whether data have renewal, if having, then gets back to step 1, the binocular image of new collection is carried out depth image reconstruction (module 101); If no, then processing finishes.
The below is described in detail the key step that relates in the said method.
Described human body typically refers to hand region.Before the hand region in the realistic space is transformed into virtual scene, need to carries out first the reconstruction of depth image, thereby obtain hand region in the position of realistic space.Fig. 3 is the process flow diagram according to the reconstruction depth map of one embodiment of the present invention, in this embodiment, and the two dimensional image of two width of cloth hand region about described two dimensional image is taken by a binocular mobile device.As shown in Figure 3, the specific implementation process of depth image reconstruction further may further comprise the steps:
Need to calibrate pre-service work to the binocular mobile device at this.Obtain focal length, imaging initial point and the distortion factor of each image collecting device by calibration, thereby determine the correspondent transform relation of image coordinate and world coordinates.Carry out afterwards the solid calibration, obtain rotation matrix and the translation vector of binocular mobile device relative position.The correct image that uses the polar curve correction principle that the binocular mobile device is obtained, make two width of cloth images to polar curve just in the same horizontal line, any point just must have identical line number with its corresponding point on another width of cloth image on the piece image like this, only needs to carry out linear search at this row and can match corresponding point.
In this embodiment, the generation method of pixel describer is: the grey scale pixel value to the present frame of binocular image carries out respectively horizontal and vertical Sobel operator planar convolution, and laterally the nuclear of convolution is
Vertically the nuclear of convolution is
The window of employing 5 * 5 is as the extraction window of unique point describer, take laterally and the image after vertical Sobel convolution the Sobel convolution value of pixel is described the extraction of device as the object interlacing, extract 12 Sobel convolution values that pixel is corresponding in the window after horizontal Sobel convolution, extract 4 Sobel convolution values that pixel is corresponding in the window after vertical Sobel convolution.What Fig. 4 represented is the particular location that this step is extracted the convolution value, and a Sobel convolution value is extracted in numeral 1 representative in the grid, and twice Sobel convolution value extracted in numeral 2 representatives, extracts altogether 16 convolution values as the feature describer.
In order to improve computing velocity, in this embodiment, when calculating parallax, take the method for interlacing calculating, and carry out the pixel disparity computation one time every certain step-length.
In this embodiment, carry out the disparity computation of a pixel every five step-lengths, judge that at first whether 16 convolution values of describer sum of current pixel point is greater than the texture threshold value, thereby the texture that guarantees this pixel position is too not smooth, if less than the texture threshold value of setting, it is invalid then the parallax value of this pixel to be set to; Centered by each remarkable pixel, create the window of 5 * 5 sizes, extract the corresponding describer of pixel at four angles of this window as the matching parameter of central pixel point, with left figure as the reference image, suppose that corresponding pixel parallax is d among the pixel of parallax to be calculated among the left figure and the right figure, then can obtain pixel corresponding to right figure, to four jiaos of describers of the corresponding window of the left and right sides view of this pixel, it is matching parameter, ask poor and carry out the absolute value summation, because parallax d has a span, the absolute value sum of being on duty hour, the parallax value that this moment, corresponding parallax d was this pixel.
In this embodiment, the remarkable Stereo matching point of tentatively choosing is spread, adopt the De Laonei Triangulation Method that these sparse points are carried out triangulation, remarkable Stereo matching point is split into triangle gridding, can obtain three corresponding pixel coordinates in summit of triangle gridding, thereby can calculate the straight-line equation on three limits, then can determine the coordinate of each pixel of being surrounded by this triangle gridding, at last all pixels that are included in the triangle gridding be carried out disparity computation.
In this embodiment, judge that whether 16 convolution values of describer sum of current pixel point is greater than the texture threshold value, thereby the texture that guarantees this pixel position is too not smooth, if less than the texture threshold value of setting, it is invalid then the parallax value of this pixel to be set to; To calculate left disparity map as example, with left view as the reference image, the parallax of supposing certain pixel is d_curr, then can obtain pixel corresponding to right view, left and right sides view description device corresponding to this pixel asked poor and carried out the absolute value summation, be designated as SSE, because parallax d_curr has a span, when SSE+p hour, the parallax that adopts this moment is the parallax value of this pixel.Wherein
In this embodiment, this step 105 comprises left and right sides consistency check; Remove the zonule; Interpolation; Filtering.
As shown in Figure 5, the process of step 105 generating depth map further comprises again following step:
Owing to when carrying out spatial interaction, only need to obtain the motion process of human body (for example hand), so the depth map that obtains need to be carried out cutting apart of hand region.Adopt the method that connected domain increases and colour of skin judgement combines when cutting apart.As shown in Figure 6, the hand region segmentation procedure further comprises following step:
In spatial interaction, hand often is positioned at the place ahead of spatial scene, so as long as find the corresponding pixel of depth capacity in the depth map, can determine the approximate location of hand region.
In this embodiment, the pixel neighborhood of a point refers to this pixel zone of four pixels on every side.
As shown in Figure 7, the process of the neighborhood of step 202 judgement further comprises again following step:
This step namely judges whether zero growth point, in this embodiment when the depth value difference of all pixels that are positioned at the connected domain edge and neighborhood territory pixel point during all the time greater than defined threshold, then explanation this moment zero growth point.
After the method for using connected domain to increase roughly splits hand region, need to carry out the colour of skin and judge, thereby accurately extract hand region.As shown in Figure 8, the process of the colour of skin of step 204 judgement further comprises again following step:
The pixel chromatic information that step 2041, extraction hand split.Because being the colour of skin, this step judges, so the information of obtaining should be chromatic information.
Owing to the depth image of rebuilding and the coordinate disunity of virtual scene, two different coordinate systems need to be carried out normalizing, and the size of the virtual hand behind the normalizing is adjusted to corresponding ratio with the virtual scene size.As shown in Figure 9, the Unitary coordinate step further comprises following step:
Step 301 is set virtual scene.Virtual scene can be taken from reality scene, also can be three-dimensional material, also can be both combinations.
Step 302 is determined the ratio between the hand region in virtual scene and the depth image.In virtual scene, choose object of reference, can obtain between the two ratio according to coordinate and the image coordinate of hand region of object of reference in the virtual scene in display screen.
Step 303 is mapped to the image coordinate of hand region in the depth image in the virtual scene, i.e. the generating virtual hand.According to the ratio between the ratio of object of reference and hand region and object of reference and the virtual hand that will be projected, can determine the projection pantograph ratio of hand region, the image coordinate of hand region can be mapped in the virtual scene by this pantograph ratio.
Step 4, detect in the described virtual scene human body whether with this virtual scene in other objects bump, if bump, then forward step 6 to, if do not bump, then forward step 7 to.
Hand region in the realistic space is mapped in the Virtual Space forms after the virtual hand, need to detect the collision relation between virtual hand and the virtual scene, in order to make virtual scene make corresponding reaction.As shown in figure 10, the collision detection step may further comprise the steps:
Step 401 is enclosed virtual hand and virtual scene with the encirclement bag respectively.Encirclement ball in the virtual scene is defined as comprising the smallest sphere of certain object, must there be two points to belong to the virtual objects that it comprises on the sphere of this smallest sphere, only have when two smallest sphere of surrounding virtual hand and encirclement virtual objects bump, virtual hand and virtual objects just may bump.Describe the encirclement ball and only need two parameters, i.e. sphere centre coordinate and the radius of a ball.
Step 402, judge whether the encirclement ball of described virtual hand region and described virtual scene bumps.
The collision detection standard is: the centre of sphere distance of two balls and the relation of two radius of a ball sums, if the centre of sphere illustrates then that apart from less than or equal to the radius sum collision has occured two spheroids, if the centre of sphere illustrates then that apart from greater than the radius sum two spheroids do not bump.
Step 5, determine body language according to the variation in time of the human body in the described virtual scene.
Obtain the rule such as direction of motion, speed, acceleration of hand according to continuous several frames and determine the hand exercise state.The present invention has stipulated six kinds of gestures, specifically describes as shown in the table:
As shown in figure 11, motion process description and gesture identification step further comprise following step:
In this embodiment of the present invention, described gesture model is aforesaid six kinds of gesture models.
Step 6, described virtual scene are made response according to described body language.
Step 7 judges whether the image of the human body that described a plurality of video camera is taken upgrades, if upgrade, then turns back to step 1, if do not have, then this method finishes.
Above-described specific embodiment; purpose of the present invention, technical scheme and beneficial effect are further described; be understood that; the above only is specific embodiments of the invention; be not limited to the present invention; within the spirit and principles in the present invention all, any modification of making, be equal to replacement, improvement etc., all should be included within protection scope of the present invention.
Claims (14)
1. spatial interaction method that is applied to mobile device, described mobile device refers to have the portable information processing device of interactive capability and image camera function, it is characterized in that, comprises the steps:
Step 1, utilize mobile device to take two dimensional image, rebuild the depth map of this two dimensional image, comprise the image of human body in the described two dimensional image;
Step 2, described depth map is carried out Region Segmentation, obtain the human body image in this depth map;
Step 3, with in human body image mapped to a virtual scene in the described depth map;
Step 4, detect in the described virtual scene human body whether with this scene in other objects bump, if bump, then forward step 6 to, if do not bump, then forward step 7 to;
Step 5, determine people's body language according to the spatial variations in time of the human body in the described virtual scene;
Step 6, described virtual scene are made response according to described body language;
Step 7, judge whether the image of the human body that described mobile device is taken upgrades, if upgrade, then turns back to step 1, if do not upgrade, then the method finishes.
2. spatial interaction method as claimed in claim 1 is characterized in that, in step 1, described human body is hand, and described body language is gesture.
3. spatial interaction method as claimed in claim 2 is characterized in that, and is described in step 1, and described mobile device is the mobile device with two camera heads, two width of cloth two dimensional images about the hand that described two dimensional image is taken for this mobile device.
4. spatial interaction method as claimed in claim 3 is characterized in that, described step 1 comprises the steps:
Step 101, described two width of cloth two dimensional images are carried out polar curve proofread and correct;
Step 102, to described two width of cloth two dimensional image interlacing to each pixel extraction pixel describer, this pixel describer be used for to be described the situation of change of gray-scale value around the pixel;
Step 103, in described two width of cloth two dimensional images, determine remarkable Stereo matching point, and calculate its parallax that significantly the Stereo matching point refers to be positioned at assigned address and texture value greater than the pixel of defined threshold;
Step 104, spread from described remarkable Stereo matching point, obtain the parallax of all pixels of described two width of cloth two dimensional images;
Step 105, rebuild depth map according to the parallax of described each pixel.
5. spatial interaction method as claimed in claim 4 is characterized in that, described step 102 comprises the steps: the grey scale pixel value of the present frame of described two width of cloth two dimensional images is carried out respectively horizontal and vertical Sobel operator planar convolution, and laterally the nuclear of convolution is
Vertically the nuclear of convolution is
The window of employing 5 * 5 is as the extraction window of unique point describer, take laterally and the image after vertical Sobel convolution the Sobel convolution value of pixel is described the extraction of device as the object interlacing, extract 12 Sobel convolution values that pixel is corresponding in the window after horizontal Sobel convolution, extract 4 Sobel convolution values that pixel is corresponding in the window after vertical Sobel convolution.
6. spatial interaction method as claimed in claim 4 is characterized in that, described step 103 is taked the method for interlacing calculating, and carries out the pixel disparity computation one time every certain step-length.
7. spatial interaction method as claimed in claim 4, it is characterized in that, described step 104 comprises: adopt the De Laonei Triangulation Method that these sparse points are carried out triangulation, described remarkable Stereo matching point is split into triangle gridding, obtain three corresponding pixel coordinates in summit of triangle gridding, calculate the straight-line equation on three limits, determine by the coordinate of each pixel that this triangle gridding surrounded, at last all pixels that are included in the triangle gridding are carried out disparity computation.
8. spatial interaction method as claimed in claim 4 is characterized in that, described step 105 comprises the steps:
Step 1051, the left and right sides consistance of the pixel of described two width of cloth two dimensional images is checked;
Step 1052, remove the zonule with disparity consistency;
Step 1053, the interpolation of parallax value that the pixel that has invalid parallax value carried out;
Step 1054, the pixel of described two width of cloth two dimensional images is carried out filtering.
9. spatial interaction method as claimed in claim 2 is characterized in that, described step 2 comprises the steps:
Step 201, search pixel corresponding to depth capacity in the described depth map, it tentatively is defined as the hand position pixel;
Step 202, judge whether the described neighborhood territory pixel point that tentatively is defined as each pixel of hand region is positioned at the same area with this pixel;
Step 203, judge whether also to exist in the pixel map and else belong to the pixel of hand region;
Step 204, the described pixel that is defined as hand region is carried out the colour of skin judge, to remove the pixel that does not obviously belong to hand region.
10. spatial interaction method as claimed in claim 9 is characterized in that, described step 202 comprises the steps:
Step 2021, determine initial pixel: when beginning, this initial pixel is pixel corresponding to depth capacity, judges each time that later on the initial pixel that adopts all is the pixel in the last same connected domain of determining when judging;
Step 2022, read the depth value of each pixel in this initial pixel neighborhood of a point;
Step 2023, when the difference of the depth value of each pixel in the described initial pixel neighborhood of a point during less than a threshold value, then should initial pixel neighborhood of a point as a connected domain, judge that namely the pixel in this initial pixel neighborhood of a point is positioned at the same area with this initial pixel, and return step 2021.
11. spatial interaction method as claimed in claim 9 is characterized in that described step 204 comprises the steps:
The pixel chromatic information that step 2041, extraction hand split.Because being the colour of skin, this step judges, so the information of obtaining should be chromatic information;
Step 2042, judge that according to the chromatic information of pixel whether this pixel satisfies given colour of skin condition, if certain pixel is confirmed as colour of skin point, then forwards step 2044 to, otherwise forwards step 2043 to;
Step 2043, the gray-scale value of this pixel is set to (0,0,0);
Step 2044, keep the gray value information of this pixel;
Step 2045, judge whether to exist without the hand region pixel of judging, if exist the pixel that is positioned at hand region without judgement, then get back to step 2041, finish judgement if all are positioned at the pixel of hand region, then finish.
12. spatial interaction method as claimed in claim 2 is characterized in that described step 3 comprises the steps:
Step 301, virtual scene is calibrated;
The coordinate of step 302, acquisition depth map;
Step 303, try to achieve the transition matrix of virtual scene coordinate and depth map coordinate;
The size of step 304, unified virtual hand and virtual scene.
13. spatial interaction method as claimed in claim 2 is characterized in that described step 4 comprises the steps:
Step 401, enclose virtual hand and virtual scene with the encirclement bag respectively;
Step 402, judge whether the described encirclement ball of described virtual hand region and described virtual scene bumps.
14. spatial interaction method as claimed in claim 2 is characterized in that described step 5 comprises the steps:
Step 501, the track of the virtual hand of record on each frame;
Step 502, adopt the method for fitting a straight line to carry out match to described track;
Step 503, whether judge described fitting a straight line two ends length greater than a certain threshold value, if this length, determines then that described virtual hand is the motion in the two-dimensional space greater than this threshold value, forward step 504 to, otherwise forward step 505 to;
Step 504, with described fitting a straight line integer representation;
Step 505, obtain virtual hand region area situation of change, determine the direction that seesaws of virtual hand;
Step 506, according to the specific description of carrying out for the gesture of virtual hand of step 504 and step 505, mate with the Pre-defined gesture model, thereby determine this virtual gesture.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201210320166.2A CN102902355B (en) | 2012-08-31 | 2012-08-31 | The space interaction method of mobile device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201210320166.2A CN102902355B (en) | 2012-08-31 | 2012-08-31 | The space interaction method of mobile device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN102902355A true CN102902355A (en) | 2013-01-30 |
| CN102902355B CN102902355B (en) | 2015-12-02 |
Family
ID=47574643
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201210320166.2A Active CN102902355B (en) | 2012-08-31 | 2012-08-31 | The space interaction method of mobile device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN102902355B (en) |
Cited By (26)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103778632A (en) * | 2014-01-18 | 2014-05-07 | 南京理工大学 | Method for stereo matching based on FPGA |
| CN104407696A (en) * | 2014-11-06 | 2015-03-11 | 北京京东尚科信息技术有限公司 | Virtual ball simulation and control method of mobile device |
| CN104915010A (en) * | 2015-06-28 | 2015-09-16 | 合肥金诺数码科技股份有限公司 | Gesture recognition based virtual book flipping system |
| CN105912102A (en) * | 2016-03-31 | 2016-08-31 | 联想(北京)有限公司 | Information processing method and electronic equipment |
| CN105929947A (en) * | 2016-04-15 | 2016-09-07 | 济南大学 | Scene situation perception based man-machine interaction method |
| CN107430437A (en) * | 2015-02-13 | 2017-12-01 | 厉动公司 | The system and method that real crawl experience is created in virtual reality/augmented reality environment |
| CN107680164A (en) * | 2016-08-01 | 2018-02-09 | 中兴通讯股份有限公司 | A kind of virtual objects scale adjusting method and device |
| CN108014492A (en) * | 2016-11-01 | 2018-05-11 | 宏达国际电子股份有限公司 | virtual reality interactive processing method and electronic device |
| CN108255351A (en) * | 2017-12-22 | 2018-07-06 | 潍坊歌尔电子有限公司 | Determining method and device, projecting apparatus, the optical projection system of user's finger location information |
| WO2018177337A1 (en) * | 2017-03-29 | 2018-10-04 | 北京市商汤科技开发有限公司 | Method and apparatus for determining three-dimensional hand data, and electronic device |
| CN105528082B (en) * | 2016-01-08 | 2018-11-06 | 北京暴风魔镜科技有限公司 | Three dimensions and gesture identification tracking exchange method, device and system |
| CN109344718A (en) * | 2018-09-03 | 2019-02-15 | 先临三维科技股份有限公司 | Finger tip recognition methods, device, storage medium and processor |
| CN109407838A (en) * | 2018-10-17 | 2019-03-01 | 福建星网视易信息系统有限公司 | Interface interaction method and computer readable storage medium |
| CN109407825A (en) * | 2018-08-30 | 2019-03-01 | 百度在线网络技术（北京）有限公司 | Interactive approach and device based on virtual objects |
| CN109461199A (en) * | 2018-11-15 | 2019-03-12 | 腾讯科技（深圳）有限公司 | Picture rendering method and device, storage medium and electronic device |
| CN109710077A (en) * | 2018-12-30 | 2019-05-03 | 郑州畅想高科股份有限公司 | Dummy object collision judgment method, device and locomotive experience system based on VR |
| CN109739358A (en) * | 2019-01-03 | 2019-05-10 | 京东方科技集团股份有限公司 | Gesture collision checking method and equipment based on naked eye 3D |
| CN110838138A (en) * | 2019-09-26 | 2020-02-25 | 北京迈格威科技有限公司 | Repetitive texture detection method, device, computer equipment and storage medium |
| CN111340874A (en) * | 2020-02-14 | 2020-06-26 | 芜湖启迪睿视信息技术有限公司 | Goods shelf anti-collision detection method based on image semantic segmentation and 3D reconstruction |
| CN111713090A (en) * | 2018-02-15 | 2020-09-25 | 奇跃公司 | Mixed reality musical instrument |
| US11003917B2 (en) | 2013-10-17 | 2021-05-11 | Drägerwerk AG & Co. KGaA | Method for monitoring a patient within a medical monitoring area |
| CN113744803A (en) * | 2020-05-29 | 2021-12-03 | 鸿富锦精密电子(天津)有限公司 | Gene sequencing progress management method and device, computer device and storage medium |
| US11237625B2 (en) | 2015-02-13 | 2022-02-01 | Ultrahaptics IP Two Limited | Interaction engine for creating a realistic experience in virtual reality/augmented reality environments |
| US11337023B2 (en) | 2019-12-20 | 2022-05-17 | Magic Leap, Inc. | Physics-based audio and haptic synthesis |
| US11636843B2 (en) | 2020-05-29 | 2023-04-25 | Magic Leap, Inc. | Surface appropriate collisions |
| CN117075730A (en) * | 2023-08-18 | 2023-11-17 | 广东早安文化发展有限公司 | 3D virtual exhibition hall control system based on image recognition technology |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| EP0991023A2 (en) * | 1998-10-02 | 2000-04-05 | CSELT Centro Studi e Laboratori Telecomunicazioni S.p.A. | A method of creating 3-D facial models starting from face images |
| US20070127810A1 (en) * | 2003-08-08 | 2007-06-07 | Microsoft Corporation | System and method for modeling three dimensional objects from a single image |
| CN101273368A (en) * | 2005-08-29 | 2008-09-24 | 埃韦里克斯技术股份有限公司 | Interactivity via mobile image recognition |
| CN101393497A (en) * | 2008-10-30 | 2009-03-25 | 上海交通大学 | Multi-point touch method based on binocular stereo vision |
| CN101893935A (en) * | 2010-07-14 | 2010-11-24 | 北京航空航天大学 | Cooperative construction method for enhancing realistic table-tennis system based on real rackets |
| CN102253713A (en) * | 2011-06-23 | 2011-11-23 | 康佳集团股份有限公司 | Display system orienting to three-dimensional images |
-
2012
- 2012-08-31 CN CN201210320166.2A patent/CN102902355B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| EP0991023A2 (en) * | 1998-10-02 | 2000-04-05 | CSELT Centro Studi e Laboratori Telecomunicazioni S.p.A. | A method of creating 3-D facial models starting from face images |
| US20070127810A1 (en) * | 2003-08-08 | 2007-06-07 | Microsoft Corporation | System and method for modeling three dimensional objects from a single image |
| CN101273368A (en) * | 2005-08-29 | 2008-09-24 | 埃韦里克斯技术股份有限公司 | Interactivity via mobile image recognition |
| CN101393497A (en) * | 2008-10-30 | 2009-03-25 | 上海交通大学 | Multi-point touch method based on binocular stereo vision |
| CN101893935A (en) * | 2010-07-14 | 2010-11-24 | 北京航空航天大学 | Cooperative construction method for enhancing realistic table-tennis system based on real rackets |
| CN102253713A (en) * | 2011-06-23 | 2011-11-23 | 康佳集团股份有限公司 | Display system orienting to three-dimensional images |
Cited By (41)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US11003917B2 (en) | 2013-10-17 | 2021-05-11 | Drägerwerk AG & Co. KGaA | Method for monitoring a patient within a medical monitoring area |
| CN103778632A (en) * | 2014-01-18 | 2014-05-07 | 南京理工大学 | Method for stereo matching based on FPGA |
| CN104407696A (en) * | 2014-11-06 | 2015-03-11 | 北京京东尚科信息技术有限公司 | Virtual ball simulation and control method of mobile device |
| CN104407696B (en) * | 2014-11-06 | 2016-10-05 | 北京京东尚科信息技术有限公司 | The virtual ball simulation of mobile device and the method for control |
| CN107430437A (en) * | 2015-02-13 | 2017-12-01 | 厉动公司 | The system and method that real crawl experience is created in virtual reality/augmented reality environment |
| US11237625B2 (en) | 2015-02-13 | 2022-02-01 | Ultrahaptics IP Two Limited | Interaction engine for creating a realistic experience in virtual reality/augmented reality environments |
| US11392212B2 (en) | 2015-02-13 | 2022-07-19 | Ultrahaptics IP Two Limited | Systems and methods of creating a realistic displacement of a virtual object in virtual reality/augmented reality environments |
| CN104915010A (en) * | 2015-06-28 | 2015-09-16 | 合肥金诺数码科技股份有限公司 | Gesture recognition based virtual book flipping system |
| CN105528082B (en) * | 2016-01-08 | 2018-11-06 | 北京暴风魔镜科技有限公司 | Three dimensions and gesture identification tracking exchange method, device and system |
| CN105912102A (en) * | 2016-03-31 | 2016-08-31 | 联想(北京)有限公司 | Information processing method and electronic equipment |
| CN105929947A (en) * | 2016-04-15 | 2016-09-07 | 济南大学 | Scene situation perception based man-machine interaction method |
| CN107680164B (en) * | 2016-08-01 | 2023-01-10 | 中兴通讯股份有限公司 | Virtual object size adjusting method and device |
| CN107680164A (en) * | 2016-08-01 | 2018-02-09 | 中兴通讯股份有限公司 | A kind of virtual objects scale adjusting method and device |
| CN108014492A (en) * | 2016-11-01 | 2018-05-11 | 宏达国际电子股份有限公司 | virtual reality interactive processing method and electronic device |
| CN108014492B (en) * | 2016-11-01 | 2021-01-26 | 宏达国际电子股份有限公司 | Virtual reality interaction processing method and electronic device |
| WO2018177337A1 (en) * | 2017-03-29 | 2018-10-04 | 北京市商汤科技开发有限公司 | Method and apparatus for determining three-dimensional hand data, and electronic device |
| US11120254B2 (en) | 2017-03-29 | 2021-09-14 | Beijing Sensetime Technology Development Co., Ltd. | Methods and apparatuses for determining hand three-dimensional data |
| CN108255351A (en) * | 2017-12-22 | 2018-07-06 | 潍坊歌尔电子有限公司 | Determining method and device, projecting apparatus, the optical projection system of user's finger location information |
| CN111713090A (en) * | 2018-02-15 | 2020-09-25 | 奇跃公司 | Mixed reality musical instrument |
| US11978171B2 (en) | 2018-02-15 | 2024-05-07 | Magic Leap, Inc. | Mixed reality musical instrument |
| CN111713090B (en) * | 2018-02-15 | 2023-02-17 | 奇跃公司 | Mixed reality musical instrument |
| US11657585B2 (en) | 2018-02-15 | 2023-05-23 | Magic Leap, Inc. | Mixed reality musical instrument |
| US11403825B2 (en) | 2018-02-15 | 2022-08-02 | Magic Leap, Inc. | Mixed reality musical instrument |
| CN109407825A (en) * | 2018-08-30 | 2019-03-01 | 百度在线网络技术（北京）有限公司 | Interactive approach and device based on virtual objects |
| CN109344718A (en) * | 2018-09-03 | 2019-02-15 | 先临三维科技股份有限公司 | Finger tip recognition methods, device, storage medium and processor |
| CN109407838A (en) * | 2018-10-17 | 2019-03-01 | 福建星网视易信息系统有限公司 | Interface interaction method and computer readable storage medium |
| CN109461199A (en) * | 2018-11-15 | 2019-03-12 | 腾讯科技（深圳）有限公司 | Picture rendering method and device, storage medium and electronic device |
| CN109710077B (en) * | 2018-12-30 | 2022-05-06 | 郑州畅想高科股份有限公司 | Virtual object collision judgment method and device based on VR and locomotive practical training system |
| CN109710077A (en) * | 2018-12-30 | 2019-05-03 | 郑州畅想高科股份有限公司 | Dummy object collision judgment method, device and locomotive experience system based on VR |
| CN109739358B (en) * | 2019-01-03 | 2022-05-24 | 京东方科技集团股份有限公司 | Gesture collision detection method and device based on naked eye 3D |
| CN109739358A (en) * | 2019-01-03 | 2019-05-10 | 京东方科技集团股份有限公司 | Gesture collision checking method and equipment based on naked eye 3D |
| CN110838138A (en) * | 2019-09-26 | 2020-02-25 | 北京迈格威科技有限公司 | Repetitive texture detection method, device, computer equipment and storage medium |
| US11337023B2 (en) | 2019-12-20 | 2022-05-17 | Magic Leap, Inc. | Physics-based audio and haptic synthesis |
| US11632646B2 (en) | 2019-12-20 | 2023-04-18 | Magic Leap, Inc. | Physics-based audio and haptic synthesis |
| CN111340874A (en) * | 2020-02-14 | 2020-06-26 | 芜湖启迪睿视信息技术有限公司 | Goods shelf anti-collision detection method based on image semantic segmentation and 3D reconstruction |
| CN111340874B (en) * | 2020-02-14 | 2023-05-19 | 芜湖启迪睿视信息技术有限公司 | Goods shelf anti-collision detection method for image semantic segmentation and 3D reconstruction |
| CN113744803A (en) * | 2020-05-29 | 2021-12-03 | 鸿富锦精密电子(天津)有限公司 | Gene sequencing progress management method and device, computer device and storage medium |
| US11900912B2 (en) | 2020-05-29 | 2024-02-13 | Magic Leap, Inc. | Surface appropriate collisions |
| US11636843B2 (en) | 2020-05-29 | 2023-04-25 | Magic Leap, Inc. | Surface appropriate collisions |
| CN117075730A (en) * | 2023-08-18 | 2023-11-17 | 广东早安文化发展有限公司 | 3D virtual exhibition hall control system based on image recognition technology |
| CN117075730B (en) * | 2023-08-18 | 2024-04-30 | 广东早安文化发展有限公司 | 3D virtual exhibition hall control system based on image recognition technology |
Also Published As
| Publication number | Publication date |
|---|---|
| CN102902355B (en) | 2015-12-02 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN102902355B (en) | The space interaction method of mobile device | |
| KR102647351B1 (en) | Modeling method and modeling apparatus using 3d point cloud | |
| CN111126272B (en) | Posture acquisition method, and training method and device of key point coordinate positioning model | |
| CN108898676B (en) | Method and system for detecting collision and shielding between virtual and real objects | |
| US9594950B2 (en) | Depth mapping with enhanced resolution | |
| CN102697508B (en) | Method for performing gait recognition by adopting three-dimensional reconstruction of monocular vision | |
| CN105719352B (en) | Face three-dimensional point cloud super-resolution fusion method and apply its data processing equipment | |
| CN108256504A (en) | A kind of Three-Dimensional Dynamic gesture identification method based on deep learning | |
| CN105654492A (en) | Robust real-time three-dimensional (3D) reconstruction method based on consumer camera | |
| CN104933718A (en) | Physical coordinate positioning method based on binocular vision | |
| CN112257605B (en) | Three-dimensional target detection method, system and device based on self-labeling training sample | |
| CN104317391A (en) | Stereoscopic vision-based three-dimensional palm posture recognition interactive method and system | |
| CN103839277A (en) | Mobile augmented reality registration method of outdoor wide-range natural scene | |
| US20150138193A1 (en) | Method and device for panorama-based inter-viewpoint walkthrough, and machine readable medium | |
| CN103247045A (en) | Method of obtaining artificial scene main directions and image edges from multiple views | |
| CN102799646B (en) | A kind of semantic object segmentation method towards multi-view point video | |
| CN104794737A (en) | Depth-information-aided particle filter tracking method | |
| CN101794459A (en) | Seamless integration method of stereoscopic vision image and three-dimensional virtual object | |
| CN111126116A (en) | Unmanned ship river channel garbage identification method and system | |
| KR102001636B1 (en) | Apparatus and method of processing a depth image using a relative angle between an image sensor and a target object | |
| CN112070782A (en) | Method and device for identifying scene contour, computer readable medium and electronic equipment | |
| CN106022266A (en) | Target tracking method and target tracking apparatus | |
| CN103077536B (en) | Space-time mutative scale moving target detecting method | |
| CN105138979A (en) | Method for detecting the head of moving human body based on stereo visual sense | |
| CN116630423A (en) | ORB (object oriented analysis) feature-based multi-target binocular positioning method and system for micro robot |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C14 | Grant of patent or utility model | ||
| GR01 | Patent grant |