CN111160088A - VR (virtual reality) somatosensory data detection method and device, computer equipment and storage medium - Google Patents
VR (virtual reality) somatosensory data detection method and device, computer equipment and storage medium Download PDFInfo
- Publication number
- CN111160088A CN111160088A CN201911154959.XA CN201911154959A CN111160088A CN 111160088 A CN111160088 A CN 111160088A CN 201911154959 A CN201911154959 A CN 201911154959A CN 111160088 A CN111160088 A CN 111160088A
- Authority
- CN
- China
- Prior art keywords
- action
- data
- standard
- current
- human body
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/20—Movements or behaviour, e.g. gesture recognition
Abstract
The invention discloses a VR somatosensory data detection method, a VR somatosensory data detection device, computer equipment and a storage medium. The method comprises the steps of performing action decomposition on standard action characteristic data to obtain standard 3D node data; performing data conversion on the standard 3D node data according to a human body action mapping table to obtain a corresponding standard action data set; receiving current action characteristic data uploaded by a target terminal, and obtaining a corresponding current action data set through action decomposition and data conversion according to a human body action mapping table in sequence; acquiring the current similarity between each human body action sequence in the current action data set and the corresponding human body action sequence in the standard action data set; and if the similarity is lower than the similarity threshold, sending the reminding information to the corresponding target terminal. The method realizes real-time scanning of human body actions by VR, matching and recognition with standard actions, accurately recognizes the similarity between the current action and the standard actions, and timely prompts wrong actions.
Description
Technical Field
The invention relates to the technical field of image recognition, in particular to a VR (virtual reality) somatosensory data detection method and device, computer equipment and a storage medium.
Background
At present, when a VR device (i.e. a virtual reality device) collects a human body action, the VR device is generally based on a pattern recognition technology.
When a person sees something or a phenomenon, the person first collects all the information about the object or phenomenon, then compares the behavior characteristics with the relevant information in mind, and if a match of the same or similar is found, the person can identify the object or phenomenon. Therefore, the information related to an object or a phenomenon, such as spatial information, temporal information, etc., constitutes a pattern of the object or the phenomenon.
Pattern recognition research has focused on two aspects, namely, on studying how an organism (including a human) perceives an object, which belongs to the field of cognitive science, and on how to implement the theory and method of pattern recognition by a computer under a given task. The automatic mode recognition simply means that a machine can automatically classify a specific sample into a certain mode without human interference, and the automatic mode recognition technology is an important component of the artificial intelligence technology. Automatic pattern recognition is mainly realized by applying a relevant method in machine learning. Common pattern recognition methods include statistical pattern recognition, syntactic structure pattern recognition, and artificial neural network pattern recognition.
However, the traditional pattern recognition technology has the disadvantages of high difficulty in human body action recognition and calculation, low pattern recognition effectiveness, especially low recognition rate of continuous actions, and incapability of timely correcting wrong actions.
Disclosure of Invention
The embodiment of the invention provides a VR (virtual reality) somatosensory data detection method, a VR somatosensory data detection device, computer equipment and a storage medium, and aims to solve the problems that in the prior art, the recognition rate of continuous human body action recognition through a pattern recognition technology is low, and wrong actions cannot be corrected in time.
In a first aspect, an embodiment of the present invention provides a VR somatosensory data detection method, including:
receiving standard action characteristic data acquired and uploaded by each key sensor in an acquisition terminal;
performing action decomposition on the standard action characteristic data to obtain standard 3D node data;
performing data conversion on the standard 3D node data according to a preset human body action mapping table to obtain a corresponding standard action data set; the human body action mapping table stores mapping relations between various standard 3D node data and standard action data;
receiving current action characteristic data collected and uploaded by a target terminal, and obtaining a corresponding current action data set through action decomposition and data conversion according to the human body action mapping table in sequence;
acquiring the current similarity between the human body action sequence in the current action data set and the corresponding human body action sequence in the standard action data set; the human body action sequence corresponding to the current action data set is composed of a plurality of human body action values arranged in time sequence in the current action data set, and the human body action sequence corresponding to the standard action data set is composed of a plurality of human body action values arranged in time sequence in the standard action data set; and
and if the similarity is lower than a preset similarity threshold, sending the reminding information of the current similarity to a corresponding target terminal.
In a second aspect, an embodiment of the present invention provides a VR somatosensory data detection apparatus, including:
the initial standard data acquisition unit is used for receiving standard action characteristic data acquired and uploaded by each key sensor in the acquisition terminal;
the standard 3D node data acquisition unit is used for performing action decomposition on the standard action characteristic data to obtain standard 3D node data;
the standard action data set acquisition unit is used for performing data conversion on the standard 3D node data according to a preset human body action mapping table to obtain a corresponding standard action data set; the human body action mapping table stores mapping relations between various standard 3D node data and standard action data;
the current action data set acquisition unit is used for receiving current action characteristic data acquired and uploaded by the target terminal, and obtaining a corresponding current action data set through action decomposition and data conversion according to the human body action mapping table in sequence;
the current similarity calculation unit is used for acquiring the current similarity between the human motion sequence in the current motion data set and the corresponding human motion sequence in the standard motion data set; the human body action sequence corresponding to the current action data set is composed of a plurality of human body action values arranged in time sequence in the current action data set, and the human body action sequence corresponding to the standard action data set is composed of a plurality of human body action values arranged in time sequence in the standard action data set; and
and the notification unit is used for sending the reminding information of the current similarity to the corresponding target terminal if the similarity is lower than a preset similarity threshold.
In a third aspect, an embodiment of the present invention further provides a computer device, which includes a memory, a processor, and a computer program stored on the memory and executable on the processor, where the processor, when executing the computer program, implements the VR somatosensory data detection method according to the first aspect.
In a fourth aspect, an embodiment of the present invention further provides a computer-readable storage medium, where the computer-readable storage medium stores a computer program, and the computer program, when executed by a processor, causes the processor to execute the VR somatosensory data detection method according to the first aspect.
The embodiment of the invention provides a VR (virtual reality) somatosensory data detection method, a VR somatosensory data detection device, computer equipment and a storage medium.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings needed to be used in the description of the embodiments are briefly introduced below, and it is obvious that the drawings in the following description are some embodiments of the present invention, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without creative efforts.
Fig. 1 is a schematic view of an application scenario of a VR somatosensory data detection method provided by an embodiment of the invention;
fig. 2 is a schematic flow chart of a VR somatosensory data detection method provided by an embodiment of the invention;
fig. 3 is a schematic sub-flow diagram of a VR somatosensory data detection method according to an embodiment of the present invention;
fig. 4 is another schematic sub-flow diagram of a VR somatosensory data detection method according to an embodiment of the present invention;
fig. 5 is another schematic sub-flow diagram of a VR somatosensory data detection method according to an embodiment of the present invention;
fig. 6 is another schematic sub-flow diagram of a VR somatosensory data detection method according to an embodiment of the present invention;
fig. 7 is a schematic block diagram of a VR somatosensory data detection apparatus according to an embodiment of the present invention;
fig. 8 is a schematic block diagram of a sub-unit of a VR somatosensory data detection apparatus according to an embodiment of the present invention;
fig. 9 is a schematic block diagram of another subunit of the VR somatosensory data detection apparatus according to the embodiment of the present invention;
fig. 10 is a schematic block diagram of another subunit of the VR somatosensory data detection apparatus according to the embodiment of the present invention;
fig. 11 is a schematic block diagram of another subunit of the VR somatosensory data detection apparatus according to the embodiment of the present invention;
FIG. 12 is a schematic block diagram of a computer device provided by an embodiment of the present invention.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are some, not all, embodiments of the present invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
It will be understood that the terms "comprises" and/or "comprising," when used in this specification and the appended claims, specify the presence of stated features, integers, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and/or groups thereof.
It is also to be understood that the terminology used in the description of the invention herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. As used in the specification of the present invention and the appended claims, the singular forms "a," "an," and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise.
It should be further understood that the term "and/or" as used in this specification and the appended claims refers to and includes any and all possible combinations of one or more of the associated listed items.
Referring to fig. 1 and fig. 2, fig. 1 is a schematic view of an application scenario of a VR somatosensory data detection method according to an embodiment of the present invention; fig. 2 is a schematic flow diagram of a VR somatosensory data detection method according to an embodiment of the present invention, where the VR somatosensory data detection method is applied to a server and is executed by application software installed in the server.
As shown in fig. 2, the method includes steps S110 to S160.
And S110, receiving the standard action characteristic data which are collected and uploaded by each key sensor in the collection terminal.
In this embodiment, the VR device is used as a collection terminal to collect continuous actions of a human body. Since VR devices, i.e., virtual reality hardware devices, typically include interactive devices including position trackers, data gloves, three-dimensional mice, motion capture devices, eye trackers, force feedback devices, and other interactive devices. For example, when the VR device is a motion capture device, it includes a plurality of key sensors (the key sensors are generally acceleration sensors or gesture sensors), and when the VR device is worn by a user, the plurality of key sensors in the VR device are distributed at a plurality of key positions on the user, such as the head, the palm of the left hand, the elbow joint of the left hand, the palm of the right hand, the elbow joint of the right hand, the knee joint of the left hand, the knee joint of the right hand, and the like. And scanning the human body action in real time through VR equipment to obtain a set of standard action. And then the VR equipment acquires the action characteristics of the set of standard actions so as to obtain standard action characteristic data, and then the standard action characteristic data is uploaded to a server by the VR equipment.
After the VR equipment has been worn to the user promptly, generally all set up the sensor to the human joint node position department that needs gather, these nodes are key sensor node. When a user makes an action, action characteristic data corresponding to the action can be collected. Because the standard action is recorded firstly, the standard action characteristic data corresponding to the standard action is collected through the VR equipment firstly.
And S120, performing action decomposition on the standard action characteristic data to obtain standard 3D node data.
In this embodiment, when performing motion decomposition on the standard motion feature data, the conversion is performed based on the point cloud data and the matching matrix. When the standard motion characteristic data is used for motion decomposition, the original multi-frame color image corresponding to the standard motion characteristic data is based on.
In an embodiment, step S120 further includes:
collecting a color image corresponding to the standard action characteristic data;
and carrying out graying processing on the color image to obtain a grayscale image.
In this embodiment, the Kinect fusion Explorer-D2D, a developer tool for Kinect cameras available from Microsoft, can be used to obtain the standard motion feature data in stl format. A color image corresponding to the standard feature data can also be acquired using another developer tool of the Kinect camera, Kinect Explorer-D2D. In order to reduce the image size while maintaining the image characteristics of the image to the maximum, the color image may be subjected to the graying processing by the maximum. The maximum value of R, G, B values of each pixel point in the color image is taken as the gray value of the pixel point, so that the color image is grayed to obtain a gray image.
In one embodiment, as shown in fig. 3, step S120 includes:
s121, converting the collected standard action characteristic data into point cloud data;
s122, acquiring screen coordinates corresponding to mark points of each key sensor node on the gray level image;
s123, acquiring point cloud feature points in the point cloud data to form a point cloud feature point set;
s124, acquiring point cloud feature points of 5 finger tips in the point cloud feature point set and 3D node data corresponding to the point cloud feature points of the 5 finger tips;
s125, correspondingly acquiring a matching matrix according to a screen coordinate matrix corresponding to the point cloud characteristic points of the 5 finger fingertips and an inverse matrix of a three-dimensional coordinate matrix corresponding to the 3D node data corresponding to the 5 finger fingertips;
s126, obtaining the remaining mark points of 5 finger tips of all key sensor nodes in the mark points on the gray level image to obtain a remaining mark point set on the gray level image;
and S127, multiplying the screen coordinates corresponding to each mark point in the residual mark point set by the matching matrix to obtain standard 3D node data corresponding to the standard action characteristic data.
In this embodiment, in order to more clearly understand the process of obtaining 3D node data from standard motion characteristic data through motion decomposition, a hand gesture is taken as an example and described below.
After the Kinect Fusion Explorer-D2D provided by Microsoft obtains the standard motion feature data in stl format, the standard motion feature data can be converted into point cloud data by Geomagic software (Jie magic software).
And then, acquiring screen coordinates corresponding to the mark points of the key sensor nodes on the gray level image in the gray level image, and realizing one-to-one mapping of the key sensor nodes on the gray level image.
After the point cloud data is obtained, the point cloud data can be subjected to surface fitting and normal vector calculation by adopting Geomagic software, so that a normal vector included angle between a point and a point in a surface is obtained to extract feature points. Specifically, when the included angle between the normal vector of the point and the neighborhood point in the curved surface is greater than or equal to a preset included angle threshold value, the point is a feature point; on the contrary, if the included angle between the point and the normal vector of the neighborhood point is smaller than the included angle threshold value, the point is not the feature point until all the feature points in the point cloud data are extracted to obtain the point cloud feature points.
Then, point cloud characteristic points of 5 finger fingertips in the point cloud characteristic point set and 3D node data corresponding to the point cloud characteristic points of the 5 finger fingertips are obtained, screen coordinates corresponding to the 5 finger fingertips in the color image are obtained, and a matching matrix is obtained according to a three-dimensional coordinate matrix formed by the 3D node data corresponding to the point cloud characteristic points of the 5 finger tips and a screen coordinate matrix formed by the screen coordinates corresponding to the 5 finger fingertips.
And finally, multiplying the residual mark points in the color image by the matching matrix to obtain the corresponding standard 3D node data. By acquiring the matching matrix, the mark points of each key sensor node in the color image can be effectively converted into standard 3D node data.
In one embodiment, as shown in fig. 4, step S122 includes:
s1221, obtaining an initial gray threshold value according to the maximum gray value and the minimum gray value of the gray image;
s1222, dividing the gray image into a target area and a background area according to the initial gray threshold to form a divided image, and obtaining a first average gray value corresponding to the target area and a second average gray value corresponding to the background area;
and S1223, acquiring screen coordinates corresponding to the mark points of each key sensor node on the segmented image.
In this embodiment, an initial gray threshold is obtained according to the maximum gray value and the minimum gray value of the gray image; namely T0＝(fmax+fmin) /2 wherein fmaxIs the maximum gray value of the gray image, fminIs the minimum gray value of the gray image.
Then, the gray image is divided into a target area and a background area according to the initial gray threshold value to form a segmentation image, and a first average gray value corresponding to the target area and a second average gray value corresponding to the background area are obtained.
And finally, acquiring screen coordinates corresponding to the mark points of each key sensor node on the segmentation image in the segmentation image.
Because the mark points of the key sensor nodes on the gray level image can still be remained on the gray level image after graying, the screen coordinates corresponding to the mark points are obtained by referring to the following formula (1) and formula (2):
where f (i, j) is the gray scale value of the point (i, j) on the gray scale image, N (i, j) is the weight of the point (i, j) on the gray scale image, generally N (i, j) is the number of f (i, j), W is the total number of pixel points in the width direction on the gray scale image, and H is the total number of pixel points in the height direction on the gray scale image.
In one embodiment, as shown in fig. 5, step S125 includes:
s1251, acquiring 3D node data corresponding to the point cloud feature points of the 5 finger fingertips according to the point cloud feature points of the 5 finger fingertips to form a three-dimensional coordinate matrix;
s1252, acquiring screen coordinates corresponding to 5 finger tips in the color image to form a screen coordinate matrix;
and S1253, multiplying the inverse matrix of the three-dimensional coordinate matrix by the screen coordinate matrix to obtain a corresponding matching matrix.
In this embodiment, if a screen coordinate matrix composed of screen coordinates corresponding to 5 finger tips in the color image is a, and a three-dimensional coordinate matrix composed of 3D node data corresponding to point cloud feature points of 5 finger tips is B, then B is used-1And A is H, wherein H is a matching matrix. A matching matrix calculated by taking a three-dimensional coordinate matrix formed by a screen coordinate matrix corresponding to 5 finger tips in a color image and 3D node data corresponding to point cloud characteristic points of the 5 finger tips as a reference can be used as a conversion matrix with higher precision, and mark points of each key sensor node in the color image are effectively converted into standard 3D node data.
S130, performing data conversion on the standard 3D node data according to a preset human body action mapping table to obtain a corresponding standard action data set; the human body action mapping table stores mapping relations between various standard 3D node data and standard action data.
In this embodiment, the 3D node data is converted into corresponding descriptive node data, and the subtle changes of the actions can be identified through the descriptive node data, so as to finally obtain a set of standard action data sets.
Specifically, the 3D node data may be understood as three-dimensional space coordinate data corresponding to a human body key node, each 3D node data corresponding to the color image of each frame may form a 3D node data set corresponding to the color image of the frame, a difference value between the 3D node data sets between each two adjacent frames is calculated (this difference value may be recorded as descriptive node data), a human body action value corresponding to the difference value is queried in a preset human body action mapping table according to the difference value, and a corresponding standard action data set may be obtained by combining a plurality of human body action values. The human body action mapping table stores mapping relations between various standard 3D node data and standard action data.
And S140, receiving the current action characteristic data collected and uploaded by the target terminal, and sequentially performing action decomposition and data conversion according to the human body action mapping table to obtain a corresponding current action data set.
In this embodiment, after the standard motion data set is obtained, the VR device scans the human body motion in real time to obtain the current motion. And acquiring the motion characteristic of the current motion to obtain the current motion characteristic data. And then, performing action decomposition on the current action characteristic data to obtain current 3D node data. And converting the current 3D node data into corresponding current descriptive node data, and identifying slight changes of actions through the current descriptive node data to finally obtain a set of current action data set. The specific process refers to step S110-step S130. That is, in step S140, the step S120 is referred to for the specific process of performing motion decomposition on the current motion characteristic data, and in step S140, the step S130 is referred to for the specific process of converting the current motion characteristic data according to the human motion mapping table data.
S150, acquiring the current similarity between the human motion sequence in the current motion data set and the corresponding human motion sequence in the standard motion data set; the plurality of human body action values arranged in time sequence in the current action data set form a human body action sequence corresponding to the current action data set, and the plurality of human body action values arranged in time sequence in the standard action data set form a human body action sequence corresponding to the standard action data set.
In the embodiment, human body actions are collected in real time through VR equipment, and are matched and identified with standard actions, and the similarity between the current actions and the standard actions is identified, so that wrong actions are prompted in time.
Specifically, since the current motion data set includes a plurality of human motion values (e.g., [1323579]), this current motion data set can be regarded as a row vector. Similarly, the standard motion data set also comprises a plurality of human body motion value sets as row vectors, and the Euclidean distance between the two row vectors is calculated, so that the current similarity between the current motion data set and the corresponding data set in the standard motion data set can be obtained.
In one embodiment, as shown in fig. 6, step S150 includes:
s151, acquiring a first one-dimensional row vector corresponding to each human motion sequence in the current motion data set;
s152, acquiring a second one-dimensional row vector corresponding to each human motion sequence in the standard motion data set;
s153, acquiring the Euclidean distance between the first one-dimensional row vector and the second one-dimensional row vector, and taking the Euclidean distance as the current similarity.
In this embodiment, after the first one-dimensional row vector corresponding to the current motion data set and the second one-dimensional row vector corresponding to the standard motion data set are obtained, the euclidean distance between the two row vectors may be calculated, and the euclidean distance is used as the current similarity. Through the calculation mode, the similarity calculation can be carried out on the current action data set and the standard action data set which are acquired in the same time length, so that the similarity calculation can be used as a parameter basis for timely prompting whether to carry out non-standard action or not.
And S160, if the similarity is lower than a preset similarity threshold, sending the reminding information of the current similarity to a corresponding target terminal.
In this embodiment, if the similarity is lower than a preset similarity threshold, the reminding information of the current similarity is sent to the corresponding target terminal, which indicates that the similarity between the collected current action data set and the standard action data set is low after the same duration, that is, the current action corresponding to the current action feature data is not standard, and the user needs to be prompted to correct the action in time. The reminding information comprises a similarity value and text information with the prompting similarity lower than a similarity threshold value. For example, the reminding information is: the similarity of your current action is 90%, which is lower than the standard similarity of 95%, please pay attention to the corrective action.
The method realizes real-time scanning of human body actions by VR, matching and recognition with standard actions, accurately recognizes the similarity between the current action and the standard actions, and timely prompts wrong actions.
The embodiment of the invention also provides a VR body feeling data detection device which is used for executing any embodiment of the VR body feeling data detection method. Specifically, please refer to fig. 7, fig. 7 is a schematic block diagram of a VR somatosensory data detection apparatus according to an embodiment of the present invention. The VR body sensing data detecting apparatus 100 may be disposed in a server.
As shown in fig. 7, the VR somatosensory data detection apparatus 100 includes an initial standard data acquisition unit 110, a standard 3D node data acquisition unit 120, a standard action data set acquisition unit 130, a current action data set acquisition unit 140, a current similarity calculation unit 150, and a notification unit 160.
And the initial standard data acquisition unit 110 is used for receiving standard action characteristic data acquired and uploaded by each key sensor in the acquisition terminal.
In this embodiment, the VR device is used as a collection terminal to collect continuous actions of a human body. Since VR devices, i.e., virtual reality hardware devices, typically include interactive devices including position trackers, data gloves, three-dimensional mice, motion capture devices, eye trackers, force feedback devices, and other interactive devices. For example, when the VR device is a motion capture device, it includes a plurality of key sensors (the key sensors are generally acceleration sensors or gesture sensors), and when the VR device is worn by a user, the plurality of key sensors in the VR device are distributed at a plurality of key positions on the user, such as the head, the palm of the left hand, the elbow joint of the left hand, the palm of the right hand, the elbow joint of the right hand, the knee joint of the left hand, the knee joint of the right hand, and the like. And scanning the human body action in real time through VR equipment to obtain a set of standard action. And then the VR equipment acquires the action characteristics of the set of standard actions so as to obtain standard action characteristic data, and then the standard action characteristic data is uploaded to a server by the VR equipment.
After the VR equipment has been worn to the user promptly, generally all set up the sensor to the human joint node position department that needs gather, these nodes are key sensor node. When a user makes an action, action characteristic data corresponding to the action can be collected. Because the standard action is recorded firstly, the standard action characteristic data corresponding to the standard action is collected through the VR equipment firstly.
A standard 3D node data obtaining unit 120, configured to obtain standard 3D node data by performing action decomposition on the standard action feature data.
In this embodiment, when performing motion decomposition on the standard motion feature data, the conversion is performed based on the point cloud data and the matching matrix. When the standard motion characteristic data is used for motion decomposition, the original multi-frame color image corresponding to the standard motion characteristic data is based on.
In an embodiment, the VR somatosensory data detection apparatus 100 further includes:
the color image acquisition unit is used for acquiring a color image corresponding to the standard action characteristic data;
and the graying processing unit is used for performing graying processing on the color image to obtain a grayscale image.
In this embodiment, the developer tool Kinect fusion Explorer-D2D of Kinect camera provided by Microsoft can be used to obtain the standard motion feature data in stl format. A color image corresponding to the standard feature data can also be acquired using another developer tool of the Kinect camera, Kinect Explorer-D2D. In order to reduce the image size while maintaining the image characteristics of the image to the maximum, the color image may be subjected to the graying processing by the maximum. The maximum value of R, G, B values of each pixel point in the color image is taken as the gray value of the pixel point, so that the color image is grayed to obtain a gray image.
In one embodiment, as shown in fig. 8, the standard 3D node data obtaining unit 120 includes:
a point cloud data obtaining unit 121, configured to convert the collected standard motion feature data into point cloud data;
a first screen coordinate obtaining unit 122, configured to obtain a screen coordinate corresponding to a mark point of each key sensor node on the grayscale image;
a point cloud feature point set obtaining unit 123, configured to obtain point cloud feature points in the point cloud data to form a point cloud feature point set;
a finger tip feature obtaining unit 124, configured to obtain point cloud feature points of 5 finger tips in the point cloud feature point set and 3D node data corresponding to the point cloud feature points of the 5 finger tips;
a matching matrix obtaining unit 125, configured to obtain a matching matrix according to a screen coordinate matrix corresponding to the point cloud feature point of the 5 finger fingertips and according to an inverse matrix of a three-dimensional coordinate matrix corresponding to the 3D node data corresponding to the 5 finger fingertips;
the mark point processing unit 126 is configured to obtain remaining mark points of 5 finger tips of the mark points of each key sensor node on the grayscale image, so as to obtain a remaining mark point set on the grayscale image;
and the node data conversion unit 127 is configured to multiply the screen coordinates corresponding to each mark point in the remaining mark point set by the matching matrix to obtain standard 3D node data corresponding to the standard action characteristic data.
In this embodiment, in order to more clearly understand the process of obtaining 3D node data from standard motion characteristic data through motion decomposition, a hand gesture is taken as an example and described below.
After the Kinect Fusion Explorer-D2D provided by Microsoft obtains the standard motion feature data in stl format, the standard motion feature data can be converted into point cloud data by Geomagic software (Jie magic software).
And then, acquiring screen coordinates corresponding to the mark points of the key sensor nodes on the gray level image in the gray level image, and realizing one-to-one mapping of the key sensor nodes on the gray level image.
After the point cloud data is obtained, the point cloud data can be subjected to surface fitting and normal vector calculation by adopting Geomagic software, so that a normal vector included angle between a point and a point in a surface is obtained to extract feature points. Specifically, when the included angle between the normal vector of the point and the neighborhood point in the curved surface is greater than or equal to a preset included angle threshold value, the point is a feature point; on the contrary, if the included angle between the point and the normal vector of the neighborhood point is smaller than the included angle threshold value, the point is not the feature point until all the feature points in the point cloud data are extracted to obtain the point cloud feature points.
Then, point cloud characteristic points of 5 finger fingertips in the point cloud characteristic point set and 3D node data corresponding to the point cloud characteristic points of the 5 finger fingertips are obtained, screen coordinates corresponding to the 5 finger fingertips in the color image are obtained, and a matching matrix is obtained according to a three-dimensional coordinate matrix formed by the 3D node data corresponding to the point cloud characteristic points of the 5 finger tips and a screen coordinate matrix formed by the screen coordinates corresponding to the 5 finger fingertips.
And finally, multiplying the residual mark points in the color image by the matching matrix to obtain the corresponding standard 3D node data. By acquiring the matching matrix, the mark points of each key sensor node in the color image can be effectively converted into standard 3D node data.
In one embodiment, as shown in fig. 9, the first screen coordinate acquiring unit 122 includes:
an initial gray threshold obtaining unit 1221, configured to obtain an initial gray threshold according to the maximum gray value and the minimum gray value of the gray image;
a background separating unit 1222, configured to divide the grayscale image into a target region and a background region according to the initial grayscale threshold to form a divided image, and obtain a first average grayscale value corresponding to the target region and a second average grayscale value corresponding to the background region;
and the second screen coordinate acquiring unit 1223 is configured to acquire screen coordinates corresponding to the mark points of each key sensor node on the segmented image.
In this embodiment, an initial gray threshold is obtained according to the maximum gray value and the minimum gray value of the gray image; namely T0＝(fmax+fmin) /2 wherein fmaxIs the maximum gray value of the gray image, fminIs the minimum gray value of the gray image.
Then, the gray image is divided into a target area and a background area according to the initial gray threshold value to form a segmentation image, and a first average gray value corresponding to the target area and a second average gray value corresponding to the background area are obtained.
And finally, acquiring screen coordinates corresponding to the mark points of each key sensor node on the segmentation image in the segmentation image.
Because the mark points of the key sensor nodes on the gray-scale image can still be remained on the gray-scale image after graying, the screen coordinates corresponding to the mark points are obtained by referring to the formula (1) and the formula (2).
In one embodiment, as shown in fig. 10, the matching matrix obtaining unit 125 includes:
a three-dimensional coordinate matrix obtaining unit 1251, configured to obtain, according to the point cloud feature points of the 5 finger tips, 3D node data corresponding to the point cloud feature points of the 5 finger tips to form a three-dimensional coordinate matrix;
a screen coordinate matrix obtaining unit 1252, configured to obtain screen coordinates corresponding to 5 finger tips in the color image to form a screen coordinate matrix;
and a matching matrix calculation unit 1253, configured to multiply the inverse matrix of the three-dimensional coordinate matrix by the screen coordinate matrix to obtain a corresponding matching matrix.
In this embodiment, if a screen coordinate matrix composed of screen coordinates corresponding to 5 finger tips in the color image is a, and a three-dimensional coordinate matrix composed of 3D node data corresponding to point cloud feature points of 5 finger tips is B, then B is used-1And A is H, wherein H is a matching matrix. A matching matrix calculated by taking a three-dimensional coordinate matrix formed by a screen coordinate matrix corresponding to 5 finger tips in a color image and 3D node data corresponding to point cloud characteristic points of the 5 finger tips as a reference can be used as a conversion matrix with higher precision, and mark points of each key sensor node in the color image are effectively converted into standard 3D node data.
A standard action data set obtaining unit 130, configured to perform data conversion on the standard 3D node data according to a preset human action mapping table to obtain a corresponding standard action data set; the human body action mapping table stores mapping relations between various standard 3D node data and standard action data.
In this embodiment, the 3D node data is converted into corresponding descriptive node data, and the subtle changes of the actions can be identified through the descriptive node data, so as to finally obtain a set of standard action data sets.
Specifically, the 3D node data may be understood as three-dimensional space coordinate data corresponding to a human body key node, each 3D node data corresponding to the color image of each frame may form a 3D node data set corresponding to the color image of the frame, a difference value between the 3D node data sets between each two adjacent frames is calculated (this difference value may be recorded as descriptive node data), a human body action value corresponding to the difference value is queried in a preset human body action mapping table according to the difference value, and a corresponding standard action data set may be obtained by combining a plurality of human body action values. The human body action mapping table stores mapping relations between various standard 3D node data and standard action data.
And the current action data set acquisition unit 140 is configured to receive current action feature data acquired and uploaded by the target terminal, and obtain a corresponding current action data set through action decomposition and data conversion according to the human body action mapping table in sequence.
In this embodiment, after the standard motion data set is obtained, the VR device scans the human body motion in real time to obtain the current motion. And acquiring the motion characteristic of the current motion to obtain the current motion characteristic data. And then, performing action decomposition on the current action characteristic data to obtain current 3D node data. And converting the current 3D node data into corresponding current descriptive node data, and identifying slight changes of actions through the current descriptive node data to finally obtain a set of current action data set. The specific process refers to step S110-step S130.
A current similarity calculation unit 150, configured to obtain a current similarity between the human motion sequence in the current motion data set and the corresponding human motion sequence in the standard motion data set; the plurality of human body action values arranged in time sequence in the current action data set form a human body action sequence corresponding to the current action data set, and the plurality of human body action values arranged in time sequence in the standard action data set form a human body action sequence corresponding to the standard action data set.
In the embodiment, human body actions are collected in real time through VR equipment, and are matched and identified with standard actions, and the similarity between the current actions and the standard actions is identified, so that wrong actions are prompted in time.
Specifically, since the current motion data set includes a plurality of human motion values (e.g., [1323579]), this current motion data set can be regarded as a row vector. Similarly, the standard motion data set also comprises a plurality of human body motion value sets as row vectors, and the Euclidean distance between the two row vectors is calculated, so that the current similarity between the current motion data set and the corresponding data set in the standard motion data set can be obtained.
In one embodiment, as shown in fig. 11, the current similarity calculation unit 150 includes:
a first one-dimensional row vector obtaining unit 151, configured to obtain a first one-dimensional row vector corresponding to each human motion sequence in the current motion data set;
a second one-dimensional row vector obtaining unit 152, configured to obtain a second one-dimensional row vector corresponding to each human motion sequence in the standard motion data set;
the euclidean distance calculating unit 153 is configured to obtain a euclidean distance between the first one-dimensional row vector and the second one-dimensional row vector, and use the euclidean distance as the current similarity.
In this embodiment, after the first one-dimensional row vector corresponding to the current motion data set and the second one-dimensional row vector corresponding to the standard motion data set are obtained, the euclidean distance between the two row vectors may be calculated, and the euclidean distance is used as the current similarity. Through the calculation mode, the similarity calculation can be carried out on the current action data set and the standard action data set which are acquired in the same time length, so that the similarity calculation can be used as a parameter basis for timely prompting whether to carry out non-standard action or not.
And the notifying unit 160 is configured to send the reminding information of the current similarity to the corresponding target terminal if the similarity is lower than a preset similarity threshold.
In this embodiment, if the similarity is lower than a preset similarity threshold, the reminding information of the current similarity is sent to the corresponding target terminal, which indicates that the similarity between the collected current action data set and the standard action data set is low after the same duration, that is, the current action corresponding to the current action feature data is not standard, and the user needs to be prompted to correct the action in time. The reminding information comprises a similarity value and text information with the prompting similarity lower than a similarity threshold value. For example, the reminding information is: the similarity of your current action is 90%, which is lower than the standard similarity of 95%, please pay attention to the corrective action.
The device has realized by VR real-time scanning human action to match the discernment with the standard action, the precision discerns the current action and the similarity of standard action, in time suggests to the wrong action.
The VR body sensing data detection apparatus may be implemented in the form of a computer program that is executable on a computer device as shown in fig. 12.
Referring to fig. 12, fig. 12 is a schematic block diagram of a computer device according to an embodiment of the present invention. The computer device 500 is a server, and the server may be an independent server or a server cluster composed of a plurality of servers.
Referring to fig. 12, the computer device 500 includes a processor 502, memory, and a network interface 505 connected by a system bus 501, where the memory may include a non-volatile storage medium 503 and an internal memory 504.
The non-volatile storage medium 503 may store an operating system 5031 and a computer program 5032. The computer program 5032, when executed, may cause the processor 502 to perform a VR somatosensory data detection method.
The processor 502 is used to provide computing and control capabilities that support the operation of the overall computer device 500.
The internal memory 504 provides an environment for running the computer program 5032 in the non-volatile storage medium 503, and when the computer program 5032 is executed by the processor 502, the processor 502 may be caused to execute the VR somatosensory data detection method.
The network interface 505 is used for network communication, such as providing transmission of data information. Those skilled in the art will appreciate that the configuration shown in fig. 12 is a block diagram of only a portion of the configuration associated with aspects of the present invention and is not intended to limit the computing device 500 to which aspects of the present invention may be applied, and that a particular computing device 500 may include more or less components than those shown, or may combine certain components, or have a different arrangement of components.
The processor 502 is configured to run the computer program 5032 stored in the memory, so as to implement the VR somatosensory data detection method disclosed in the embodiment of the invention.
Those skilled in the art will appreciate that the embodiment of a computer device illustrated in fig. 12 does not constitute a limitation on the specific construction of the computer device, and that in other embodiments a computer device may include more or fewer components than those illustrated, or some components may be combined, or a different arrangement of components. For example, in some embodiments, the computer device may only include a memory and a processor, and in such embodiments, the structures and functions of the memory and the processor are consistent with those of the embodiment shown in fig. 12, and are not described herein again.
It should be understood that, in the embodiment of the present invention, the Processor 502 may be a Central Processing Unit (CPU), and the Processor 502 may also be other general purpose processors, Digital Signal Processors (DSPs), Application Specific Integrated Circuits (ASICs), Field-Programmable gate arrays (FPGAs) or other Programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, and the like. Wherein a general purpose processor may be a microprocessor or the processor may be any conventional processor or the like.
In another embodiment of the invention, a computer-readable storage medium is provided. The computer readable storage medium may be a non-volatile computer readable storage medium. The computer readable storage medium stores a computer program, wherein the computer program, when executed by a processor, implements the VR somatosensory data detection method disclosed by the embodiments of the invention.
It is clear to those skilled in the art that, for convenience and brevity of description, the specific working processes of the above-described apparatuses, devices and units may refer to the corresponding processes in the foregoing method embodiments, and are not described herein again. Those of ordinary skill in the art will appreciate that the elements and algorithm steps of the examples described in connection with the embodiments disclosed herein may be embodied in electronic hardware, computer software, or combinations of both, and that the components and steps of the examples have been described in a functional general in the foregoing description for the purpose of illustrating clearly the interchangeability of hardware and software. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the implementation. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present invention.
In the embodiments provided by the present invention, it should be understood that the disclosed apparatus, device and method can be implemented in other ways. For example, the above-described embodiments of the apparatus are merely illustrative, and for example, the division of the units is only a logical division, and there may be other divisions when the actual implementation is performed, or units having the same function may be grouped into one unit, for example, a plurality of units or components may be combined or may be integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection through some interfaces, devices or units, and may also be an electric, mechanical or other form of connection.
The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment of the present invention.
In addition, functional units in the embodiments of the present invention may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit. The integrated unit can be realized in a form of hardware, and can also be realized in a form of a software functional unit.
The integrated unit, if implemented in the form of a software functional unit and sold or used as a stand-alone product, may be stored in a storage medium. Based on such understanding, the technical solution of the present invention essentially or partially contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product stored in a storage medium and including instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the method according to the embodiments of the present invention. And the aforementioned storage medium includes: various media capable of storing program codes, such as a usb disk, a removable hard disk, a Read-Only Memory (ROM), a magnetic disk, or an optical disk.
While the invention has been described with reference to specific embodiments, the invention is not limited thereto, and various equivalent modifications and substitutions can be easily made by those skilled in the art within the technical scope of the invention. Therefore, the protection scope of the present invention shall be subject to the protection scope of the claims.
Claims (10)
1. A VR body feeling data detection method is characterized by comprising the following steps:
receiving standard action characteristic data acquired and uploaded by each key sensor in an acquisition terminal;
performing action decomposition on the standard action characteristic data to obtain standard 3D node data;
performing data conversion on the standard 3D node data according to a preset human body action mapping table to obtain a corresponding standard action data set; the human body action mapping table stores mapping relations between various standard 3D node data and standard action data;
receiving current action characteristic data collected and uploaded by a target terminal, and obtaining a corresponding current action data set through action decomposition and data conversion according to the human body action mapping table in sequence;
acquiring the current similarity between the human body action sequence in the current action data set and the corresponding human body action sequence in the standard action data set; the human body action sequence corresponding to the current action data set is composed of a plurality of human body action values arranged in time sequence in the current action data set, and the human body action sequence corresponding to the standard action data set is composed of a plurality of human body action values arranged in time sequence in the standard action data set; and
and if the similarity is lower than a preset similarity threshold, sending the reminding information of the current similarity to a corresponding target terminal.
2. The VR body feeling data detection method of claim 1, wherein before obtaining standard 3D node data by performing motion decomposition on the standard motion feature data, the VR body feeling data detection method further includes:
collecting a color image corresponding to the standard action characteristic data;
and carrying out graying processing on the color image to obtain a grayscale image.
3. The VR somatosensory data detection method of claim 2, wherein obtaining standard 3D node data by performing motion decomposition on the standard motion feature data comprises:
converting the collected standard action characteristic data into point cloud data;
acquiring screen coordinates corresponding to mark points of each key sensor node on the gray level image;
acquiring point cloud characteristic points in the point cloud data to form a point cloud characteristic point set;
acquiring point cloud feature points of 5 finger tips in a point cloud feature point set and 3D node data corresponding to the point cloud feature points of the 5 finger tips;
correspondingly acquiring a matching matrix according to a screen coordinate matrix corresponding to the point cloud characteristic points of the 5 finger fingertips and according to an inverse matrix of a three-dimensional coordinate matrix corresponding to the 3D node data corresponding to the 5 finger fingertips;
obtaining the residual mark points of 5 finger tips of each key sensor node in the mark points on the gray level image to obtain a residual mark point set on the gray level image;
and multiplying the screen coordinates corresponding to each mark point in the residual mark point set by the matching matrix to obtain standard 3D node data corresponding to the standard action characteristic data.
4. The VR somatosensory data detection method of claim 3, wherein the acquiring screen coordinates corresponding to mark points of each key sensor node on a gray-scale image comprises:
acquiring an initial gray threshold according to the maximum gray value and the minimum gray value of the gray image;
dividing the gray image into a target area and a background area according to the initial gray threshold value to form a segmentation image, and acquiring a first average gray value corresponding to the target area and a second average gray value corresponding to the background area;
and acquiring screen coordinates corresponding to the mark points of each key sensor node on the segmented image.
5. The VR body sensing data detection method of claim 3, wherein the correspondingly obtaining a matching matrix according to a screen coordinate matrix corresponding to the point cloud feature points of 5 finger tips and according to an inverse matrix of a three-dimensional coordinate matrix corresponding to 3D node data corresponding to 5 finger tips comprises:
acquiring 3D node data corresponding to the point cloud characteristic points of the 5 finger fingertips according to the point cloud characteristic points of the 5 finger fingertips to form a three-dimensional coordinate matrix;
acquiring screen coordinates corresponding to 5 finger tips in the color image to form a screen coordinate matrix;
and multiplying the inverse matrix of the three-dimensional coordinate matrix by the screen coordinate matrix to obtain a corresponding matching matrix.
6. The VR somatosensory data detection method of claim 1, wherein the obtaining of the current similarity between the human action sequences in the current action data set and the corresponding human action sequences in the standard action data set comprises:
acquiring a first one-dimensional row vector corresponding to each human body action sequence in the current action data set;
acquiring a second one-dimensional row vector corresponding to each human body action sequence in the standard action data set;
and acquiring the Euclidean distance between the first one-dimensional row vector and the second one-dimensional row vector, and taking the Euclidean distance as the current similarity.
7. The utility model provides a VR body feeling data detection device which characterized in that includes:
the initial standard data acquisition unit is used for receiving standard action characteristic data acquired and uploaded by each key sensor in the acquisition terminal;
the standard 3D node data acquisition unit is used for performing action decomposition on the standard action characteristic data to obtain standard 3D node data;
the standard action data set acquisition unit is used for performing data conversion on the standard 3D node data according to a preset human body action mapping table to obtain a corresponding standard action data set; the human body action mapping table stores mapping relations between various standard 3D node data and standard action data;
the current action data set acquisition unit is used for receiving current action characteristic data acquired and uploaded by the target terminal, and obtaining a corresponding current action data set through action decomposition and data conversion according to the human body action mapping table in sequence;
the current similarity calculation unit is used for acquiring the current similarity between the human motion sequence in the current motion data set and the corresponding human motion sequence in the standard motion data set; the human body action sequence corresponding to the current action data set is composed of a plurality of human body action values arranged in time sequence in the current action data set, and the human body action sequence corresponding to the standard action data set is composed of a plurality of human body action values arranged in time sequence in the standard action data set; and
and the notification unit is used for sending the reminding information of the current similarity to the corresponding target terminal if the similarity is lower than a preset similarity threshold.
8. The VR somatosensory data detection apparatus of claim 7, wherein the current similarity calculation unit comprises:
a first one-dimensional row vector obtaining unit, configured to obtain a first one-dimensional row vector corresponding to each human motion sequence in the current motion data set;
the second one-dimensional row vector acquisition unit is used for acquiring a second one-dimensional row vector corresponding to each human motion sequence in the standard motion data set;
and the Euclidean distance calculating unit is used for acquiring the Euclidean distance between the first one-dimensional row vector and the second one-dimensional row vector, and taking the Euclidean distance as the current similarity.
9. A computer device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, wherein the processor implements the VR somatosensory data detection method of any one of claims 1-6 when executing the computer program.
10. A computer-readable storage medium, characterized in that the computer-readable storage medium stores a computer program which, when executed by a processor, causes the processor to perform the VR somatosensory data detection method of any one of claims 1 to 6.
Priority Applications (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911154959.XA CN111160088A (en) | 2019-11-22 | 2019-11-22 | VR (virtual reality) somatosensory data detection method and device, computer equipment and storage medium |
| PCT/CN2020/087024 WO2021098147A1 (en) | 2019-11-22 | 2020-04-26 | Vr motion sensing data detection method and apparatus, computer device, and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911154959.XA CN111160088A (en) | 2019-11-22 | 2019-11-22 | VR (virtual reality) somatosensory data detection method and device, computer equipment and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN111160088A true CN111160088A (en) | 2020-05-15 |
Family
ID=70556115
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201911154959.XA Pending CN111160088A (en) | 2019-11-22 | 2019-11-22 | VR (virtual reality) somatosensory data detection method and device, computer equipment and storage medium |
Country Status (2)
| Country | Link |
|---|---|
| CN (1) | CN111160088A (en) |
| WO (1) | WO2021098147A1 (en) |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113058261A (en) * | 2021-04-22 | 2021-07-02 | 杭州当贝网络科技有限公司 | Somatosensory action recognition method and system based on reality scene and game scene |
| CN114418460A (en) * | 2022-03-28 | 2022-04-29 | 四川高速公路建设开发集团有限公司 | Construction process information analysis method and construction management system applied to BIM |
| CN114694263A (en) * | 2022-05-30 | 2022-07-01 | 深圳智华科技发展有限公司 | Action recognition method, device, equipment and storage medium |
| CN113058261B (en) * | 2021-04-22 | 2024-04-19 | 杭州当贝网络科技有限公司 | Somatosensory motion recognition method and system based on reality scene and game scene |
Families Citing this family (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113705536A (en) * | 2021-09-18 | 2021-11-26 | 深圳市领存技术有限公司 | Continuous action scoring method, device and storage medium |
| CN115661935B (en) * | 2022-10-31 | 2023-07-11 | 海信集团控股股份有限公司 | Human body action accuracy determining method and device |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2015161307A1 (en) * | 2014-04-18 | 2015-10-22 | Magic Leap, Inc. | Systems and methods for augmented and virtual reality |
| CN107030692A (en) * | 2017-03-28 | 2017-08-11 | 浙江大学 | One kind is based on the enhanced manipulator teleoperation method of perception and system |
| CN108664119A (en) * | 2017-10-31 | 2018-10-16 | 中国农业大学 | A kind of configuration body-sensing acts the method and device of the mapping relations between pseudo operation |
| CN108875708A (en) * | 2018-07-18 | 2018-11-23 | 广东工业大学 | Behavior analysis method, device, equipment, system and storage medium based on video |
| CN110020633A (en) * | 2019-04-12 | 2019-07-16 | 腾讯科技（深圳）有限公司 | Training method, image-recognizing method and the device of gesture recognition model |
-
2019
- 2019-11-22 CN CN201911154959.XA patent/CN111160088A/en active Pending
-
2020
- 2020-04-26 WO PCT/CN2020/087024 patent/WO2021098147A1/en active Application Filing
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2015161307A1 (en) * | 2014-04-18 | 2015-10-22 | Magic Leap, Inc. | Systems and methods for augmented and virtual reality |
| CN107030692A (en) * | 2017-03-28 | 2017-08-11 | 浙江大学 | One kind is based on the enhanced manipulator teleoperation method of perception and system |
| CN108664119A (en) * | 2017-10-31 | 2018-10-16 | 中国农业大学 | A kind of configuration body-sensing acts the method and device of the mapping relations between pseudo operation |
| CN108875708A (en) * | 2018-07-18 | 2018-11-23 | 广东工业大学 | Behavior analysis method, device, equipment, system and storage medium based on video |
| CN110020633A (en) * | 2019-04-12 | 2019-07-16 | 腾讯科技（深圳）有限公司 | Training method, image-recognizing method and the device of gesture recognition model |
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113058261A (en) * | 2021-04-22 | 2021-07-02 | 杭州当贝网络科技有限公司 | Somatosensory action recognition method and system based on reality scene and game scene |
| CN113058261B (en) * | 2021-04-22 | 2024-04-19 | 杭州当贝网络科技有限公司 | Somatosensory motion recognition method and system based on reality scene and game scene |
| CN114418460A (en) * | 2022-03-28 | 2022-04-29 | 四川高速公路建设开发集团有限公司 | Construction process information analysis method and construction management system applied to BIM |
| CN114418460B (en) * | 2022-03-28 | 2022-06-10 | 四川高速公路建设开发集团有限公司 | Construction process information analysis method and construction management system applied to BIM |
| CN114694263A (en) * | 2022-05-30 | 2022-07-01 | 深圳智华科技发展有限公司 | Action recognition method, device, equipment and storage medium |
| CN114694263B (en) * | 2022-05-30 | 2022-09-02 | 深圳智华科技发展有限公司 | Action recognition method, device, equipment and storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| WO2021098147A1 (en) | 2021-05-27 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111160088A (en) | VR (virtual reality) somatosensory data detection method and device, computer equipment and storage medium | |
| CN111209885B (en) | Gesture information processing method and device, electronic equipment and storage medium | |
| CN107485844B (en) | Limb rehabilitation training method and system and embedded equipment | |
| CN109448090B (en) | Image processing method, device, electronic equipment and storage medium | |
| KR102437456B1 (en) | Event camera-based deformable object tracking | |
| JP6942488B2 (en) | Image processing equipment, image processing system, image processing method, and program | |
| CN106846403B (en) | Method and device for positioning hand in three-dimensional space and intelligent equipment | |
| EP3674852B1 (en) | Method and apparatus with gaze estimation | |
| CN112926423B (en) | Pinch gesture detection and recognition method, device and system | |
| CN109934065B (en) | Method and device for gesture recognition | |
| CN107636566B (en) | Wearable device and method for providing feedback of wearable device | |
| CN104364733A (en) | Position-of-interest detection device, position-of-interest detection method, and position-of-interest detection program | |
| JP2016099982A (en) | Behavior recognition device, behaviour learning device, method, and program | |
| JP2018119833A (en) | Information processing device, system, estimation method, computer program, and storage medium | |
| JP2021060989A (en) | Multimodal dense correspondence imaging system | |
| CN107272899B (en) | VR (virtual reality) interaction method and device based on dynamic gestures and electronic equipment | |
| CN111210275B (en) | VR data-based user portrait construction method and device and computer equipment | |
| WO2022174594A1 (en) | Multi-camera-based bare hand tracking and display method and system, and apparatus | |
| CN116664620A (en) | Picture dynamic capturing method and related device based on tracking system | |
| CN110910426A (en) | Action process and action trend identification method, storage medium and electronic device | |
| JP6447521B2 (en) | Information processing apparatus, information processing method, and program | |
| Enikeev et al. | Recognition of sign language using leap motion controller data | |
| CN113407031B (en) | VR (virtual reality) interaction method, VR interaction system, mobile terminal and computer readable storage medium | |
| KR101289883B1 (en) | System and method for generating mask image applied in each threshold in region | |
| KR101868520B1 (en) | Method for hand-gesture recognition and apparatus thereof |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |