CN113706606B - Method and device for determining position coordinates of spaced hand gestures - Google Patents
Method and device for determining position coordinates of spaced hand gestures Download PDFInfo
- Publication number
- CN113706606B CN113706606B CN202110925230.9A CN202110925230A CN113706606B CN 113706606 B CN113706606 B CN 113706606B CN 202110925230 A CN202110925230 A CN 202110925230A CN 113706606 B CN113706606 B CN 113706606B
- Authority
- CN
- China
- Prior art keywords
- target
- position coordinate
- coordinates
- hand
- reference position
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 58
- 230000033001 locomotion Effects 0.000 claims description 19
- 210000003811 finger Anatomy 0.000 claims description 15
- 230000008859 change Effects 0.000 claims description 9
- 210000004932 little finger Anatomy 0.000 claims description 6
- 238000004590 computer program Methods 0.000 claims description 4
- 238000004891 communication Methods 0.000 description 13
- 238000004364 calculation method Methods 0.000 description 7
- 238000012549 training Methods 0.000 description 7
- 238000010586 diagram Methods 0.000 description 6
- 238000005516 engineering process Methods 0.000 description 6
- 230000003993 interaction Effects 0.000 description 6
- 238000001514 detection method Methods 0.000 description 5
- 244000060701 Kaempferia pandurata Species 0.000 description 4
- 235000016390 Uvaria chamae Nutrition 0.000 description 4
- 238000013500 data storage Methods 0.000 description 4
- 230000008569 process Effects 0.000 description 4
- 230000006870 function Effects 0.000 description 3
- 238000012986 modification Methods 0.000 description 3
- 230000004048 modification Effects 0.000 description 3
- 238000012545 processing Methods 0.000 description 3
- 230000009286 beneficial effect Effects 0.000 description 2
- 210000002478 hand joint Anatomy 0.000 description 2
- 238000003062 neural network model Methods 0.000 description 2
- 230000003068 static effect Effects 0.000 description 2
- 238000013473 artificial intelligence Methods 0.000 description 1
- 238000013528 artificial neural network Methods 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 238000004422 calculation algorithm Methods 0.000 description 1
- 230000001419 dependent effect Effects 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 210000004247 hand Anatomy 0.000 description 1
- 238000010801 machine learning Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000001737 promoting effect Effects 0.000 description 1
- 210000003813 thumb Anatomy 0.000 description 1
- 230000009466 transformation Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/70—Determining position or orientation of objects or cameras
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20084—Artificial neural networks [ANN]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30196—Human being; Person
Landscapes
- Engineering & Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- User Interface Of Digital Computer (AREA)
Abstract
The disclosure provides a method and a device for determining a position coordinate of a space-apart gesture, which are used for acquiring an image of a target hand at a target moment and obtaining a joint point coordinate of the target hand according to the image; according to the joint point coordinates, calculating to obtain reference position coordinates of the target hand at the target moment; acquiring a historical position coordinate of the target hand before the target moment, and further calculating to obtain a predicted position coordinate of the target hand at the target moment; and calculating the determined position coordinates of the target hand at the target moment according to the reference position coordinates and the predicted position coordinates, wherein the determined position coordinates can be used for realizing accurate, smooth and stable continuous type spaced gesture control operation.
Description
Technical Field
The disclosure relates to the technical field of man-machine interaction, in particular to a method and a device for determining a space-free gesture position coordinate.
Background
With the development of intelligent interaction technology, the space-apart gesture control technology is increasingly applied to various fields, such as the fields of vehicle-mounted systems, intelligent home, VR content interaction and the like. When the space gesture control system is used for continuous control (such as volume adjustment, cursor movement control and the like), coordinate information of an 'operator' in each frame needs to be obtained, and the movement speed of the 'operator' is calculated, so that the change speed of a controlled object is determined.
However, the accuracy of the hand position directly obtained by the existing target detection technology is not enough, and if the hand position is directly mapped to the change speed of the controlled object, the phenomenon that the change speed of the controlled object is not stable occurs, and the existing gesture position coordinate detection technology cannot be used for realizing accurate, smooth and stable continuous type spaced gesture operation.
Disclosure of Invention
Accordingly, an object of the present disclosure is to provide a method and apparatus for determining the position coordinates of a space-free gesture.
Based on the above object, the present disclosure provides a method for determining a position coordinate of a space-apart gesture, including:
Acquiring an image of a target hand at a target moment, and obtaining joint point coordinates of the target hand according to the image;
according to the joint point coordinates, calculating to obtain reference position coordinates of the target hand at the target moment;
acquiring a historical position coordinate of the target hand before the target moment, and further calculating to obtain a predicted position coordinate of the target hand at the target moment;
And calculating to obtain the determined position coordinates of the target hand at the target moment according to the reference position coordinates and the predicted position coordinates.
Based on the same inventive concept, the present disclosure provides a device for determining a position coordinate of a space-apart gesture, comprising:
The node coordinate acquisition module is configured to acquire an image of a target hand at a target moment and acquire node coordinates of the target hand according to the image;
the reference position coordinate acquisition module is configured to calculate and obtain the reference position coordinate of the target hand at the target moment according to the joint point coordinate;
the predicted position coordinate acquisition module is configured to acquire the historical position coordinate of the target hand before the target moment and further calculate the predicted position coordinate of the target hand at the target moment;
And the determined position coordinate acquisition module is configured to calculate and obtain the determined position coordinate of the target hand at the target moment according to the reference position coordinate and the predicted position coordinate.
Based on the same inventive concept, the present disclosure provides an electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, the processor implementing the method as described above when executing the program.
Based on the same inventive concept, the present disclosure provides a non-transitory computer-readable storage medium storing computer instructions for causing a computer to perform the above-described method.
From the above, it can be seen that the method and the device for determining the position coordinates of the spaced hand gestures provided by the present disclosure acquire an image of a target hand at a target moment, and obtain the coordinates of the joint points of the target hand according to the image; according to the joint point coordinates, calculating to obtain reference position coordinates of the target hand at the target moment; acquiring a historical position coordinate of the target hand before the target moment, and further calculating to obtain a predicted position coordinate of the target hand at the target moment; and calculating the determined position coordinates of the target hand at the target moment according to the reference position coordinates and the predicted position coordinates, wherein the determined position coordinates can be used for realizing accurate, smooth and stable continuous type spaced gesture control operation.
Drawings
In order to more clearly illustrate the technical solutions of the present disclosure or related art, the drawings required for the embodiments or related art description will be briefly described below, and it is apparent that the drawings in the following description are only embodiments of the present disclosure, and other drawings may be obtained according to these drawings without inventive effort to those of ordinary skill in the art.
Fig. 1 is a schematic view of an application scenario of a method for determining position coordinates of a space-apart gesture according to an embodiment of the present disclosure;
FIG. 2 is a schematic flow chart of a method for determining position coordinates of a space-apart gesture according to an embodiment of the present disclosure;
FIG. 3 is a schematic illustration of joint coordinates provided in accordance with an embodiment of the present disclosure;
FIG. 4 is a schematic illustration of a joint point number provided in accordance with an embodiment of the present disclosure;
FIG. 5 is a schematic structural diagram of an apparatus for determining position coordinates of a space-apart gesture according to an embodiment of the present disclosure;
Fig. 6 is a schematic diagram of a hardware structure of an electronic device according to an embodiment of the disclosure.
Detailed Description
For the purposes of promoting an understanding of the principles and advantages of the disclosure, reference will now be made to the embodiments illustrated in the drawings and specific language will be used to describe the same.
It should be noted that unless otherwise defined, technical or scientific terms used in the embodiments of the present disclosure should be given the ordinary meaning as understood by one of ordinary skill in the art to which the present disclosure pertains. The terms "first," "second," and the like, as used in embodiments of the present disclosure, do not denote any order, quantity, or importance, but rather are used to distinguish one element from another. The word "comprising" or "comprises", and the like, means that elements or items preceding the word are included in the element or item listed after the word and equivalents thereof, but does not exclude other elements or items. The terms "connected" or "connected," and the like, are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. "upper", "lower", "left", "right", etc. are used merely to indicate relative positional relationships, which may also be changed when the absolute position of the object to be described is changed.
Referring to fig. 1, an application scenario diagram of a method for determining position coordinates of a space gesture according to an embodiment of the present disclosure is provided. The application scenario includes a terminal device 101, a server 102, and a data storage system 103. The terminal device 101, the server 102 and the data storage system 103 may be connected through a wired or wireless communication network. Terminal device 101 includes, but is not limited to, a desktop computer, a mobile phone, a mobile computer, a tablet computer, a media player, a smart wearable device, a Personal Digital Assistant (PDA) or other electronic device capable of performing the functions described above, and the like. The server 102 and the data storage system 103 may be independent physical servers, or may be a server cluster or a distributed system formed by a plurality of physical servers, or may be a cloud server that provides cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDNs, and basic cloud computing services such as big data and artificial intelligence platforms.
The server 102 is configured to provide a service for determining position coordinates of a space-apart gesture to a user of the terminal device 101, the terminal device 101 is capable of communicating with the server 102, the terminal device 101 obtains a gesture image of the user and sends the gesture image of the user to the server 102, and the server 102 processes the gesture image of the user to obtain the determined position coordinates of the gesture of the user at a target moment, so that the terminal device 101 is applied to a space-apart gesture control scenario, such as a scenario of a vehicle-mounted system, an intelligent home, VR content interaction, and the like.
The data storage system 103 has stored therein a large amount of training data, sources of which include, but are not limited to, existing databases, data crawled from the internet, or data uploaded when a user uses a client. The server 102 may also continuously optimize the service of determining the alternate gesture position coordinates based on the newly added training data.
The method for determining the space-division gesture position coordinates can be applied to the related fields of space-division gesture control, such as the fields of vehicle-mounted systems, intelligent home, VR content interaction and the like.
A method of determining the position coordinates of a space gesture according to an exemplary embodiment of the present disclosure will be described below in conjunction with the application scenario of fig. 1. It should be noted that the above application scenario is only shown for the convenience of understanding the spirit and principles of the present disclosure, and the embodiments of the present disclosure are not limited in any way in this respect. Rather, embodiments of the present disclosure may be applied to any scenario where applicable.
FIG. 2 is a schematic flow chart of a method for determining position coordinates of a space-apart gesture according to an embodiment of the present disclosure; a method of determining position coordinates of a spaced hand gesture comprising:
S210, acquiring an image of a target hand at a target moment, and obtaining joint point coordinates of the target hand according to the image.
The method and the device are applied to the related field of the space-apart gesture control, the device for realizing the space-apart gesture control is usually externally connected or internally provided with a camera, and the device acquires an image containing a target hand (or an operation hand) in real time through the camera so as to realize the space-apart gesture control.
In some embodiments, the acquired image containing the target hand contains multiple hands, in which case the image needs to be preprocessed to obtain an image containing only one and the main hand, i.e., the target hand.
In some embodiments, the obtained image containing the target hand also contains other body parts, such as arms, trunk, etc., in which case the image needs to be preprocessed to obtain an image containing only the target hand.
In some embodiments, the coordinates of the joint points of the target hand are obtained from the image using technical methods in the related art that can be used for image processing, detection and recognition. The coordinates of the identified joint points are shown in fig. 3. As one example, a method of constructing an articulation point detection model based on a deep neural network model (DNN) framework, the method comprising:
constructing a sample set comprising a number of samples; wherein the sample comprises: sample data and tag data; the sample data comprises a training target hand image; the label data comprise target hand joint point coordinates corresponding to the training target hand image;
and constructing and training the joint point detection model through a preset machine learning algorithm such as a deep neural network according to the sample set.
The training target hand image and the target hand joint point coordinates corresponding to the training target hand image can be obtained from historical case data. It should be noted that the present disclosure is not related to improvements to deep neural network models.
In some embodiments, each joint point corresponds to a unique sequential number.
Referring to fig. 4, which is a schematic diagram of joint point numbers provided according to an embodiment of the present disclosure, as an example, 1,2,3, and 4 are numbers of four joints from a thumb root to a fingertip, and numbers of other joints are shown in fig. 4, which are not described herein, but all joint point numbers used in the present disclosure refer to fig. 4, which is an example.
The obtained joint point coordinates are corresponding to the numbers of the joint points, so that the joint point coordinates can be conveniently searched and utilized. For example, the position setting of the articulation point 5 is labeled P 5(x5,y5), and similarly, the position setting of the articulation point 9 is labeled P 9(x9,y9), and the position setting of the articulation point 13 is labeled P 13(x13,y13).
And S220, calculating the reference position coordinates of the target hand at the target moment according to the joint point coordinates.
The method specifically comprises the following steps:
And determining the joint point coordinates, the relative positions of which are less likely to change than other joint point coordinates, in the joint point coordinates as reference joint point coordinates, and calculating the reference position coordinates based on the reference joint point coordinates.
The calculating, based on the reference joint point coordinates, the reference position coordinates includes:
And calculating an average value of the reference joint point coordinates, and taking the average value as the reference position coordinates. The calculation formula is as follows:
4≤n＜m；
Wherein P h0|k denotes a reference position coordinate; p i1,Pi2,Pin-1,Pin denotes the reference joint point coordinates; n represents the number of reference joint coordinates; m represents the number of coordinates of the joint point.
Wherein the reference articulation point coordinates comprise:
and the coordinates of the joint points corresponding to the root parts of the index finger, the middle finger, the ring finger and the little finger of the target hand.
Referring to fig. 3 and 4, in the gesture transformation and movement process, the relative positions of the five joints 0, 5, 9, 13 and 17 of the 21 joints of the target hand are less prone to change than the relative positions of the other joints, namely the relative positions are more stable, and the four joints 5, 9, 13 and 17 of the joints are positioned in the middle of the whole hand, so that the four joints can be used as references to comprehensively calculate to preliminarily determine the reference position coordinates of the target hand. In this case, the reference position coordinates of the target hand are calculated based on the coordinates of the joints corresponding to the index finger root, the middle finger root, the ring finger root and the little finger root of the target hand, and the calculation formula is as follows:
Wherein P h0|k denotes a reference position coordinate; p 5,P9,P13,P17 sequentially and respectively represents the coordinates of the joint points corresponding to the root of the index finger, the root of the middle finger, the root of the ring finger and the root of the little finger.
S230, acquiring historical position coordinates of the target hand before the target moment, and further calculating to obtain predicted position coordinates of the target hand at the target moment.
If the frequency of image acquisition is fast enough, the motion between two adjacent frames can be considered uniform motion. Assuming that there are two times k-2 and k-1 before the target time (k time), the movement speed of the hand between k-2 and k-1 is v k-1(vk-1|x,vk-1|y), and the movement speed of the hand between k-1 and k is v k(vk|x,vk|y. Then there are: v k＝vk-1.
Based on this, in some embodiments, S230 specifically includes:
acquiring the historical position coordinates of the target hand at a first moment and a second moment before the target moment;
Determining a time difference between the first time and the second time, and a time difference between the target time and the first time;
And regarding the motion of the target hand between two adjacent frames of images as uniform motion, and calculating to obtain the predicted position coordinate based on the historical position coordinate, the time difference between the first time and the second time and the time difference between the target time and the first time.
The first time and the second time are corresponding to two adjacent frames of images.
The calculation formula is as follows:
Wherein the first instant k-1 is after the second instant k-2; p h1|k denotes the predicted position coordinates; Δ t(k|k-1) represents the time difference between the target time and the first time; Δ t(k-1|k-2) represents the time difference between the first time and the second time; p h|k-1 represents the historical position coordinates of the target hand at the first time; p h|k-2 denotes the historical position coordinates of the target hand at the second time.
Specifically, the movement speed of the hand between the target time and the first time is
The movement speed of the hand between the first moment and the second moment is
Based on v k＝vk-1, there are:
and S240, calculating to obtain the determined position coordinates of the target hand at the target moment according to the reference position coordinates and the predicted position coordinates.
In some embodiments, S240 specifically includes:
and acquiring the credibility of the reference position coordinates and the credibility of the predicted position coordinates.
In the actual motion process of the gesture, uniform motion cannot be completely met, so that the reliability of the calculated predicted position coordinate P h1|k is determined to be beta.
And calculating the determined position coordinates of the target hand at the target moment based on the reference position coordinates, the predicted position coordinates, the credibility of the reference position coordinates and the credibility of the predicted position coordinates. The method specifically comprises the following steps: and taking the weighted average of the reference position coordinate and the predicted position coordinate as the determined position coordinate, wherein the credibility of the reference position coordinate and the credibility of the predicted position coordinate are weights of the reference position coordinate and the predicted position coordinate respectively.
The calculation formula is as follows:
wherein P h|k represents a determined position coordinate; alpha represents the credibility of the reference position coordinates; beta represents the credibility of the predicted position coordinates; p h0|k denotes a reference position coordinate; p h1|k denotes the predicted position coordinates.
From the above, it can be seen that the method and the device for determining the position coordinates of the spaced hand gestures provided by the present disclosure acquire an image of a target hand at a target moment, and obtain the coordinates of the joint points of the target hand according to the image; according to the joint point coordinates, calculating to obtain reference position coordinates of the target hand at the target moment; acquiring a historical position coordinate of the target hand before the target moment, and further calculating to obtain a predicted position coordinate of the target hand at the target moment; and calculating the determined position coordinates of the target hand at the target moment according to the reference position coordinates and the predicted position coordinates, wherein the determined position coordinates can be used for realizing accurate, smooth and stable continuous type spaced gesture control operation.
It should be noted that the method of the embodiments of the present disclosure may be performed by a single device, such as a computer or a server. The method of the embodiment can also be applied to a distributed scene, and is completed by mutually matching a plurality of devices. In the case of such a distributed scenario, one of the devices may perform only one or more steps of the methods of embodiments of the present disclosure, the devices interacting with each other to accomplish the methods.
It should be noted that the foregoing describes some embodiments of the present disclosure. Other embodiments are within the scope of the following claims. In some cases, the actions or steps recited in the claims may be performed in a different order than in the embodiments described above and still achieve desirable results. In addition, the processes depicted in the accompanying figures do not necessarily require the particular order shown, or sequential order, to achieve desirable results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.
Based on the same inventive concept, the present disclosure also provides a device for determining the position coordinates of the spaced hand gestures, corresponding to the method of any embodiment described above.
Referring to fig. 5, the apparatus for determining the position coordinates of the spaced hand gestures includes:
The node coordinate acquisition module 510 is configured to acquire an image of a target hand at a target moment, and obtain node coordinates of the target hand according to the image;
a reference position coordinate acquiring module 520 configured to calculate, according to the joint point coordinates, a reference position coordinate of the target hand at the target time;
a predicted position coordinate obtaining module 530 configured to obtain a historical position coordinate of the target hand before the target time, and further calculate a predicted position coordinate of the target hand at the target time;
The determined position coordinate acquiring module 540 is configured to calculate a determined position coordinate of the target hand at the target moment according to the reference position coordinate and the predicted position coordinate.
In some embodiments, the reference position coordinate acquisition module 520 is specifically configured to:
And determining the joint point coordinates, the relative positions of which are less likely to change than other joint point coordinates, in the joint point coordinates as reference joint point coordinates, and calculating the reference position coordinates based on the reference joint point coordinates.
In some embodiments, the reference position coordinate acquisition module 520 is specifically configured to:
And calculating an average value of the reference joint point coordinates, and taking the average value as the reference position coordinates. The calculation formula is as follows:
4≤n＜m；
Wherein P h0|k denotes a reference position coordinate; p i1,Pi2,Pin-1,Pin denotes the reference joint point coordinates; n represents the number of reference joint coordinates; m represents the number of coordinates of the joint point.
Wherein the reference articulation point coordinates comprise:
and the coordinates of the joint points corresponding to the root parts of the index finger, the middle finger, the ring finger and the little finger of the target hand.
In some embodiments, the predicted position coordinate acquisition module 530 is specifically configured to:
acquiring the historical position coordinates of the target hand at a first moment and a second moment before the target moment;
Determining a time difference between the first time and the second time, and a time difference between the target time and the first time;
and regarding the motion of the target hand between two adjacent frames of images as uniform motion, and calculating to obtain the predicted position coordinate based on the historical position coordinate, the time difference between the first time and the second time and the time difference between the target time and the first time. The calculation formula is as follows:
Wherein the first time is after the second time; p h1|k denotes the predicted position coordinates; Δ t(k|k-1) represents the time difference between the target time and the first time; Δ t(k-1|k-2) represents the time difference between the first time and the second time; p h|k-1 represents the historical position coordinates of the target hand at the first time; p h|k-2 denotes the historical position coordinates of the target hand at the second time.
In some embodiments, the determining location coordinate acquisition module 540 is specifically configured to:
acquiring the credibility of the reference position coordinates and the credibility of the predicted position coordinates;
and calculating the determined position coordinates of the target hand at the target moment based on the reference position coordinates, the predicted position coordinates, the credibility of the reference position coordinates and the credibility of the predicted position coordinates. The method specifically comprises the following steps: and taking the weighted average of the reference position coordinate and the predicted position coordinate as the determined position coordinate, wherein the credibility of the reference position coordinate and the credibility of the predicted position coordinate are weights of the reference position coordinate and the predicted position coordinate respectively. The calculation formula is as follows:
wherein P h|k represents a determined position coordinate; alpha represents the credibility of the reference position coordinates; beta represents the credibility of the predicted position coordinates; p h0|k denotes a reference position coordinate; p h1|k denotes the predicted position coordinates.
For convenience of description, the above devices are described as being functionally divided into various modules, respectively. Of course, the functions of the various modules may be implemented in the same one or more pieces of software and/or hardware when implementing the present disclosure.
The device of the foregoing embodiment is configured to implement the method for determining the position coordinates of the spaced gesture in any of the foregoing embodiments, and has the beneficial effects of the corresponding method embodiments, which are not described herein.
Based on the same inventive concept, the present disclosure also provides an electronic device corresponding to the method of any embodiment, including a memory, a processor, and a computer program stored on the memory and capable of running on the processor, where the processor implements the method for determining the position coordinates of the space gesture according to any embodiment when executing the program.
Fig. 6 shows a more specific hardware architecture of an electronic device according to this embodiment, where the device may include: a processor 1010, a memory 1020, an input/output interface 1030, a communication interface 1040, and a bus 1050. Wherein processor 1010, memory 1020, input/output interface 1030, and communication interface 1040 implement communication connections therebetween within the device via a bus 1050.
The processor 1010 may be implemented by a general-purpose CPU (Central Processing Unit ), a microprocessor, an Application SPECIFIC INTEGRATED Circuit (ASIC), or one or more integrated circuits, etc. for executing related programs to implement the technical solutions provided in the embodiments of the present disclosure.
The Memory 1020 may be implemented in the form of ROM (Read Only Memory), RAM (RandomAccess Memory ), static storage, dynamic storage, etc. Memory 1020 may store an operating system and other application programs, and when the embodiments of the present specification are implemented in software or firmware, the associated program code is stored in memory 1020 and executed by processor 1010.
The input/output interface 1030 is used to connect with an input/output module for inputting and outputting information. The input/output module may be configured as a component in a device (not shown in the figure) or may be external to the device to provide corresponding functionality. Wherein the input devices may include a keyboard, mouse, touch screen, microphone, various types of sensors, etc., and the output devices may include a display, speaker, vibrator, indicator lights, etc.
Communication interface 1040 is used to connect communication modules (not shown) to enable communication interactions of the present device with other devices. The communication module may implement communication through a wired manner (such as USB, network cable, etc.), or may implement communication through a wireless manner (such as mobile network, WIFI, bluetooth, etc.).
Bus 1050 includes a path for transferring information between components of the device (e.g., processor 1010, memory 1020, input/output interface 1030, and communication interface 1040).
It should be noted that although the above-described device only shows processor 1010, memory 1020, input/output interface 1030, communication interface 1040, and bus 1050, in an implementation, the device may include other components necessary to achieve proper operation. Furthermore, it will be understood by those skilled in the art that the above-described apparatus may include only the components necessary to implement the embodiments of the present description, and not all the components shown in the drawings.
The electronic device of the foregoing embodiment is configured to implement the method for determining the position coordinates of the spaced gesture in any of the foregoing embodiments, and has the beneficial effects of the corresponding method embodiments, which are not described herein.
Based on the same inventive concept, corresponding to any of the above embodiments of the method, the present disclosure further provides a non-transitory computer readable storage medium storing computer instructions for causing the computer to perform the method of determining the space-free gesture position coordinates as described in any of the above embodiments.
The computer readable media of the present embodiments, including both permanent and non-permanent, removable and non-removable media, may be used to implement information storage by any method or technology. The information may be computer readable instructions, data structures, modules of a program, or other data. Examples of storage media for a computer include, but are not limited to, phase change memory (PRAM), static Random Access Memory (SRAM), dynamic Random Access Memory (DRAM), other types of Random Access Memory (RAM), read Only Memory (ROM), electrically Erasable Programmable Read Only Memory (EEPROM), flash memory or other memory technology, compact disc read only memory (CD-ROM), digital Versatile Discs (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices, or any other non-transmission medium, which can be used to store information that can be accessed by a computing device.
The storage medium of the foregoing embodiments stores computer instructions for causing the computer to execute the method for determining the position coordinates of the spaced gesture according to any one of the foregoing embodiments, and has the advantages of the corresponding method embodiments, which are not described herein.
It should be noted that the embodiments of the present disclosure may be further described in the following manner:
A method of determining position coordinates of a spaced hand gesture comprising:
Acquiring an image of a target hand at a target moment, and obtaining joint point coordinates of the target hand according to the image;
according to the joint point coordinates, calculating to obtain reference position coordinates of the target hand at the target moment;
acquiring a historical position coordinate of the target hand before the target moment, and further calculating to obtain a predicted position coordinate of the target hand at the target moment;
And calculating to obtain the determined position coordinates of the target hand at the target moment according to the reference position coordinates and the predicted position coordinates.
Optionally, the calculating, according to the joint point coordinates, a reference position coordinate of the target hand at the target moment includes:
And determining the joint point coordinates, the relative positions of which are less likely to change than other joint point coordinates, in the joint point coordinates as reference joint point coordinates, and calculating the reference position coordinates based on the reference joint point coordinates.
Optionally, the calculating, based on the reference joint point coordinates, the reference position coordinates includes:
And calculating an average value of the reference joint point coordinates, and taking the average value as the reference position coordinates.
Optionally, the reference node coordinates include:
and the coordinates of the joint points corresponding to the root parts of the index finger, the middle finger, the ring finger and the little finger of the target hand.
Optionally, the acquiring the historical position coordinate of the target hand before the target moment, and further calculating to obtain the predicted position coordinate of the target hand at the target moment, includes:
acquiring the historical position coordinates of the target hand at a first moment and a second moment before the target moment;
Determining a time difference between the first time and the second time, and a time difference between the target time and the first time;
And regarding the motion of the target hand between two adjacent frames of images as uniform motion, and calculating to obtain the predicted position coordinate based on the historical position coordinate, the time difference between the first time and the second time and the time difference between the target time and the first time.
Optionally, the calculating, according to the reference position coordinate and the predicted position coordinate, a determined position coordinate of the target hand at the target moment includes:
acquiring the credibility of the reference position coordinates and the credibility of the predicted position coordinates;
Calculating a determined position coordinate of the target hand at the target moment based on the reference position coordinate, the predicted position coordinate, the credibility of the reference position coordinate and the credibility of the predicted position coordinate; the method specifically comprises the following steps: and taking the weighted average of the reference position coordinate and the predicted position coordinate as the determined position coordinate, wherein the credibility of the reference position coordinate and the credibility of the predicted position coordinate are weights of the reference position coordinate and the predicted position coordinate respectively.
An apparatus for determining position coordinates of a spaced hand gesture, comprising:
The node coordinate acquisition module is configured to acquire an image of a target hand at a target moment and acquire node coordinates of the target hand according to the image;
the reference position coordinate acquisition module is configured to calculate and obtain the reference position coordinate of the target hand at the target moment according to the joint point coordinate;
the predicted position coordinate acquisition module is configured to acquire the historical position coordinate of the target hand before the target moment and further calculate the predicted position coordinate of the target hand at the target moment;
And the determined position coordinate acquisition module is configured to calculate and obtain the determined position coordinate of the target hand at the target moment according to the reference position coordinate and the predicted position coordinate.
Optionally, the determining location coordinate acquiring module is specifically configured to:
acquiring the credibility of the reference position coordinates and the credibility of the predicted position coordinates;
And calculating the determined position coordinates of the target hand at the target moment based on the reference position coordinates, the predicted position coordinates, the credibility of the reference position coordinates and the credibility of the predicted position coordinates.
An electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, the processor implementing a method as described above when executing the program.
A non-transitory computer readable storage medium storing computer instructions for causing a computer to perform the above-described method.
Those of ordinary skill in the art will appreciate that: the discussion of any of the embodiments above is merely exemplary and is not intended to suggest that the scope of the disclosure, including the claims, is limited to these examples; the technical features of the above embodiments or in the different embodiments may also be combined under the idea of the present disclosure, the steps may be implemented in any order, and there are many other variations of the different aspects of the embodiments of the present disclosure as described above, which are not provided in details for the sake of brevity.
Additionally, well-known power/ground connections to Integrated Circuit (IC) chips and other components may or may not be shown within the provided figures, in order to simplify the illustration and discussion, and so as not to obscure the embodiments of the present disclosure. Furthermore, the devices may be shown in block diagram form in order to avoid obscuring the embodiments of the present disclosure, and this also accounts for the fact that specifics with respect to implementation of such block diagram devices are highly dependent upon the platform on which the embodiments of the present disclosure are to be implemented (i.e., such specifics should be well within purview of one skilled in the art). Where specific details (e.g., circuits) are set forth in order to describe example embodiments of the disclosure, it should be apparent to one skilled in the art that embodiments of the disclosure can be practiced without, or with variation of, these specific details. Accordingly, the description is to be regarded as illustrative in nature and not as restrictive.
While the present disclosure has been described in conjunction with specific embodiments thereof, many alternatives, modifications, and variations of those embodiments will be apparent to those skilled in the art in light of the foregoing description. For example, other memory architectures (e.g., dynamic RAM (DRAM)) may use the embodiments discussed.
The disclosed embodiments are intended to embrace all such alternatives, modifications and variances which fall within the broad scope of the appended claims. Accordingly, any omissions, modifications, equivalents, improvements, and the like, which are within the spirit and principles of the embodiments of the disclosure, are intended to be included within the scope of the disclosure.
Claims (10)
1. A method of determining position coordinates of a spaced hand gesture comprising:
Acquiring an image of a target hand at a target moment, and obtaining joint point coordinates of the target hand according to the image;
according to the joint point coordinates, calculating to obtain reference position coordinates of the target hand at the target moment;
acquiring a historical position coordinate of the target hand before the target moment, and further calculating to obtain a predicted position coordinate of the target hand at the target moment;
And calculating to obtain the determined position coordinates of the target hand at the target moment according to the reference position coordinates and the predicted position coordinates.
2. The method according to claim 1, wherein calculating the reference position coordinates of the target hand at the target time according to the joint point coordinates includes:
And determining the joint point coordinates, the relative positions of which are less likely to change than other joint point coordinates, in the joint point coordinates as reference joint point coordinates, and calculating the reference position coordinates based on the reference joint point coordinates.
3. The method of claim 2, wherein the calculating the reference position coordinates based on the reference joint point coordinates comprises:
And calculating an average value of the reference joint point coordinates, and taking the average value as the reference position coordinates.
4. The method of claim 2, wherein the reference joint point coordinates comprise:
and the coordinates of the joint points corresponding to the root parts of the index finger, the middle finger, the ring finger and the little finger of the target hand.
5. The method of claim 1, wherein the obtaining historical position coordinates of the target hand prior to the target time and further calculating predicted position coordinates of the target hand at the target time comprises:
acquiring the historical position coordinates of the target hand at a first moment and a second moment before the target moment;
Determining a time difference between the first time and the second time, and a time difference between the target time and the first time;
And regarding the motion of the target hand between two adjacent frames of images as uniform motion, and calculating to obtain the predicted position coordinate based on the historical position coordinate, the time difference between the first time and the second time and the time difference between the target time and the first time.
6. The method according to claim 1, wherein the calculating the determined position coordinates of the target hand at the target time according to the reference position coordinates and the predicted position coordinates includes:
acquiring the credibility of the reference position coordinates and the credibility of the predicted position coordinates;
Calculating a determined position coordinate of the target hand at the target moment based on the reference position coordinate, the predicted position coordinate, the credibility of the reference position coordinate and the credibility of the predicted position coordinate; the method specifically comprises the following steps: and taking the weighted average of the reference position coordinate and the predicted position coordinate as the determined position coordinate, wherein the credibility of the reference position coordinate and the credibility of the predicted position coordinate are weights of the reference position coordinate and the predicted position coordinate respectively.
7. An apparatus for determining the position coordinates of a spaced hand gesture, comprising:
The node coordinate acquisition module is configured to acquire an image of a target hand at a target moment and acquire node coordinates of the target hand according to the image;
the reference position coordinate acquisition module is configured to calculate and obtain the reference position coordinate of the target hand at the target moment according to the joint point coordinate;
the predicted position coordinate acquisition module is configured to acquire the historical position coordinate of the target hand before the target moment and further calculate the predicted position coordinate of the target hand at the target moment;
And the determined position coordinate acquisition module is configured to calculate and obtain the determined position coordinate of the target hand at the target moment according to the reference position coordinate and the predicted position coordinate.
8. The apparatus of claim 7, wherein the determined location coordinate acquisition module is specifically configured to:
acquiring the credibility of the reference position coordinates and the credibility of the predicted position coordinates;
Calculating a determined position coordinate of the target hand at the target moment based on the reference position coordinate, the predicted position coordinate, the credibility of the reference position coordinate and the credibility of the predicted position coordinate; the method specifically comprises the following steps: and taking the weighted average of the reference position coordinate and the predicted position coordinate as the determined position coordinate, wherein the credibility of the reference position coordinate and the credibility of the predicted position coordinate are weights of the reference position coordinate and the predicted position coordinate respectively.
9. An electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, the processor implementing the method of any one of claims 1 to 6 when the program is executed.
10. A non-transitory computer readable storage medium storing computer instructions for causing a computer to perform the method of any one of claims 1 to 6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110925230.9A CN113706606B (en) | 2021-08-12 | 2021-08-12 | Method and device for determining position coordinates of spaced hand gestures |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110925230.9A CN113706606B (en) | 2021-08-12 | 2021-08-12 | Method and device for determining position coordinates of spaced hand gestures |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113706606A CN113706606A (en) | 2021-11-26 |
| CN113706606B true CN113706606B (en) | 2024-04-30 |
Family
ID=78652473
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110925230.9A Active CN113706606B (en) | 2021-08-12 | 2021-08-12 | Method and device for determining position coordinates of spaced hand gestures |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113706606B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2023189838A1 (en) * | 2022-04-01 | 2023-10-05 | ソニーグループ株式会社 | Information processing device, information processing method, and program |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104992171A (en) * | 2015-08-04 | 2015-10-21 | 易视腾科技有限公司 | Method and system for gesture recognition and man-machine interaction based on 2D video sequence |
| CN110443154A (en) * | 2019-07-15 | 2019-11-12 | 北京达佳互联信息技术有限公司 | Three-dimensional coordinate localization method, device, electronic equipment and the storage medium of key point |
| CN111414837A (en) * | 2020-03-16 | 2020-07-14 | 苏州交驰人工智能研究院有限公司 | Gesture recognition method and device, computer equipment and storage medium |
| CN112486326A (en) * | 2020-12-14 | 2021-03-12 | 安徽鸿程光电有限公司 | Position coordinate prediction method for gesture control movement, intelligent terminal and medium |
Family Cites Families (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108052202B (en) * | 2017-12-11 | 2021-06-11 | 深圳市星野信息技术有限公司 | 3D interaction method and device, computer equipment and storage medium |
| CN109446994B (en) * | 2018-10-30 | 2020-10-30 | 北京达佳互联信息技术有限公司 | Gesture key point detection method and device, electronic equipment and storage medium |
-
2021
- 2021-08-12 CN CN202110925230.9A patent/CN113706606B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104992171A (en) * | 2015-08-04 | 2015-10-21 | 易视腾科技有限公司 | Method and system for gesture recognition and man-machine interaction based on 2D video sequence |
| CN110443154A (en) * | 2019-07-15 | 2019-11-12 | 北京达佳互联信息技术有限公司 | Three-dimensional coordinate localization method, device, electronic equipment and the storage medium of key point |
| CN111414837A (en) * | 2020-03-16 | 2020-07-14 | 苏州交驰人工智能研究院有限公司 | Gesture recognition method and device, computer equipment and storage medium |
| CN112486326A (en) * | 2020-12-14 | 2021-03-12 | 安徽鸿程光电有限公司 | Position coordinate prediction method for gesture control movement, intelligent terminal and medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113706606A (en) | 2021-11-26 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111401406B (en) | Neural network training method, video frame processing method and related equipment | |
| WO2018194994A2 (en) | Enhancing processing performance of a dnn module by bandwidth control of fabric interface | |
| US11126660B1 (en) | High dimensional time series forecasting | |
| CN111428805B (en) | Method for detecting salient object, model, storage medium and electronic device | |
| CN112106042A (en) | Electronic device and control method thereof | |
| CN110956131B (en) | Single-target tracking method, device and system | |
| CN114241597A (en) | Posture recognition method and related equipment thereof | |
| CN113706606B (en) | Method and device for determining position coordinates of spaced hand gestures | |
| CN112288483A (en) | Method and device for training model and method and device for generating information | |
| CN111601129B (en) | Control method, control device, terminal and storage medium | |
| WO2023246735A1 (en) | Item recommendation method and related device therefor | |
| CN114202799A (en) | Method and device for determining change speed of controlled object, electronic equipment and storage medium | |
| CN110069126B (en) | Virtual object control method and device | |
| CN111368668A (en) | Three-dimensional hand recognition method and device, electronic equipment and storage medium | |
| CN114707070A (en) | User behavior prediction method and related equipment thereof | |
| CN115623242A (en) | Video processing method and related equipment thereof | |
| CN114648556A (en) | Visual tracking method and device and electronic equipment | |
| CN114841361A (en) | Model training method and related equipment thereof | |
| KR20220022480A (en) | Neural Processing Elements with Increased Precision | |
| Ding et al. | Image‐based localisation using shared‐information double stream hourglass networks | |
| CN116703498B (en) | Commodity recommendation method and device, electronic equipment and storage medium | |
| WO2023236900A1 (en) | Item recommendation method and related device thereof | |
| CN113407046A (en) | User action recognition method and device, electronic equipment and storage medium | |
| CN109255095B (en) | IMU data integration method and device, computer readable medium and electronic equipment | |
| CN117349514A (en) | Model training method and related equipment thereof |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |