CN113706606A - Method and device for determining position coordinates of spaced gestures - Google Patents
Method and device for determining position coordinates of spaced gestures Download PDFInfo
- Publication number
- CN113706606A CN113706606A CN202110925230.9A CN202110925230A CN113706606A CN 113706606 A CN113706606 A CN 113706606A CN 202110925230 A CN202110925230 A CN 202110925230A CN 113706606 A CN113706606 A CN 113706606A
- Authority
- CN
- China
- Prior art keywords
- target
- coordinates
- position coordinates
- time
- position coordinate
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 55
- 210000003811 finger Anatomy 0.000 claims description 18
- 230000033001 locomotion Effects 0.000 claims description 17
- 230000008859 change Effects 0.000 claims description 9
- 210000004932 little finger Anatomy 0.000 claims description 6
- 238000004590 computer program Methods 0.000 claims description 4
- 238000004891 communication Methods 0.000 description 12
- 238000010586 diagram Methods 0.000 description 9
- 238000005516 engineering process Methods 0.000 description 9
- 238000004364 calculation method Methods 0.000 description 8
- 238000012549 training Methods 0.000 description 7
- 238000001514 detection method Methods 0.000 description 6
- 230000003993 interaction Effects 0.000 description 6
- 238000013500 data storage Methods 0.000 description 4
- 230000006870 function Effects 0.000 description 4
- 230000008569 process Effects 0.000 description 4
- 230000009286 beneficial effect Effects 0.000 description 3
- 238000012986 modification Methods 0.000 description 3
- 230000004048 modification Effects 0.000 description 3
- 238000012545 processing Methods 0.000 description 3
- 210000002478 hand joint Anatomy 0.000 description 2
- 230000006872 improvement Effects 0.000 description 2
- 238000003062 neural network model Methods 0.000 description 2
- 230000003068 static effect Effects 0.000 description 2
- 244000060701 Kaempferia pandurata Species 0.000 description 1
- 235000016390 Uvaria chamae Nutrition 0.000 description 1
- 238000013473 artificial intelligence Methods 0.000 description 1
- 238000013528 artificial neural network Methods 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 238000004422 calculation algorithm Methods 0.000 description 1
- 230000001419 dependent effect Effects 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 210000004247 hand Anatomy 0.000 description 1
- 238000010801 machine learning Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000001737 promoting effect Effects 0.000 description 1
- 230000009466 transformation Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/70—Determining position or orientation of objects or cameras
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20084—Artificial neural networks [ANN]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30196—Human being; Person
Landscapes
- Engineering & Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- User Interface Of Digital Computer (AREA)
Abstract
The disclosure provides a method and a device for determining an empty gesture position coordinate, which are used for acquiring an image of a target hand at a target moment and obtaining a joint point coordinate of the target hand according to the image; calculating to obtain the reference position coordinates of the target hand at the target moment according to the joint point coordinates; obtaining historical position coordinates of the target hand before the target time, and further calculating to obtain predicted position coordinates of the target hand at the target time; and calculating to obtain the determined position coordinate of the target hand at the target moment according to the reference position coordinate and the predicted position coordinate, and can be used for realizing accurate, smooth and stable continuous type spaced gesture control operation.
Description
Technical Field
The present disclosure relates to the field of human-computer interaction technologies, and in particular, to a method and an apparatus for determining an empty gesture position coordinate.
Background
With the development of intelligent interaction technology, the air-isolated gesture control technology is more and more applied to various fields, such as the fields of vehicle-mounted systems, smart homes, VR content interaction and the like. When the air gesture control system is used for continuous control (such as volume adjustment, cursor movement control and the like), the coordinate information of the 'operator' in each frame needs to be obtained, and the moving speed of the 'operator' is calculated, so that the change speed of the controlled object is determined.
However, the accuracy of the hand position directly obtained by the existing target detection technology is not sufficient, and if the hand position is directly mapped to the change speed of the controlled object, the change speed of the controlled object is not stable, and the existing gesture position coordinate detection technology cannot be used for realizing accurate, smooth and stable continuous type spaced gesture operation.
Disclosure of Invention
In view of the above, the present disclosure is directed to a method and an apparatus for determining an empty gesture position coordinate.
In view of the above, the present disclosure provides a method of determining an empty gesture position coordinate, comprising:
acquiring an image of a target hand at a target moment, and obtaining a joint point coordinate of the target hand according to the image;
calculating to obtain the reference position coordinates of the target hand at the target moment according to the joint point coordinates;
obtaining historical position coordinates of the target hand before the target time, and further calculating to obtain predicted position coordinates of the target hand at the target time;
and calculating to obtain the determined position coordinate of the target hand at the target moment according to the reference position coordinate and the predicted position coordinate.
Based on the same inventive concept, the present disclosure provides an apparatus for determining an empty gesture position coordinate, comprising:
the joint point coordinate acquisition module is configured to acquire an image of a target hand at a target moment and obtain joint point coordinates of the target hand according to the image;
a reference position coordinate acquisition module configured to calculate a reference position coordinate of the target hand at the target time according to the joint point coordinate;
a predicted position coordinate obtaining module configured to obtain a historical position coordinate of the target hand before the target time, and further calculate a predicted position coordinate of the target hand at the target time;
and the determined position coordinate acquisition module is configured to calculate the determined position coordinate of the target hand at the target moment according to the reference position coordinate and the predicted position coordinate.
Based on the same inventive concept, the present disclosure provides an electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, which when executing the program implements the method as described above.
Based on the same inventive concept, the present disclosure provides a non-transitory computer-readable storage medium storing computer instructions for causing a computer to perform the above-described method.
From the above description, the method and the device for determining the position coordinates of the potential of the spaced hand provided by the present disclosure acquire an image of the target hand at a target moment, and obtain the joint point coordinates of the target hand according to the image; calculating to obtain the reference position coordinates of the target hand at the target moment according to the joint point coordinates; obtaining historical position coordinates of the target hand before the target time, and further calculating to obtain predicted position coordinates of the target hand at the target time; and calculating to obtain the determined position coordinate of the target hand at the target moment according to the reference position coordinate and the predicted position coordinate, and can be used for realizing accurate, smooth and stable continuous type spaced gesture control operation.
Drawings
In order to more clearly illustrate the technical solutions in the present disclosure or related technologies, the drawings needed to be used in the description of the embodiments or related technologies are briefly introduced below, and it is obvious that the drawings in the following description are only embodiments of the present disclosure, and for those skilled in the art, other drawings can be obtained according to these drawings without creative efforts.
Fig. 1 is a schematic view of an application scenario of a method for determining an empty gesture position coordinate according to an embodiment of the present disclosure;
FIG. 2 is a schematic flow chart illustrating a method for determining spaced gesture position coordinates according to an embodiment of the present disclosure;
FIG. 3 is a schematic diagram of joint coordinates provided in accordance with an embodiment of the present disclosure;
FIG. 4 is a schematic diagram of joint numbering provided in accordance with an embodiment of the present disclosure;
FIG. 5 is a schematic structural diagram of an apparatus for determining a position coordinate of an empty gesture according to an embodiment of the present disclosure;
fig. 6 is a more specific hardware structure diagram of an electronic device according to an embodiment of the present disclosure.
Detailed Description
For the purpose of promoting a better understanding of the objects, aspects and advantages of the present disclosure, reference is made to the following detailed description taken in conjunction with the accompanying drawings.
It is to be noted that technical terms or scientific terms used in the embodiments of the present disclosure should have a general meaning as understood by those having ordinary skill in the art to which the present disclosure belongs, unless otherwise defined. The use of "first," "second," and similar terms in the embodiments of the disclosure is not intended to indicate any order, quantity, or importance, but rather to distinguish one element from another. The word "comprising" or "comprises", and the like, means that the element or item listed before the word covers the element or item listed after the word and its equivalents, but does not exclude other elements or items. The terms "connected" or "coupled" and the like are not restricted to physical or mechanical connections, but may include electrical connections, whether direct or indirect. "upper", "lower", "left", "right", and the like are used merely to indicate relative positional relationships, and when the absolute position of the object being described is changed, the relative positional relationships may also be changed accordingly.
Reference is made to fig. 1, which is a schematic diagram illustrating an application scenario of a method for determining a position coordinate of an empty gesture according to an embodiment of the present disclosure. The application scenario includes a terminal device 101, a server 102, and a data storage system 103. The terminal device 101, the server 102, and the data storage system 103 may be connected through a wired or wireless communication network. The terminal device 101 includes, but is not limited to, a desktop computer, a mobile phone, a mobile computer, a tablet computer, a media player, a smart wearable device, a Personal Digital Assistant (PDA), or other electronic devices capable of implementing the above functions. The server 102 and the data storage system 103 may be independent physical servers, may also be a server cluster or distributed system formed by a plurality of physical servers, and may also be cloud servers providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDNs, and big data and artificial intelligence platforms.
The server 102 is used for providing a service for determining the position coordinates of the spaced gesture for a user of the terminal device 101, the terminal device 101 can communicate with the server 102, the terminal device 101 obtains a gesture image of the user and sends the gesture image of the user to the server 102, the server 102 processes the gesture image of the user to obtain the position coordinates of the gesture of the user at a target moment, and therefore the terminal device 101 is applied to a spaced gesture control scene, such as a vehicle-mounted system, smart home, VR content interaction and other scenes.
The data storage system 103 stores a large amount of training data, the sources of which include, but are not limited to, existing databases, data crawled from the internet, or data uploaded while the user is using the client. Server 102 may also continually optimize the service of determining the coordinates of the spaced gesture locations based on additional training data.
The method for determining the position coordinates of the spaced gestures in the embodiment of the disclosure can be applied to the related fields of spaced gesture control, such as the fields of vehicle-mounted systems, smart homes, VR content interaction and the like.
A method of determining the spaced gesture position coordinates according to an exemplary embodiment of the present disclosure is described below in conjunction with the application scenario of fig. 1. It should be noted that the above application scenarios are merely illustrated for the convenience of understanding the spirit and principles of the present disclosure, and the embodiments of the present disclosure are not limited in this respect. Rather, embodiments of the present disclosure may be applied to any scenario where applicable.
FIG. 2 is a schematic flow chart illustrating a method for determining spaced gesture position coordinates according to an embodiment of the present disclosure; a method of determining spaced gesture position coordinates, comprising:
s210, obtaining an image of the target hand at the target moment, and obtaining the joint point coordinates of the target hand according to the image.
The utility model discloses be applied to the relevant field of air gesture control, realize that the equipment of air gesture control is external usually or built-in to have the camera, equipment passes through the camera acquires the image that contains target hand (or called operative hand) in real time for realize air gesture control.
In some embodiments, the acquired image including the target hand includes multiple hands, in which case the image needs to be preprocessed to obtain an image including only one and the main hand, i.e., the target hand.
In some embodiments, the acquired image including the target hand further includes other human body parts, such as an arm, a torso, and the like, in which case, the image needs to be preprocessed to obtain an image including only the target hand.
In some embodiments, the coordinates of the target hand's joints are derived from the image using techniques available in the related art for image processing, detection and recognition. The coordinates of the joint points obtained by the recognition are shown in fig. 3. As an example, an articulation point detection model is constructed based on a deep neural network model (DNN) framework, and a method of constructing the articulation point detection model includes:
constructing a sample set comprising a plurality of samples; wherein the sample comprises: sample data and tag data; the sample data comprises a target hand image for training; the label data comprises target hand joint point coordinates corresponding to the training target hand image;
and constructing and training the joint point detection model according to the sample set through a preset machine learning algorithm such as a deep neural network.
The training target hand image and the target hand joint point coordinates corresponding to the training target hand image can be obtained from historical case data. It should be noted that the present disclosure does not relate to an improvement to the deep neural network model.
In some embodiments, each joint point corresponds to a unique sequence number.
Referring to fig. 4, which is a schematic diagram of joint numbers provided according to an embodiment of the present disclosure, as an example, where 1, 2, 3, and 4 are numbers of four joint points from a finger root to a fingertip, and numbers of other joint points are shown in fig. 4, which are not described herein again, it should be noted that all joint point numbers used in the present disclosure refer to fig. 4, and are taken as an example.
And corresponding the obtained joint point coordinates with the serial numbers of the joint points so as to facilitate the search and utilization of the joint point coordinates. For example, the position coordinates of the joint point 5 are denoted as P5(x5，y5) Similarly, the position coordinate of the joint point 9 is denoted as P9(x9，y9) The position coordinates of the joint point 13 are denoted as P13(x13，y13)。
And S220, calculating the reference position coordinates of the target hand at the target moment according to the joint point coordinates.
Wherein, specifically include:
and determining joint point coordinates of which the relative positions are not easy to change compared with other joint point coordinates in the joint point coordinates as reference joint point coordinates, and calculating to obtain the reference position coordinates based on the reference joint point coordinates.
Wherein the calculating the reference position coordinates based on the reference joint point coordinates comprises:
and calculating to obtain an average value of the coordinates of the reference joint point, and taking the average value as the coordinates of the reference position. The calculation formula is as follows:
wherein, Ph0|kRepresenting reference position coordinates; pi1，Pi2，Pin-1，PinRepresenting the coordinates of the reference joint points; n represents the number of reference joint point coordinates; m represents the number of joint coordinates.
Wherein the reference joint point coordinates comprise:
and the joint point coordinates corresponding to the root of the index finger, the root of the middle finger, the root of the ring finger and the root of the little finger of the target hand.
Referring to fig. 3 and 4, in the process of gesture transformation and movement, the relative positions of five joint points 0, 5, 9, 13 and 17 among the 21 joint points of the target hand are less likely to change than other joint points, that is, the relative positions are more stable, and the four joint points 5, 9, 13 and 17 are located at the middle position of the whole hand, so that the reference position coordinates of the target hand can be preliminarily determined by comprehensive calculation with the four joint points as the reference. In this case, the reference position coordinates of the target hand are calculated based on the joint point coordinates corresponding to the root of the index finger, the root of the middle finger, the root of the ring finger, and the root of the little finger of the target hand, and the calculation formula is:
wherein, Ph0|kRepresenting reference position coordinates; p5，P9，P13，P17Sequentially and respectively representing the coordinates of the corresponding joint points of the root of the index finger, the root of the middle finger, the root of the ring finger and the root of the little finger.
S230, obtaining the historical position coordinates of the target hand before the target time, and further calculating to obtain the predicted position coordinates of the target hand at the target time.
If the frequency of image acquisition is fast enough, the motion between two adjacent frames can be considered as uniform motion. Assuming that there are two times k-2 and k-1 before the target time (k time), the moving speed of the hand between k-2 and k-1 is recorded as vk-1(vk-1|x，vk-1|y) The movement speed v of the hand between the time k-1 and the time k is recordedk(vk|x，vk|y). Then there are: v. ofk＝vk-1。
Based on this, in some embodiments, S230 specifically includes:
acquiring the historical position coordinates of the target hand at a first time and a second time before the target time;
determining a time difference between the first time and the second time, and a time difference between the target time and the first time;
and regarding the motion of the target hand between two adjacent frames of images as uniform motion, and calculating to obtain the predicted position coordinate based on the historical position coordinate, the time difference between the first time and the second time and the time difference between the target time and the first time.
The first time and the second time are corresponding to two adjacent frames of images.
The calculation formula is as follows:
wherein the first time k-1 is after the second time k-2; ph1|kRepresenting predicted position coordinates; deltat(k|k-1)Representing a time difference between the target time and the first time; deltat(k-1|k-2)Representing a time difference between the first time and the second time; ph|k-1Representing historical position coordinates of the target hand at a first time; ph|k-2Representing the historical position coordinates of the target hand at the second time.
Specifically, the movement speed of the hand between the target time and the first time is
The movement speed of the hand between the first time and the second time is
Based on vk＝vk-1Then, there are:
and S240, calculating to obtain the determined position coordinate of the target hand at the target moment according to the reference position coordinate and the predicted position coordinate.
In some embodiments, S240 specifically includes:
and acquiring the reliability of the reference position coordinates and the reliability of the predicted position coordinates.
The gesture cannot completely accord with uniform motion in the actual motion process, so that the candidate position coordinate P is determined and calculatedh1|kThe confidence level of (b) is β.
And calculating the determined position coordinate of the target hand at the target moment based on the reference position coordinate, the predicted position coordinate, the reliability of the reference position coordinate and the reliability of the predicted position coordinate. The method specifically comprises the following steps: and taking a weighted average of the reference position coordinates and the predicted position coordinates as the determined position coordinates, wherein the reliability of the reference position coordinates and the reliability of the predicted position coordinates are respectively weights of the reference position coordinates and the predicted position coordinates.
The calculation formula is as follows:
wherein, Ph|kIndicating the determined location coordinates; α represents the reliability of the reference position coordinates; β represents the confidence level of the predicted position coordinates; ph0|kRepresenting reference position coordinates; ph1|kRepresenting the predicted position coordinates.
From the above description, the method and the device for determining the position coordinates of the potential of the spaced hand provided by the present disclosure acquire an image of the target hand at a target moment, and obtain the joint point coordinates of the target hand according to the image; calculating to obtain the reference position coordinates of the target hand at the target moment according to the joint point coordinates; obtaining historical position coordinates of the target hand before the target time, and further calculating to obtain predicted position coordinates of the target hand at the target time; and calculating to obtain the determined position coordinate of the target hand at the target moment according to the reference position coordinate and the predicted position coordinate, and can be used for realizing accurate, smooth and stable continuous type spaced gesture control operation.
It should be noted that the method of the embodiments of the present disclosure may be executed by a single device, such as a computer or a server. The method of the embodiment can also be applied to a distributed scene and completed by the mutual cooperation of a plurality of devices. In such a distributed scenario, one of the devices may only perform one or more steps of the method of the embodiments of the present disclosure, and the devices may interact with each other to complete the method.
It should be noted that the above describes some embodiments of the disclosure. Other embodiments are within the scope of the following claims. In some cases, the actions or steps recited in the claims may be performed in a different order than in the embodiments described above and still achieve desirable results. In addition, the processes depicted in the accompanying figures do not necessarily require the particular order shown, or sequential order, to achieve desirable results. In some embodiments, multitasking and parallel processing may also be possible or may be advantageous.
Based on the same inventive concept, corresponding to the method of any embodiment, the disclosure further provides a device for determining the position coordinates of the spaced gesture.
Referring to fig. 5, the apparatus for determining an empty hand position coordinate includes:
a joint point coordinate obtaining module 510 configured to obtain an image of a target hand at a target time, and obtain a joint point coordinate of the target hand according to the image;
a reference position coordinate obtaining module 520, configured to calculate, according to the joint coordinates, a reference position coordinate of the target hand at the target time;
a predicted position coordinate obtaining module 530 configured to obtain a historical position coordinate of the target hand before the target time, and further calculate a predicted position coordinate of the target hand at the target time;
a determined position coordinate obtaining module 540 configured to calculate a determined position coordinate of the target hand at the target time according to the reference position coordinate and the predicted position coordinate.
In some embodiments, the reference location coordinate acquisition module 520 is specifically configured to:
and determining joint point coordinates of which the relative positions are not easy to change compared with other joint point coordinates in the joint point coordinates as reference joint point coordinates, and calculating to obtain the reference position coordinates based on the reference joint point coordinates.
In some embodiments, the reference location coordinate acquisition module 520 is specifically configured to:
and calculating to obtain an average value of the coordinates of the reference joint point, and taking the average value as the coordinates of the reference position. The calculation formula is as follows:
wherein, Ph0|kRepresenting reference position coordinates; pi1，Pi2，Pin-1，PinRepresenting the coordinates of the reference joint points; n represents the number of reference joint point coordinates; m represents the number of joint coordinates.
Wherein the reference joint point coordinates comprise:
and the joint point coordinates corresponding to the root of the index finger, the root of the middle finger, the root of the ring finger and the root of the little finger of the target hand.
In some embodiments, the predicted location coordinate acquisition module 530 is specifically configured to:
acquiring the historical position coordinates of the target hand at a first time and a second time before the target time;
determining a time difference between the first time and the second time, and a time difference between the target time and the first time;
and regarding the motion of the target hand between two adjacent frames of images as uniform motion, and calculating to obtain the predicted position coordinate based on the historical position coordinate, the time difference between the first time and the second time and the time difference between the target time and the first time. The calculation formula is as follows:
wherein the first time is after the second time; ph1|kRepresenting predicted position coordinates; deltat(k|k-1)Representing a time difference between the target time and the first time; deltat(k-1|k-2)Representing a time difference between the first time and the second time; ph|k-1Representing historical position coordinates of the target hand at a first time; ph|k-2Representing the historical position coordinates of the target hand at the second time.
In some embodiments, the determined location coordinates acquisition module 540 is specifically configured to:
obtaining the reliability of the reference position coordinates and the reliability of the predicted position coordinates;
and calculating the determined position coordinate of the target hand at the target moment based on the reference position coordinate, the predicted position coordinate, the reliability of the reference position coordinate and the reliability of the predicted position coordinate. The method specifically comprises the following steps: and taking a weighted average of the reference position coordinates and the predicted position coordinates as the determined position coordinates, wherein the reliability of the reference position coordinates and the reliability of the predicted position coordinates are respectively weights of the reference position coordinates and the predicted position coordinates. The calculation formula is as follows:
wherein, Ph|kIndicating the determined location coordinates; α represents the reliability of the reference position coordinates; β represents the confidence level of the predicted position coordinates; ph0|kRepresenting reference position coordinates; ph1|kRepresenting the predicted position coordinates.
For convenience of description, the above devices are described as being divided into various modules by functions, and are described separately. Of course, the functionality of the various modules may be implemented in the same one or more software and/or hardware implementations of the present disclosure.
The device of the foregoing embodiment is used to implement the method for determining the position coordinates of the spaced gesture in any of the foregoing embodiments, and has the beneficial effects of the corresponding method embodiment, which are not described herein again.
Based on the same inventive concept, corresponding to the method of any embodiment described above, the present disclosure further provides an electronic device, including a memory, a processor, and a computer program stored on the memory and executable on the processor, where the processor executes the program to implement the method for determining the position coordinates of the spaced gesture according to any embodiment described above.
Fig. 6 is a schematic diagram illustrating a more specific hardware structure of an electronic device according to this embodiment, where the electronic device may include: a processor 1010, a memory 1020, an input/output interface 1030, a communication interface 1040, and a bus 1050. Wherein the processor 1010, memory 1020, input/output interface 1030, and communication interface 1040 are communicatively coupled to each other within the device via bus 1050.
The processor 1010 may be implemented by a general-purpose CPU (Central Processing Unit), a microprocessor, an Application Specific Integrated Circuit (ASIC), or one or more Integrated circuits, and is configured to execute related programs to implement the technical solutions provided in the embodiments of the present disclosure.
The Memory 1020 may be implemented in the form of a ROM (Read Only Memory), a RAM (random access Memory), a static storage device, a dynamic storage device, or the like. The memory 1020 may store an operating system and other application programs, and when the technical solution provided by the embodiments of the present specification is implemented by software or firmware, the relevant program codes are stored in the memory 1020 and called to be executed by the processor 1010.
The input/output interface 1030 is used for connecting an input/output module to input and output information. The input/output module may be configured as a component in a device (not shown) or may be external to the device to provide a corresponding function. The input devices may include a keyboard, a mouse, a touch screen, a microphone, various sensors, etc., and the output devices may include a display, a speaker, a vibrator, an indicator light, etc.
The communication interface 1040 is used for connecting a communication module (not shown in the drawings) to implement communication interaction between the present apparatus and other apparatuses. The communication module can realize communication in a wired mode (such as USB, network cable and the like) and also can realize communication in a wireless mode (such as mobile network, WIFI, Bluetooth and the like).
It should be noted that although the above-mentioned device only shows the processor 1010, the memory 1020, the input/output interface 1030, the communication interface 1040 and the bus 1050, in a specific implementation, the device may also include other components necessary for normal operation. In addition, those skilled in the art will appreciate that the above-described apparatus may also include only those components necessary to implement the embodiments of the present description, and not necessarily all of the components shown in the figures.
The electronic device of the foregoing embodiment is used to implement the method for determining the position coordinates of the spaced gesture in any one of the foregoing embodiments, and has the beneficial effects of the corresponding method embodiment, which are not described herein again.
Based on the same inventive concept, corresponding to any of the above-described embodiment methods, the present disclosure also provides a non-transitory computer-readable storage medium storing computer instructions for causing the computer to perform the method of determining an empty gesture position coordinate as described in any of the above embodiments.
Computer-readable media of the present embodiments, including both non-transitory and non-transitory, removable and non-removable media, may implement information storage by any method or technology. The information may be computer readable instructions, data structures, modules of a program, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), Static Random Access Memory (SRAM), Dynamic Random Access Memory (DRAM), other types of Random Access Memory (RAM), Read Only Memory (ROM), Electrically Erasable Programmable Read Only Memory (EEPROM), flash memory or other memory technology, compact disc read only memory (CD-ROM), Digital Versatile Discs (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices, or any other non-transmission medium that can be used to store information that can be accessed by a computing device.
The computer instructions stored in the storage medium of the above embodiment are used to enable the computer to execute the method for determining the coordinates of the spaced gesture positions according to any of the above embodiments, and have the beneficial effects of corresponding method embodiments, and will not be described herein again.
It should be noted that the embodiments of the present disclosure can be further described in the following ways:
a method of determining spaced gesture position coordinates, comprising:
acquiring an image of a target hand at a target moment, and obtaining a joint point coordinate of the target hand according to the image;
calculating to obtain the reference position coordinates of the target hand at the target moment according to the joint point coordinates;
obtaining historical position coordinates of the target hand before the target time, and further calculating to obtain predicted position coordinates of the target hand at the target time;
and calculating to obtain the determined position coordinate of the target hand at the target moment according to the reference position coordinate and the predicted position coordinate.
Optionally, the calculating, according to the joint point coordinates, reference position coordinates of the target hand at the target time includes:
and determining joint point coordinates of which the relative positions are not easy to change compared with other joint point coordinates in the joint point coordinates as reference joint point coordinates, and calculating to obtain the reference position coordinates based on the reference joint point coordinates.
Optionally, wherein the calculating the reference position coordinates based on the coordinates of the reference joint point includes:
and calculating to obtain an average value of the coordinates of the reference joint point, and taking the average value as the coordinates of the reference position.
Optionally, wherein the reference joint point coordinates include:
and the joint point coordinates corresponding to the root of the index finger, the root of the middle finger, the root of the ring finger and the root of the little finger of the target hand.
Optionally, the obtaining the historical position coordinates of the target hand before the target time and further calculating to obtain the predicted position coordinates of the target hand at the target time include:
acquiring the historical position coordinates of the target hand at a first time and a second time before the target time;
determining a time difference between the first time and the second time, and a time difference between the target time and the first time;
and regarding the motion of the target hand between two adjacent frames of images as uniform motion, and calculating to obtain the predicted position coordinate based on the historical position coordinate, the time difference between the first time and the second time and the time difference between the target time and the first time.
Optionally, the calculating, according to the reference position coordinate and the predicted position coordinate, to obtain a determined position coordinate of the target hand at the target time includes:
obtaining the reliability of the reference position coordinates and the reliability of the predicted position coordinates;
calculating to obtain the determined position coordinate of the target hand at the target moment based on the reference position coordinate, the predicted position coordinate, the reliability of the reference position coordinate and the reliability of the predicted position coordinate; the method specifically comprises the following steps: and taking a weighted average of the reference position coordinates and the predicted position coordinates as the determined position coordinates, wherein the reliability of the reference position coordinates and the reliability of the predicted position coordinates are respectively weights of the reference position coordinates and the predicted position coordinates.
An apparatus for determining an open hand position coordinate, comprising:
the joint point coordinate acquisition module is configured to acquire an image of a target hand at a target moment and obtain joint point coordinates of the target hand according to the image;
a reference position coordinate acquisition module configured to calculate a reference position coordinate of the target hand at the target time according to the joint point coordinate;
a predicted position coordinate obtaining module configured to obtain a historical position coordinate of the target hand before the target time, and further calculate a predicted position coordinate of the target hand at the target time;
and the determined position coordinate acquisition module is configured to calculate the determined position coordinate of the target hand at the target moment according to the reference position coordinate and the predicted position coordinate.
Optionally, the determining position coordinate obtaining module is specifically configured to:
obtaining the reliability of the reference position coordinates and the reliability of the predicted position coordinates;
and calculating the determined position coordinate of the target hand at the target moment based on the reference position coordinate, the predicted position coordinate, the reliability of the reference position coordinate and the reliability of the predicted position coordinate.
An electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, the processor implementing the method as described above when executing the program.
A non-transitory computer readable storage medium storing computer instructions for causing a computer to perform the above method.
Those of ordinary skill in the art will understand that: the discussion of any embodiment above is meant to be exemplary only, and is not intended to intimate that the scope of the disclosure, including the claims, is limited to these examples; within the idea of the present disclosure, also technical features in the above embodiments or in different embodiments may be combined, steps may be implemented in any order, and there are many other variations of the different aspects of the embodiments of the present disclosure as described above, which are not provided in detail for the sake of brevity.
In addition, well-known power/ground connections to Integrated Circuit (IC) chips and other components may or may not be shown in the provided figures for simplicity of illustration and discussion, and so as not to obscure the embodiments of the disclosure. Furthermore, devices may be shown in block diagram form in order to avoid obscuring embodiments of the present disclosure, and this also takes into account the fact that specifics with respect to implementation of such block diagram devices are highly dependent upon the platform within which the embodiments of the present disclosure are to be implemented (i.e., specifics should be well within purview of one skilled in the art). Where specific details (e.g., circuits) are set forth in order to describe example embodiments of the disclosure, it should be apparent to one skilled in the art that the embodiments of the disclosure can be practiced without, or with variation of, these specific details. Accordingly, the description is to be regarded as illustrative instead of restrictive.
While the present disclosure has been described in conjunction with specific embodiments thereof, many alternatives, modifications, and variations of these embodiments will be apparent to those of ordinary skill in the art in light of the foregoing description. For example, other memory architectures (e.g., dynamic ram (dram)) may use the discussed embodiments.
The disclosed embodiments are intended to embrace all such alternatives, modifications and variances which fall within the broad scope of the appended claims. Therefore, any omissions, modifications, equivalents, improvements, and the like that may be made within the spirit and principles of the embodiments of the disclosure are intended to be included within the scope of the disclosure.
Claims (10)
1. A method of determining spaced gesture position coordinates, comprising:
acquiring an image of a target hand at a target moment, and obtaining a joint point coordinate of the target hand according to the image;
calculating to obtain the reference position coordinates of the target hand at the target moment according to the joint point coordinates;
obtaining historical position coordinates of the target hand before the target time, and further calculating to obtain predicted position coordinates of the target hand at the target time;
and calculating to obtain the determined position coordinate of the target hand at the target moment according to the reference position coordinate and the predicted position coordinate.
2. The method of claim 1, wherein the calculating the reference position coordinates of the target hand at the target time from the joint coordinates comprises:
and determining joint point coordinates of which the relative positions are not easy to change compared with other joint point coordinates in the joint point coordinates as reference joint point coordinates, and calculating to obtain the reference position coordinates based on the reference joint point coordinates.
3. The method of claim 2, wherein said calculating the reference position coordinates based on the reference joint point coordinates comprises:
and calculating to obtain an average value of the coordinates of the reference joint point, and taking the average value as the coordinates of the reference position.
4. The method of claim 2, wherein the reference joint coordinates comprise:
and the joint point coordinates corresponding to the root of the index finger, the root of the middle finger, the root of the ring finger and the root of the little finger of the target hand.
5. The method of claim 1, wherein obtaining historical position coordinates of the target hand prior to the target time and further calculating predicted position coordinates of the target hand at the target time comprises:
acquiring the historical position coordinates of the target hand at a first time and a second time before the target time;
determining a time difference between the first time and the second time, and a time difference between the target time and the first time;
and regarding the motion of the target hand between two adjacent frames of images as uniform motion, and calculating to obtain the predicted position coordinate based on the historical position coordinate, the time difference between the first time and the second time and the time difference between the target time and the first time.
6. The method of claim 1, wherein calculating the determined position coordinates of the target hand at the target time based on the reference position coordinates and the predicted position coordinates comprises:
obtaining the reliability of the reference position coordinates and the reliability of the predicted position coordinates;
calculating to obtain the determined position coordinate of the target hand at the target moment based on the reference position coordinate, the predicted position coordinate, the reliability of the reference position coordinate and the reliability of the predicted position coordinate; the method specifically comprises the following steps: and taking a weighted average of the reference position coordinates and the predicted position coordinates as the determined position coordinates, wherein the reliability of the reference position coordinates and the reliability of the predicted position coordinates are respectively weights of the reference position coordinates and the predicted position coordinates.
7. An apparatus for determining spaced gesture position coordinates, comprising:
the joint point coordinate acquisition module is configured to acquire an image of a target hand at a target moment and obtain joint point coordinates of the target hand according to the image;
a reference position coordinate acquisition module configured to calculate a reference position coordinate of the target hand at the target time according to the joint point coordinate;
a predicted position coordinate obtaining module configured to obtain a historical position coordinate of the target hand before the target time, and further calculate a predicted position coordinate of the target hand at the target time;
and the determined position coordinate acquisition module is configured to calculate the determined position coordinate of the target hand at the target moment according to the reference position coordinate and the predicted position coordinate.
8. The apparatus according to claim 7, wherein the determined position coordinate acquisition module is specifically configured to:
obtaining the reliability of the reference position coordinates and the reliability of the predicted position coordinates;
calculating to obtain the determined position coordinate of the target hand at the target moment based on the reference position coordinate, the predicted position coordinate, the reliability of the reference position coordinate and the reliability of the predicted position coordinate; the method specifically comprises the following steps: and taking a weighted average of the reference position coordinates and the predicted position coordinates as the determined position coordinates, wherein the reliability of the reference position coordinates and the reliability of the predicted position coordinates are respectively weights of the reference position coordinates and the predicted position coordinates.
9. An electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, the processor implementing the method of any one of claims 1 to 6 when executing the program.
10. A non-transitory computer readable storage medium storing computer instructions for causing a computer to perform the method of any one of claims 1 to 6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110925230.9A CN113706606B (en) | 2021-08-12 | 2021-08-12 | Method and device for determining position coordinates of spaced hand gestures |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110925230.9A CN113706606B (en) | 2021-08-12 | 2021-08-12 | Method and device for determining position coordinates of spaced hand gestures |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113706606A true CN113706606A (en) | 2021-11-26 |
| CN113706606B CN113706606B (en) | 2024-04-30 |
Family
ID=78652473
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110925230.9A Active CN113706606B (en) | 2021-08-12 | 2021-08-12 | Method and device for determining position coordinates of spaced hand gestures |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113706606B (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2023189838A1 (en) * | 2022-04-01 | 2023-10-05 | ソニーグループ株式会社 | Information processing device, information processing method, and program |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104992171A (en) * | 2015-08-04 | 2015-10-21 | 易视腾科技有限公司 | Method and system for gesture recognition and man-machine interaction based on 2D video sequence |
| US20190179417A1 (en) * | 2017-12-11 | 2019-06-13 | Shenzhen Starfield Information Technologies Co., Ltd. | 3D Interaction Method, Device, Computer Equipment and Storage Medium |
| CN110443154A (en) * | 2019-07-15 | 2019-11-12 | 北京达佳互联信息技术有限公司 | Three-dimensional coordinate localization method, device, electronic equipment and the storage medium of key point |
| CN111414837A (en) * | 2020-03-16 | 2020-07-14 | 苏州交驰人工智能研究院有限公司 | Gesture recognition method and device, computer equipment and storage medium |
| CN112486326A (en) * | 2020-12-14 | 2021-03-12 | 安徽鸿程光电有限公司 | Position coordinate prediction method for gesture control movement, intelligent terminal and medium |
| US20210097270A1 (en) * | 2018-10-30 | 2021-04-01 | Beijing Dajia Internet Information Technology Co., Ltd. | Method and device for detecting hand gesture key points |
-
2021
- 2021-08-12 CN CN202110925230.9A patent/CN113706606B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104992171A (en) * | 2015-08-04 | 2015-10-21 | 易视腾科技有限公司 | Method and system for gesture recognition and man-machine interaction based on 2D video sequence |
| US20190179417A1 (en) * | 2017-12-11 | 2019-06-13 | Shenzhen Starfield Information Technologies Co., Ltd. | 3D Interaction Method, Device, Computer Equipment and Storage Medium |
| US20210097270A1 (en) * | 2018-10-30 | 2021-04-01 | Beijing Dajia Internet Information Technology Co., Ltd. | Method and device for detecting hand gesture key points |
| CN110443154A (en) * | 2019-07-15 | 2019-11-12 | 北京达佳互联信息技术有限公司 | Three-dimensional coordinate localization method, device, electronic equipment and the storage medium of key point |
| CN111414837A (en) * | 2020-03-16 | 2020-07-14 | 苏州交驰人工智能研究院有限公司 | Gesture recognition method and device, computer equipment and storage medium |
| CN112486326A (en) * | 2020-12-14 | 2021-03-12 | 安徽鸿程光电有限公司 | Position coordinate prediction method for gesture control movement, intelligent terminal and medium |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2023189838A1 (en) * | 2022-04-01 | 2023-10-05 | ソニーグループ株式会社 | Information processing device, information processing method, and program |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113706606B (en) | 2024-04-30 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US10916021B2 (en) | Visual target tracking method and apparatus based on deeply and densely connected neural network | |
| CN104350509B (en) | Quick attitude detector | |
| CN110189246B (en) | Image stylization generation method and device and electronic equipment | |
| CN110457414A (en) | Offline map processing, virtual objects display methods, device, medium and equipment | |
| EP3757817A1 (en) | Electronic device and control method therefor | |
| CN109902588B (en) | Gesture recognition method and device and computer readable storage medium | |
| CN111598149B (en) | Loop detection method based on attention mechanism | |
| CN111796925A (en) | Method and device for screening algorithm model, storage medium and electronic equipment | |
| CN110956131B (en) | Single-target tracking method, device and system | |
| CN111199169A (en) | Image processing method and device | |
| CN113706606B (en) | Method and device for determining position coordinates of spaced hand gestures | |
| CN111126358A (en) | Face detection method, face detection device, storage medium and equipment | |
| CN111160308A (en) | Gesture motion recognition method, device, equipment and readable storage medium | |
| CN111601129B (en) | Control method, control device, terminal and storage medium | |
| CN114202799A (en) | Method and device for determining change speed of controlled object, electronic equipment and storage medium | |
| CN110069126B (en) | Virtual object control method and device | |
| CN116934591A (en) | Image stitching method, device and equipment for multi-scale feature extraction and storage medium | |
| CN111258413A (en) | Control method and device of virtual object | |
| CN111797986A (en) | Data processing method, data processing device, storage medium and electronic equipment | |
| CN111797874A (en) | Behavior prediction method, behavior prediction device, storage medium and electronic equipment | |
| CN114049403A (en) | Multi-angle three-dimensional face reconstruction method and device and storage medium | |
| CN113223007A (en) | Visual odometer implementation method and device and electronic equipment | |
| CN111797869A (en) | Model training method and device, storage medium and electronic equipment | |
| CN111103967A (en) | Control method and device of virtual object | |
| US11836839B2 (en) | Method for generating animation figure, electronic device and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |