CN111282261A - Human-computer interaction method and device and motion sensing game equipment - Google Patents
Human-computer interaction method and device and motion sensing game equipment Download PDFInfo
- Publication number
- CN111282261A CN111282261A CN202010074602.7A CN202010074602A CN111282261A CN 111282261 A CN111282261 A CN 111282261A CN 202010074602 A CN202010074602 A CN 202010074602A CN 111282261 A CN111282261 A CN 111282261A
- Authority
- CN
- China
- Prior art keywords
- user
- image
- human
- motion
- computer interaction
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- A—HUMAN NECESSITIES
- A63—SPORTS; GAMES; AMUSEMENTS
- A63F—CARD, BOARD, OR ROULETTE GAMES; INDOOR GAMES USING SMALL MOVING PLAYING BODIES; VIDEO GAMES; GAMES NOT OTHERWISE PROVIDED FOR
- A63F13/00—Video games, i.e. games using an electronically generated display having two or more dimensions
- A63F13/20—Input arrangements for video game devices
- A63F13/21—Input arrangements for video game devices characterised by their sensors, purposes or types
-
- A—HUMAN NECESSITIES
- A63—SPORTS; GAMES; AMUSEMENTS
- A63F—CARD, BOARD, OR ROULETTE GAMES; INDOOR GAMES USING SMALL MOVING PLAYING BODIES; VIDEO GAMES; GAMES NOT OTHERWISE PROVIDED FOR
- A63F13/00—Video games, i.e. games using an electronically generated display having two or more dimensions
- A63F13/20—Input arrangements for video game devices
- A63F13/21—Input arrangements for video game devices characterised by their sensors, purposes or types
- A63F13/213—Input arrangements for video game devices characterised by their sensors, purposes or types comprising photodetecting means, e.g. cameras, photodiodes or infrared cells
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/20—Movements or behaviour, e.g. gesture recognition
- G06V40/23—Recognition of whole body movements, e.g. for sport training
-
- A—HUMAN NECESSITIES
- A63—SPORTS; GAMES; AMUSEMENTS
- A63F—CARD, BOARD, OR ROULETTE GAMES; INDOOR GAMES USING SMALL MOVING PLAYING BODIES; VIDEO GAMES; GAMES NOT OTHERWISE PROVIDED FOR
- A63F2300/00—Features of games using an electronically generated display having two or more dimensions, e.g. on a television screen, showing representations related to the game
- A63F2300/10—Features of games using an electronically generated display having two or more dimensions, e.g. on a television screen, showing representations related to the game characterized by input arrangements for converting player-generated signals into game device control signals
- A63F2300/1087—Features of games using an electronically generated display having two or more dimensions, e.g. on a television screen, showing representations related to the game characterized by input arrangements for converting player-generated signals into game device control signals comprising photodetecting means, e.g. a camera
- A63F2300/1093—Features of games using an electronically generated display having two or more dimensions, e.g. on a television screen, showing representations related to the game characterized by input arrangements for converting player-generated signals into game device control signals comprising photodetecting means, e.g. a camera using visible light
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Human Computer Interaction (AREA)
- Health & Medical Sciences (AREA)
- Computer Vision & Pattern Recognition (AREA)
- General Health & Medical Sciences (AREA)
- Psychiatry (AREA)
- Social Psychology (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Image Analysis (AREA)
Abstract
The invention discloses a man-machine interaction method and device and a motion sensing game device. One embodiment of the method comprises: acquiring an image of a user, and detecting a motion track of a target in an acquisition field of the image in real time by using at least one ultrasonic transducer array; performing image segmentation on the image of the user to extract a region of at least one part of the user's body in the image; and corresponding the target to the part according to the position of the target to obtain the motion track of the part of the body of the user, and obtaining a control instruction based on the motion track of the part of the body of the user. According to the embodiment, on the basis of ensuring the body sensing identification precision, the real-time performance of body sensing identification is improved, and therefore the user experience is improved.
Description
Technical Field
The invention relates to the technical field of human-computer interaction. More particularly, the invention relates to a man-machine interaction method and device and a motion sensing game device.
Background
At present, in electronic devices based on a motion sensing technology, such as motion sensing game devices and the like, a 3D imaging technology for realizing motion sensing human-computer interaction is mainly applied to a Kinect motion sensing technology, and the principle of the Kinect motion sensing technology is a skeleton tracking technology.
The method for realizing somatosensory human-computer interaction can accurately realize somatosensory recognition, but a depth camera is required to collect depth images of a user in real time (for example, continuously collect the depth images at the frequency of 2 kHz), each frame of depth image is required to be stored, and each frame of depth image is subjected to image segmentation and human skeleton system model establishment and storage, wherein each frame of depth image contains tens of thousands of data information, the calculated and stored data amount is large, the calculation speed is low, the somatosensory recognition efficiency is low, certain time delay is caused, and the user experience is influenced.
Therefore, a new human-computer interaction method and device and a motion sensing game device need to be provided.
Disclosure of Invention
The invention aims to provide a man-machine interaction method and device and a motion sensing game device, and aims to solve at least one of the problems in the prior art.
In order to achieve the purpose, the invention adopts the following technical scheme:
the invention provides a man-machine interaction method in a first aspect, which comprises the following steps:
acquiring an image of a user, and detecting a motion track of a target in an acquisition field of the image in real time by using at least one ultrasonic transducer array;
performing image segmentation on the image of the user to extract a region of at least one part of the user's body in the image;
and corresponding the target to the part according to the position of the target to obtain the motion track of the part of the body of the user, and obtaining a control instruction based on the motion track of the part of the body of the user.
According to the man-machine interaction method provided by the first aspect of the invention, the somatosensory interaction in the three-dimensional space is realized based on a mode of combining optical imaging and ultrasonic detection, and the high-resolution advantage of optical imaging and the high-efficiency and low-cost advantage of ultrasonic detection of the target motion track are organically combined, so that on the basis of ensuring the somatosensory identification precision, the calculated data amount is reduced, the storage space and the calculation resources are saved, the calculation speed is increased, the real-time performance of somatosensory identification can be greatly improved, and the game experience of a user is greatly improved.
Optionally, the method further comprises: and judging whether a part moves according to the motion track of the part of the body of the user, and if so, acquiring the image of the user.
According to the optional mode, after the image of the initial frame user is acquired at the initial moment, the image of the user does not need to be continuously acquired, but the target and the middle part of the image of the initial frame user correspond to each other according to the position of the target obtained by real-time detection to obtain the motion track of the part of the body of the user, and when the part is judged to move, the image of the next frame user is acquired again to continue to acquire the subsequent control instruction, so that the situation that the same/similar images are repeatedly acquired when the part of the user does not move is avoided, the calculated data amount can be further reduced, the storage space and the calculation resource are saved, the calculation speed is increased, and the real-time performance of somatosensory recognition is improved.
Optionally, the method further comprises: images of the user are acquired at a preset frequency.
By adopting the optional mode, the accuracy of the corresponding of the target and the part can be ensured.
Optionally, the method further comprises: and judging whether a part moves according to the motion track of the part of the user body, and if so, storing the acquired image of the user.
This alternative may relieve the storage space from stress when capturing images of the user at a higher preset frequency.
Optionally, the method further comprises: and interpolating the image of the user according to the motion track of the part of the body of the user to fit to obtain an image between the image of the user and the image of the next frame of user, so as to form a dynamic image of the user.
According to the optional mode, the motion of the motion part is completely supplemented according to the motion track of the part of the body of the user, so that a complete dynamic image of the user is obtained through integration, and 3D dynamic imaging can be realized.
Optionally, the detecting the motion track of the target in the image acquisition region in real time by using at least one ultrasonic transducer array further comprises:
the method comprises the steps of equally dividing an image into a plurality of sub-areas, and detecting the motion tracks of targets in the acquisition visual field of the sub-areas in a one-to-one corresponding mode by utilizing a plurality of ultrasonic transducer arrays.
By adopting the optional mode, the precision of the motion trail of the ultrasonic detection target can be ensured, so that the motion sensing recognition precision is further improved.
Optionally, the method further comprises: and judging whether a part moves according to the motion track of the part of the body of the user, and if so, acquiring the image of the user.
Optionally, the method further comprises: images of the user are acquired at a preset frequency.
Optionally, the method further comprises: and judging whether a part moves according to the motion track of the part of the user body, and if so, storing a sub-region corresponding to the part with the motion in the acquired image of the user.
This alternative may further relieve the storage space from stress when capturing images of the user at a higher preset frequency.
Optionally, the method further comprises: and interpolating a subregion corresponding to the part with motion according to the motion track of the part of the body of the user to fit to obtain a subregion corresponding to the part with motion between the image of the user and the image of the next frame of user, and splicing the subregion corresponding to the part with motion between the image of the user and the image of the next frame of user and other subregions except for the subregion corresponding to the part with motion in the image of the user to obtain an image between the image of the user and the image of the next frame of user to form a dynamic image of the user.
By adopting the optional mode, on one hand, the precision of image fitting through interpolation can be further improved, so that a dynamic image of a user with higher reduction degree is formed, on the other hand, compared with a mode of fitting the whole image to obtain an intermediate image, the mode of only performing image fitting through interpolation on the subareas corresponding to the small-range moving parts and then splicing with the static subareas to obtain the intermediate image can further reduce the calculated data amount, save the calculation resources and improve the calculation speed, so that the real-time performance of motion sensing identification is further improved.
Optionally, the method further comprises: each ultrasonic transducer array is configured to detect with ultrasonic waves having a frequency that is different from the other ultrasonic transducer arrays.
By adopting the optional mode, the one-to-one correspondence between the ultrasonic transducer arrays and the sub-areas can be further ensured when each ultrasonic transducer array detects the motion track of the target in the acquisition view of the corresponding sub-area, the sound wave interference of other ultrasonic transducer arrays is avoided, and the precision of the motion track of the ultrasonic detection target is further ensured.
A second aspect of the present invention provides a human-computer interaction device for executing the human-computer interaction method provided by the first aspect of the present invention, including: the system comprises a processor, an image acquisition module and at least one ultrasonic transducer array;
the image acquisition module is used for acquiring images of users;
the ultrasonic transducer array is used for detecting the motion track of a target in the acquisition field of view of the image in real time;
the processor is used for carrying out image segmentation on the image of the user so as to extract a region of at least one part of the body of the user in the image; and corresponding the target to the part according to the position of the target to obtain the motion track of the part of the body of the user, and obtaining a control instruction based on the motion track of the part of the body of the user.
The human-computer interaction device provided by the second aspect of the invention realizes three-dimensional somatosensory interaction based on a mode of combining optical imaging and ultrasonic detection, and organically combines the high-resolution advantage of optical imaging and the high-efficiency and low-cost advantage of ultrasonic detection of the target motion track, so that on the basis of ensuring the somatosensory identification precision, the calculated data amount is reduced, the storage space and the calculation resources are saved, the calculation speed is increased, the real-time performance of somatosensory identification can be greatly improved, and the game experience of users is greatly improved.
Optionally, the ultrasonic transducer array includes a transmitting transducer array composed of a plurality of transmitting transducers and a receiving transducer array composed of a plurality of receiving transducers, the transmitting transducers in one ultrasonic transducer array are used for transmitting ultrasonic waves with different frequencies from other ultrasonic transducer arrays, and the receiving transducers are used for carrying out frequency-based filtering on the received ultrasonic waves so as to only reserve the ultrasonic waves transmitted by the transmitting transducers belonging to the same ultrasonic transducer array.
By adopting the optional mode, the one-to-one correspondence between the ultrasonic transducer arrays and the sub-areas can be further ensured when each ultrasonic transducer array detects the motion track of the target in the acquisition view of the corresponding sub-area, the receiving transducer is prevented from being interfered by the sound wave of the ultrasonic wave emitted by the transmitting transducers in other ultrasonic transducer arrays, and the precision of the motion track of the target detected by the ultrasonic wave is further ensured.
Optionally, in an ultrasound transducer array, the size of the receiving transducer array is larger than the size of the transmitting transducer array.
By adopting the optional mode, the ultrasonic transducer array can be set to have directivity pointing to the acquisition field of the corresponding sub-region, and the higher directivity of the ultrasonic transducer array is formed under the condition that one ultrasonic transducer array detects the motion track of the target in the acquisition field of one sub-region in real time, so that the one-to-one correspondence between the ultrasonic transducer array and the sub-region can be formed when each ultrasonic transducer array detects the motion track of the target in the acquisition field of the corresponding sub-region, the ultrasonic transducer array is further prevented from being interfered by sound waves caused by other ultrasonic transducer arrays and forming the sound waves to other ultrasonic transducer arrays, and the precision of the motion track of the target detected by ultrasonic waves is further ensured.
The invention provides a body sensing game device in a third aspect, which comprises a display device and the human-computer interaction device provided by the second aspect of the invention.
The invention has the following beneficial effects:
according to the technical scheme, the somatosensory interaction in the three-dimensional space is realized based on a mode of combining optical imaging and ultrasonic detection, on the basis of ensuring the somatosensory identification precision, the calculated data volume is reduced, the storage space and the calculation resources are saved, the calculation speed is increased, and therefore the real-time performance of somatosensory identification is improved. The technical scheme of the invention is suitable for various scenes such as motion sensing games and the like, and can greatly improve the interactive experience of users.
Drawings
The following describes embodiments of the present invention in further detail with reference to the accompanying drawings;
fig. 1 shows a flowchart of a human-computer interaction method provided by an embodiment of the present invention.
Fig. 2 shows a schematic representation of an aliquot image represented by a display screen into a plurality of sub-regions.
Fig. 3 shows a schematic representation of an ultrasound transducer array corresponding to an acquisition field of view of a subregion.
Fig. 4 is a schematic diagram illustrating an arrangement of a transmitting transducer array and a receiving transducer array in an ultrasonic transducer array.
Detailed Description
In order to more clearly illustrate the invention, the invention is further described below with reference to preferred embodiments and the accompanying drawings. Similar parts in the figures are denoted by the same reference numerals. It is to be understood by persons skilled in the art that the following detailed description is illustrative and not restrictive, and is not to be taken as limiting the scope of the invention.
As shown in fig. 1, an embodiment of the present invention provides a human-computer interaction method, including:
acquiring an image of a user and detecting (e.g., detecting at a frequency of 8 kHz) in real time a motion trajectory of an object in an acquisition field of view of the image using at least one ultrasonic transducer array;
performing image segmentation on the image of the user to extract a region of at least one part of the user's body in the image, in a specific example, the extracted region of the user's body part includes a region of the user's head, a region of the left hand, a region of the right hand, a region of the left foot, and a region of the right foot;
and corresponding the target to the part according to the position of the target to obtain the motion track of the part of the body of the user, and obtaining a control instruction based on the motion track of the part of the body of the user.
The ultrasonic technology can realize the identification and tracking of targets in a three-dimensional space, and the basic principle is an echo ranging principle: the transmitting transducer transmits ultrasonic waves, the ultrasonic waves are reflected after meeting a target and are received by the receiving transducer, the position of the target can be determined by utilizing the propagation time of the ultrasonic waves, and when more than two receiving transducers are adopted, the position and the distance of the target can be determined according to the time difference of signals received by the two receiving transducers. However, the resolution of the ultrasonic technology for realizing three-dimensional space detection is not high, the transverse resolution is centimeter-level, and the longitudinal resolution is ten centimeter-level, so that a contour image is difficult to form, and the hand, the head and other parts of the body of the user cannot be accurately segmented. Based on the principle, the man-machine interaction method provided by the embodiment realizes three-dimensional somatosensory interaction based on a mode of combining optical imaging and ultrasonic detection, and organically combines the high-resolution advantage of optical imaging and the high-efficiency and low-cost advantage of ultrasonic detection of the target motion track, so that on the basis of ensuring the somatosensory identification precision, the calculated data volume is reduced, the storage space and the calculation resources are saved, the calculation speed is increased, the real-time performance of somatosensory identification can be greatly improved, and the game experience of a user is greatly improved.
It should be noted that, the image of the user may be acquired by using a depth camera similar to the existing method, or may be acquired by using a common optical camera, because the motion trajectory is acquired by using ultrasonic detection in this embodiment, rather than using two adjacent frames of images, the acquired image of the user does not need to include depth information, and actually, the detection of the depth information is already included when the motion trajectory is acquired by ultrasonic detection.
In some optional implementation manners of the present embodiment, the human-computer interaction method provided by the present embodiment further includes: and judging whether a part moves according to the motion track of the part of the body of the user, and acquiring the image of the user or acquiring the image of the user again if the part moves.
The realization mode can acquire the image of the user in the initial frame at the initial moment without continuously acquiring the image of the user, but the target can be corresponded with the middle part of the image of the user in the initial frame according to the position of the target obtained by real-time detection to obtain the motion track of the part of the body of the user, and the image of the user in the next frame is collected to continue the subsequent control instruction acquisition when the part is judged to move (after the part moves and moves, the corresponding of the target and the part can not be carried out according to the region of the part in the image extracted by carrying out image segmentation on the image of the user in the initial frame, and the image of the user is collected again), thereby avoiding the repeated collection of the same/similar image when the part of the user does not move, the method can further reduce the calculated data quantity, save the storage space and the calculation resources, and improve the calculation speed, thereby improving the real-time performance of somatosensory recognition.
In some optional implementation manners of the present embodiment, the human-computer interaction method provided by the present embodiment further includes: images of the user are acquired at a preset frequency.
By adopting the implementation mode, the accuracy of the corresponding of the target and the part can be ensured. In one specific example, the preset frequency may be set to 1Hz, i.e. the user's image is acquired once per second, or may be set to 0.5Hz, etc. In addition, the step of acquiring the images of the user at the preset frequency can be executed simultaneously with the step of judging whether the part moves according to the movement track of the part of the body of the user, and if so, the step of acquiring the images of the user is executed simultaneously, so that the accuracy is further ensured.
Further, in some optional implementation manners of the present embodiment, the human-computer interaction method provided by the present embodiment further includes: and judging whether a part moves according to the motion track of the part of the user body, and if so, storing the acquired image of the user.
This implementation may relieve the storage space pressure when capturing images of the user at a higher preset frequency. For example, the preset frequency is set to 1kHz, in this case, it may be considered that images of the user are continuously acquired or acquired in real time, and at this time, if each frame of image is stored, a large storage space pressure may be brought, and this implementation may reduce the storage space pressure while ensuring the accuracy of the target corresponding to the location.
In some optional implementation manners of the present embodiment, the human-computer interaction method provided by the present embodiment further includes: and interpolating the image of the user according to the motion track of the part of the body of the user to fit to obtain an image between the image of the user and the image of the next frame of user, so as to form a dynamic image of the user.
According to the implementation mode, the motion of the motion part is completely supplemented according to the motion track of the part of the body of the user, so that a complete dynamic image of the user is obtained through integration, and 3D dynamic imaging can be realized. In the scenes such as motion sensing games and the like which need complex dynamic imaging and gesture control, a user is generally required to watch dynamic images which make various actions in real time, and the dynamic images of the user can be displayed by utilizing the implementation mode. In addition, if the user's image is captured at a high preset frequency of, for example, 1kHz, the user's moving image may be formed directly from continuous images captured at a high frequency, instead of interpolating the user's image according to the movement locus of the part to fit it to obtain an intermediate image.
In some optional implementations of this embodiment, the detecting, in real time, a motion trajectory of the target in the image acquisition region by using at least one ultrasound transducer array further includes:
the method comprises the steps of equally dividing an image into a plurality of sub-areas, and detecting the motion tracks of targets in the acquisition visual field of the sub-areas in a one-to-one corresponding mode by utilizing a plurality of ultrasonic transducer arrays.
By adopting the implementation mode, the precision of the ultrasonic detection target motion trail can be ensured, and the motion sensing recognition precision is further improved. In a specific example, as shown in fig. 2, the user image may be equally divided into 25 sub-regions, and the acquisition field of view of the image may be divided into 20cm × 20cm units by taking the detection region of the multiple ultrasound transducer arrays as an example of the acquisition field of view of the image 1m × 1m away from the ultrasound transducer array 1m, and the acquisition field of view of the image may be divided into 5 × 5 sub-regions including region 1, region 2, and the like shown in fig. 2. It should be noted that the sub-regions in this implementation are obtained by equally dividing the image, and are not the same concept as the region of the user body part in the image extracted by performing image segmentation, and the region of the user body part may be irregularly shaped, and there may be a case where the region of one user body part corresponds to one sub-region obtained by equally dividing the image, a plurality of sub-regions obtained by equally dividing the image, or a part of one sub-region obtained by equally dividing the image. The motion track of the target in the acquisition field of view of the sub-area is detected in real time by using a plurality of ultrasonic transducer arrays in a one-to-one correspondence manner, as shown in fig. 3, each ultrasonic transducer array (array 1, array 2, etc. in fig. 3) can be arranged to have directivity pointing to the acquisition field of view of the corresponding sub-area, for example, array 1 in fig. 3 has directivity pointing to area 1.
In some optional implementations of this embodiment, the detecting a motion trajectory of the target in the image acquisition region in real time by using at least one ultrasound transducer array further includes: the method for human-computer interaction provided by the embodiment further comprises the following steps of equally dividing the image into a plurality of sub-regions, and detecting the motion tracks of the targets in the acquisition visual field of the sub-regions in real time in a one-to-one correspondence manner by utilizing a plurality of ultrasonic transducer arrays: and judging whether a part moves according to the motion track of the part of the user body, and if so, storing a sub-region corresponding to the part with the motion in the acquired image of the user.
This implementation may further relieve the storage space pressure when capturing images of the user at a higher preset frequency.
Further, in some optional implementation manners of the present embodiment, the human-computer interaction method provided by the present embodiment further includes: and interpolating a subregion corresponding to the part with motion according to the motion track of the part of the body of the user to fit to obtain a subregion corresponding to the part with motion between the image of the user and the image of the next frame of user, and splicing the subregion corresponding to the part with motion between the image of the user and the image of the next frame of user and other subregions except for the subregion corresponding to the part with motion in the image of the user to obtain an image between the image of the user and the image of the next frame of user to form a dynamic image of the user.
By adopting the implementation mode, on one hand, the precision of image fitting through interpolation can be further improved, so that a user dynamic image with higher reduction degree is formed, and on the other hand, compared with a mode of fitting the whole image to obtain an intermediate image, the mode of only performing image fitting through interpolation on the subareas corresponding to the small-range moving parts and then splicing with the static subareas to obtain the intermediate image can further reduce the calculated data amount, save the calculation resources and improve the calculation speed, so that the real-time performance of motion sensing identification is further improved.
In some optional implementation manners of the present embodiment, the human-computer interaction method provided by the present embodiment further includes: each ultrasonic transducer array is configured to detect with ultrasonic waves having a frequency that is different from the other ultrasonic transducer arrays.
By adopting the implementation mode, the one-to-one correspondence between the ultrasonic transducer arrays and the sub-areas can be further ensured when each ultrasonic transducer array detects the motion track of the target in the acquisition view of the corresponding sub-area, the sound wave interference of other ultrasonic transducer arrays is avoided, and the precision of the motion track of the ultrasonic detection target is further ensured.
Another embodiment of the present invention provides a motion sensing game apparatus, including a display device and a human-computer interaction device, wherein the human-computer interaction device includes: the display device comprises a processor, an image acquisition module and at least one ultrasonic transducer array, wherein the image acquisition module and the ultrasonic transducer array can be respectively arranged on the display device (such as the frame position of a display screen);
the image acquisition module is used for acquiring images of users;
the ultrasonic transducer array is used for detecting the motion track of a target in the acquisition field of view of the image in real time;
the processor is used for carrying out image segmentation on the image of the user so as to extract a region of at least one part of the body of the user in the image; and corresponding the target to the part according to the position of the target to obtain a motion track of the part of the body of the user, and obtaining a control instruction based on the motion track of the part of the body of the user, wherein the processor can be realized by a chip.
The man-machine interaction device in the motion sensing game equipment provided by the embodiment realizes three-dimensional space motion sensing interaction based on a mode combining optical imaging and ultrasonic detection, organically combines the high resolution advantage of optical imaging and the high efficiency and low cost advantage of ultrasonic detection target motion trail, thereby reducing the calculated data amount, saving the storage space and the calculation resource, and improving the calculation speed on the basis of ensuring the motion sensing identification precision, thereby greatly improving the real-time performance of motion sensing identification, and greatly improving the game experience of users.
In some optional implementation manners of this embodiment, the processor is further configured to determine whether there is a motion of a part according to a motion trajectory of the part of the body of the user, and if so, control the image acquisition module to acquire the image of the user, or control the image acquisition module to acquire the image of the user again.
In the implementation mode, after the image of the initial frame user is acquired at the initial moment, the image acquisition module does not need to be controlled to continuously acquire the image of the user, but the image acquisition module can be controlled to acquire the image of the next frame user to continue the subsequent acquisition of the control instruction when the position of the target obtained by real-time detection corresponds to the middle position of the image of the initial frame user to obtain the motion track of the part of the body of the user and judge that the part moves (after the part moves and moves, the image acquisition module can not correspond to the part according to the region of the part in the image extracted by image segmentation of the initial frame user and the image of the user needs to be acquired again), the repeated acquisition of the same/similar image when the part of the user does not move is avoided, the calculated data amount can be further reduced, the storage space and the calculation resources are saved, The calculation speed is increased, and therefore the real-time performance of body sensing recognition is improved.
In some optional implementation manners of this embodiment, the processor is further configured to control the image capturing module to capture the image of the user at a preset frequency.
By adopting the implementation mode, the accuracy of the corresponding of the target and the part can be ensured.
Further, in some optional implementation manners of this embodiment, the processor is further configured to determine whether there is a motion of a part according to a motion trajectory of the part of the body of the user, and if so, store the image of the user acquired by the image acquisition module.
This implementation may relieve the storage space pressure when capturing images of the user at a higher preset frequency.
In some optional implementations of this embodiment, the processor is further configured to interpolate the image of the user according to the motion trajectory of the part of the body of the user, so as to fit an image between the image of the user and an image of a next frame of the user, and form a dynamic image of the user.
According to the implementation mode, the motion of the motion part is completely supplemented according to the motion track of the part of the body of the user, so that a complete dynamic image of the user is obtained through integration, and 3D dynamic imaging can be realized.
In some optional implementation manners of this embodiment, the human-computer interaction device includes a plurality of ultrasonic transducer arrays, and the plurality of ultrasonic transducer arrays detect a motion trajectory of a target in an acquisition field of view of a sub-region in a one-to-one correspondence manner, where the sub-region is obtained by equally dividing an image, and the plurality of ultrasonic transducer arrays may be discretely arranged.
By adopting the implementation mode, the precision of the motion trail of the ultrasonic detection target can be further ensured, so that the motion sensing recognition precision is further improved. In one specific example, the real-time detection of the motion trajectory of the target in the acquisition field of view of the sub-region in a one-to-one correspondence by the plurality of ultrasonic transducer arrays may be realized by arranging each ultrasonic transducer array to have a directivity pointing to the acquisition field of view of the corresponding sub-region.
Further, in some optional implementation manners of this embodiment, the processor is further configured to determine whether there is a motion of a part according to a motion trajectory of the part of the body of the user, and if so, store a sub-region corresponding to the part where the motion exists in the acquired image of the user.
This implementation may further relieve the storage space pressure when capturing images of the user at a higher preset frequency.
Further, in some optional implementation manners of this embodiment, the processor is further configured to interpolate a sub-region corresponding to a part where motion exists according to a motion trajectory of the part of the body of the user, to obtain a sub-region corresponding to a part where motion exists between the image of the user and the image of the next frame of user through fitting, and to stitch the sub-region corresponding to the part where motion exists between the image of the user and the image of the next frame of user with other sub-regions of the image of the user except for the sub-region corresponding to the part where motion exists, to obtain an image between the image of the user and the image of the next frame of user, so as to form a dynamic image of the user.
By adopting the implementation mode, on one hand, the precision of image fitting through interpolation can be further improved, so that a user dynamic image with higher reduction degree is formed, and on the other hand, compared with a mode of fitting the whole image to obtain an intermediate image, the mode of only performing image fitting through interpolation on the subareas corresponding to the small-range moving parts and then splicing with the static subareas to obtain the intermediate image can further reduce the calculated data amount, save the calculation resources and improve the calculation speed, so that the real-time performance of motion sensing identification is further improved.
In some optional implementations of the present embodiment, each of the plurality of ultrasonic transducer arrays is configured to detect with ultrasonic waves having a frequency that is different from the other ultrasonic transducer arrays.
By adopting the implementation mode, the one-to-one correspondence between the ultrasonic transducer arrays and the sub-areas can be further ensured when each ultrasonic transducer array detects the motion track of the target in the acquisition view of the corresponding sub-area, the sound wave interference of other ultrasonic transducer arrays is avoided, and the precision of the motion track of the ultrasonic detection target is further ensured.
In some optional implementations of the embodiment, the ultrasound transducer array includes a transmitting transducer array composed of a plurality of transmitting transducers and a receiving transducer array composed of a plurality of receiving transducers, the transmitting transducer in an ultrasound transducer array is used for transmitting ultrasound waves with a frequency different from that of other ultrasound transducer arrays, and the receiving transducer is used for performing frequency-based filtering on the received ultrasound waves so as to retain only the ultrasound waves transmitted by the transmitting transducers belonging to the same ultrasound transducer array.
By adopting the implementation mode, the one-to-one correspondence between the ultrasonic transducer arrays and the sub-areas can be further ensured when each ultrasonic transducer array detects the motion track of the target in the acquisition view of the corresponding sub-area, the receiving transducer is prevented from being interfered by the sound wave of the ultrasonic waves emitted by the transmitting transducers in other ultrasonic transducer arrays, and the precision of the motion track of the target detected by the ultrasonic waves is further ensured.
In some optional implementations of this embodiment, in an ultrasound transducer array, the size of the receiving transducer array is larger than the size of the transmitting transducer array.
By adopting the implementation mode, under the condition that the ultrasonic transducer array is set to have the directivity pointing to the acquisition field of the corresponding sub-area, and the motion track of the target in the acquisition field of one sub-area is detected in real time by one ultrasonic transducer array, the higher directivity of the ultrasonic transducer array is formed based on the large-size receiving transducer array, so that the resolution of the ultrasonic transducer array in the process of detecting the motion track of the target is improved, and the precision of detecting the motion track of the target by ultrasonic waves is improved. The realization mode can further ensure the one-to-one correspondence between the ultrasonic transducer arrays and the sub-areas when each ultrasonic transducer array detects the motion track of the target in the acquisition view of the corresponding sub-area, further prevents the ultrasonic transducer arrays from being interfered by sound waves caused by other ultrasonic transducer arrays and from forming the sound waves to other ultrasonic transducer arrays, and further ensures the precision of the motion track of the ultrasonic detection target.
In a specific example, as shown in fig. 4, the receiving transducer array is arranged to surround the transmitting transducer array, wherein the size of the ultrasonic radiation area of the transmitting transducer array is consistent with the size of the acquisition field of view of the sub-area corresponding to the ultrasonic transducer array to which the transmitting transducer array belongs.
It should be noted that the principle and the workflow of the human-computer interaction device in the motion sensing game device provided by this embodiment are similar to those of the above human-computer interaction method, and the two related parts can be referred to each other.
It can be understood that the motion sensing game device provided by the embodiment can realize various motion sensing games such as yoga motion sensing games, dance motion sensing games, maze motion sensing games and the like. It should be noted that, although the above description has been given by taking a motion sensing game device as an example, a human-computer interaction device in a motion sensing game device provided in this embodiment can be applied to any motion sensing device that realizes human-computer interaction based on capturing a motion of a user, such as a motion sensing rehabilitation device.
It should be noted that in the description of the present invention, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other identical elements in a process, method, article, or apparatus that comprises the element.
It should be understood that the above-mentioned embodiments of the present invention are only examples for clearly illustrating the present invention, and are not intended to limit the embodiments of the present invention, and it will be obvious to those skilled in the art that other variations and modifications can be made on the basis of the above description, and all embodiments cannot be exhaustive, and all obvious variations and modifications belonging to the technical scheme of the present invention are within the protection scope of the present invention.
Claims (15)
1. A human-computer interaction method, comprising:
acquiring an image of a user, and detecting a motion track of a target in an acquisition field of the image in real time by using at least one ultrasonic transducer array;
performing image segmentation on the image of the user to extract a region of at least one part of the user's body in the image;
and corresponding the target to the part according to the position of the target to obtain the motion track of the part of the body of the user, and obtaining a control instruction based on the motion track of the part of the body of the user.
2. The human-computer interaction method of claim 1, further comprising: and judging whether a part moves according to the motion track of the part of the body of the user, and if so, acquiring the image of the user.
3. The human-computer interaction method of claim 1, further comprising: images of the user are acquired at a preset frequency.
4. A human-computer interaction method according to claim 3, characterized in that the method further comprises: and judging whether a part moves according to the motion track of the part of the user body, and if so, storing the acquired image of the user.
5. A human-computer interaction method according to any one of claims 2-4, characterized in that the method further comprises: and interpolating the image of the user according to the motion track of the part of the body of the user to fit to obtain an image between the image of the user and the image of the next frame of user, so as to form a dynamic image of the user.
6. The human-computer interaction method according to claim 1, wherein the detecting the motion track of the target in the image acquisition area in real time by using at least one ultrasonic transducer array further comprises:
the method comprises the steps of equally dividing an image into a plurality of sub-areas, and detecting the motion tracks of targets in the acquisition visual field of the sub-areas in a one-to-one corresponding mode by utilizing a plurality of ultrasonic transducer arrays.
7. A human-computer interaction method according to claim 6, characterized in that the method further comprises: and judging whether a part moves according to the motion track of the part of the body of the user, and if so, acquiring the image of the user.
8. A human-computer interaction method according to claim 6, characterized in that the method further comprises: images of the user are acquired at a preset frequency.
9. A human-computer interaction method according to claim 8, characterized in that the method further comprises: and judging whether a part moves according to the motion track of the part of the user body, and if so, storing a sub-region corresponding to the part with the motion in the acquired image of the user.
10. A human-computer interaction method according to any one of claims 7-9, characterized in that the method further comprises: and interpolating a subregion corresponding to the part with motion according to the motion track of the part of the body of the user to fit to obtain a subregion corresponding to the part with motion between the image of the user and the image of the next frame of user, and splicing the subregion corresponding to the part with motion between the image of the user and the image of the next frame of user and other subregions except for the subregion corresponding to the part with motion in the image of the user to obtain an image between the image of the user and the image of the next frame of user to form a dynamic image of the user.
11. A human-computer interaction method according to claim 6, characterized in that the method further comprises: each ultrasonic transducer array is configured to detect with ultrasonic waves having a frequency that is different from the other ultrasonic transducer arrays.
12. A human-computer interaction device for performing the human-computer interaction method according to any one of claims 1 to 11, comprising: the system comprises a processor, an image acquisition module and at least one ultrasonic transducer array;
the image acquisition module is used for acquiring images of users;
the ultrasonic transducer array is used for detecting the motion track of a target in the acquisition field of view of the image in real time;
the processor is used for carrying out image segmentation on the image of the user so as to extract a region of at least one part of the body of the user in the image; and corresponding the target to the part according to the position of the target to obtain the motion track of the part of the body of the user, and obtaining a control instruction based on the motion track of the part of the body of the user.
13. The human-computer interaction device of claim 12, wherein the ultrasound transducer array comprises a transmitting transducer array comprising a plurality of transmitting transducers and a receiving transducer array comprising a plurality of receiving transducers, the transmitting transducers in an ultrasound transducer array are used for transmitting ultrasound waves with a frequency different from that of other ultrasound transducer arrays, and the receiving transducers are used for performing frequency-based filtering on the received ultrasound waves so as to only retain the ultrasound waves transmitted by the transmitting transducers in the same ultrasound transducer array.
14. A human-computer interaction device according to claim 13, wherein the size of the array of receiving transducers in an array of ultrasonic transducers is larger than the size of the array of transmitting transducers.
15. A motion sensing gaming device comprising a display means and a human interaction means as claimed in any one of claims 12 to 14.
Priority Applications (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010074602.7A CN111282261B (en) | 2020-01-22 | 2020-01-22 | Man-machine interaction method and device and somatosensory game equipment |
| PCT/CN2021/073250 WO2021147989A1 (en) | 2020-01-22 | 2021-01-22 | Human-machine interaction method and apparatus, and motion sensing game device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010074602.7A CN111282261B (en) | 2020-01-22 | 2020-01-22 | Man-machine interaction method and device and somatosensory game equipment |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111282261A true CN111282261A (en) | 2020-06-16 |
| CN111282261B CN111282261B (en) | 2023-08-08 |
Family
ID=71018107
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010074602.7A Active CN111282261B (en) | 2020-01-22 | 2020-01-22 | Man-machine interaction method and device and somatosensory game equipment |
Country Status (2)
| Country | Link |
|---|---|
| CN (1) | CN111282261B (en) |
| WO (1) | WO2021147989A1 (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2021147989A1 (en) * | 2020-01-22 | 2021-07-29 | 京东方科技集团股份有限公司 | Human-machine interaction method and apparatus, and motion sensing game device |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101437124A (en) * | 2008-12-17 | 2009-05-20 | 三星电子（中国）研发中心 | Method for processing dynamic gesture identification signal facing (to)television set control |
| US20160195920A1 (en) * | 2015-01-05 | 2016-07-07 | Boe Technology Group Co., Ltd. | Somatosensory recognition system and recognition method thereof |
| WO2016192438A1 (en) * | 2015-06-05 | 2016-12-08 | 深圳奥比中光科技有限公司 | Motion sensing interaction system activation method, and motion sensing interaction method and system |
| CN107544673A (en) * | 2017-08-25 | 2018-01-05 | 上海视智电子科技有限公司 | Body feeling interaction method and body feeling interaction system based on depth map information |
| CN108089702A (en) * | 2017-12-07 | 2018-05-29 | 深圳大学 | A kind of man-machine body-sensing exchange method and system based on ultrasonic wave |
Family Cites Families (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| TWI476706B (en) * | 2012-04-30 | 2015-03-11 | Pixart Imaging Inc | Method for outputting command by detecting object movement and system thereof |
| CN108924438B (en) * | 2018-06-26 | 2021-03-02 | Oppo广东移动通信有限公司 | Shooting control method and related product |
| CN109656355A (en) * | 2018-10-23 | 2019-04-19 | 西安交通大学 | A kind of exchange method and device of mobile phone and other display equipment |
| CN111282261B (en) * | 2020-01-22 | 2023-08-08 | 京东方科技集团股份有限公司 | Man-machine interaction method and device and somatosensory game equipment |
-
2020
- 2020-01-22 CN CN202010074602.7A patent/CN111282261B/en active Active
-
2021
- 2021-01-22 WO PCT/CN2021/073250 patent/WO2021147989A1/en active Application Filing
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101437124A (en) * | 2008-12-17 | 2009-05-20 | 三星电子（中国）研发中心 | Method for processing dynamic gesture identification signal facing (to)television set control |
| US20160195920A1 (en) * | 2015-01-05 | 2016-07-07 | Boe Technology Group Co., Ltd. | Somatosensory recognition system and recognition method thereof |
| WO2016192438A1 (en) * | 2015-06-05 | 2016-12-08 | 深圳奥比中光科技有限公司 | Motion sensing interaction system activation method, and motion sensing interaction method and system |
| CN107544673A (en) * | 2017-08-25 | 2018-01-05 | 上海视智电子科技有限公司 | Body feeling interaction method and body feeling interaction system based on depth map information |
| CN108089702A (en) * | 2017-12-07 | 2018-05-29 | 深圳大学 | A kind of man-machine body-sensing exchange method and system based on ultrasonic wave |
Non-Patent Citations (1)
| Title |
|---|
| 封家鹏: "双目标三维超声定位笔式人机交互装置", 《中国优秀硕士学位论文全文数据库》, pages 140 - 1 * |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2021147989A1 (en) * | 2020-01-22 | 2021-07-29 | 京东方科技集团股份有限公司 | Human-machine interaction method and apparatus, and motion sensing game device |
Also Published As
| Publication number | Publication date |
|---|---|
| WO2021147989A1 (en) | 2021-07-29 |
| CN111282261B (en) | 2023-08-08 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US9098116B2 (en) | Object and movement detection | |
| CN204480228U (en) | motion sensing and imaging device | |
| CN104106267B (en) | Signal enhancing beam forming in augmented reality environment | |
| EP3470963B1 (en) | Control using movements | |
| CN102542566B (en) | Orienting the position of a sensor | |
| CN101976330B (en) | Gesture recognition method and system | |
| CN105518584A (en) | Recognizing interactions with hot zones | |
| US20200097092A1 (en) | Gesture recognition using 3d mm-wave radar | |
| US20130129224A1 (en) | Combined depth filtering and super resolution | |
| CN103970264B (en) | Gesture recognition and control method and device | |
| CN109683699A (en) | The method, device and mobile terminal of augmented reality are realized based on deep learning | |
| CN101937267A (en) | Method for simulating mouse input and device thereof | |
| CN102508578A (en) | Projection positioning device and method as well as interaction system and method | |
| CN107066081B (en) | Interactive control method and device of virtual reality system and virtual reality equipment | |
| US10416305B2 (en) | Positioning device and positioning method | |
| CN111282261B (en) | Man-machine interaction method and device and somatosensory game equipment | |
| KR20120026956A (en) | Method and apparatus for motion recognition | |
| CN110572600A (en) | video processing method and electronic equipment | |
| CN106951077B (en) | Prompting method and first electronic device | |
| CN102843515A (en) | Image acquisition method and system for image recognition | |
| CN104412123A (en) | System and method for 3d ultrasound volume measurements | |
| US20160073087A1 (en) | Augmenting a digital image with distance data derived based on acoustic range information | |
| CN102495709A (en) | Method and device for regulating sampling frame of ultrasonic imaging device | |
| CN115048954A (en) | Retina-imitating target detection method and device, storage medium and terminal | |
| CN110989844A (en) | Input method, watch, system and storage medium based on ultrasonic waves |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |