CN117055722A - Vehicle-mounted VR gesture recognition method, device and system and electronic equipment - Google Patents

Vehicle-mounted VR gesture recognition method, device and system and electronic equipment Download PDF

Info

Publication number
CN117055722A
CN117055722A CN202211213036.9A CN202211213036A CN117055722A CN 117055722 A CN117055722 A CN 117055722A CN 202211213036 A CN202211213036 A CN 202211213036A CN 117055722 A CN117055722 A CN 117055722A
Authority
CN
China
Prior art keywords
gesture
vehicle
data
fingerstall
posture
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202211213036.9A
Other languages
Chinese (zh)
Inventor
丁彬
傅强
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Beijing Rockwell Technology Co Ltd
Original Assignee
Beijing Rockwell Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Beijing Rockwell Technology Co Ltd filed Critical Beijing Rockwell Technology Co Ltd
Priority to CN202211213036.9A priority Critical patent/CN117055722A/en
Publication of CN117055722A publication Critical patent/CN117055722A/en
Pending legal-status Critical Current

Links

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/017Gesture based interaction, e.g. based on a set of recognized hand gestures
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/011Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
    • G06F3/014Hand-worn input/output arrangements, e.g. data gloves

Abstract

The invention discloses a vehicle-mounted VR gesture recognition method, device, system and electronic equipment, and relates to the technical field of vehicles. By adopting the fingerstall to collect the gesture information of the finger in real time, and processing and identifying the gesture information of the finger in the vehicle, the low-cost and accurate hand gesture identification is realized on the basis of not using a vehicle camera.

Description

Vehicle-mounted VR gesture recognition method, device and system and electronic equipment
Technical Field
The disclosure relates to the technical field of vehicles, in particular to a vehicle-mounted VR gesture recognition method, device, system and electronic equipment.
Background
With the development of social economy, the requirements of people on living standards are also higher and higher, so that in-car entertainment activities are also becoming more and more important. Compared with the traditional entertainment activities, the VR technology can relieve the motion sickness symptoms of passengers, so that the vehicle-mounted VR technology has great development prospect.
At present, for gesture recognition of vehicle-mounted VR, gesture recognition of a user is performed on two cameras preset on VR glasses and a plurality of cameras in the vehicle at the same time, and the gesture recognition of the scheme is very accurate for a wearer.
Although the above solution can accurately recognize the gesture of the wearer, it requires a large number of cameras to be arranged, is extremely costly, requires a strong computing power to support real-time operation, and is difficult to realize for a vehicle with low cost or no arrangement position.
Disclosure of Invention
The disclosure provides a vehicle-mounted VR gesture recognition method, device and system and electronic equipment. The hand gesture data acquisition device mainly aims to solve the problems that in the related art, a large number of cameras are arranged, the cost is increased, and a strong computing power is needed to support a real-time operation mode, so that accurate acquisition of hand gesture data is realized.
According to a first aspect of the present disclosure, there is provided a method of in-vehicle VR gesture pose for a vehicle, comprising:
receiving identification information of a fingerstall and first posture data of the fingerstall, which are sent by fingerstall acquisition equipment, wherein the fingerstall acquisition equipment comprises at least three fingerstalls, and each fingerstall corresponds to one piece of identification information;
judging whether the first gesture data need to be converted according to the current driving state of the vehicle;
if the first posture data are required to be converted, second posture data of the vehicle in the current driving state are obtained, and the first posture data are corrected according to the second posture data to obtain relative posture data;
and recognizing the gesture according to the relative gesture data.
Optionally, the correcting the first gesture data according to the second gesture data to obtain a relative position
The gesture data includes:
and carrying out difference on the second gesture data and the first gesture data to obtain the relative gesture data.
Optionally, before the gesture is identified according to the relative gesture data, the method further includes:
detecting whether the relative gesture data contains preset number of fingerstall identification information;
Discarding the relative gesture data if the relative gesture data does not contain the preset number of fingerstall identification information;
if the relative gesture data contains the preset number of fingerstall identification information, gesture recognition is performed on the relative gesture data under the same acquisition time.
Optionally, the identifying the gesture with respect to the relative gesture data at the same acquisition time includes:
calculating the pointing gesture and the acceleration according to the last relative gesture data and the current relative gesture data;
according to the pointing gesture, determining the pointing gesture of the palm;
and determining the displacement characteristic of the hand according to the acceleration.
Optionally, the determining whether the first gesture data needs to be converted according to the current driving state of the vehicle includes:
confirming the current driving state of the vehicle based on the gear and the current speed per hour of the vehicle;
and if the current driving state is determined to be the running state, determining that the first gesture data need to be converted.
Optionally, the method further comprises:
and if the current driving state is determined to be a stop state, directly determining the first gesture data as the relative gesture data.
Optionally, the method further comprises:
after the third posture information of the VR glasses is obtained, rendering a corresponding picture according to the third posture information;
and superposing the recognition result aiming at the gesture into a rendering picture, and returning the superposed rendering picture to the VR glasses.
According to a second aspect of the present disclosure, there is provided a method of vehicle-mounted VR gesture pose for a finger cuff collection device, comprising:
transmitting identification information of at least three started fingerstalls to a vehicle in response to communication establishment of a wireless connection network with the vehicle;
collecting first gesture data of the finger based on the at least three started finger cuffs;
and sending the acquired first posture information and the corresponding identification information to the vehicle.
According to a third aspect of the present disclosure, there is provided a system of in-vehicle VR gesture gestures, the system comprising: vehicle, dactylotheca collection device, wherein:
the method comprises the steps that the fingerstall acquisition equipment responds to communication establishment between a wireless connection network and a vehicle, and identification information of at least three started fingerstalls is sent to the vehicle;
the vehicle receives the identification information of at least three started fingerstalls sent by the fingerstall acquisition equipment;
The fingerstall collecting equipment collects first posture data of fingers based on the at least three started fingerstalls and sends the collected first posture information and corresponding identification information to the vehicle;
the method comprises the steps that a vehicle receives identification information of a fingerstall sent by fingerstall acquisition equipment and first posture data of the fingerstall;
the vehicle judges whether the first gesture data need to be converted according to the current driving state of the vehicle;
the vehicle converts the first posture data to be converted to obtain second posture data of the vehicle in the current driving state, and corrects the first posture data according to the second posture data to obtain relative posture data;
and the vehicle recognizes the gesture according to the relative gesture data.
Optionally, the system further comprises VR glasses, wherein the vehicle is further configured to:
the VR glasses send the third gesture information to the vehicle;
the vehicle acquires third posture information of the VR glasses, and renders a corresponding picture according to the third posture information;
the vehicle superimposes the recognition result aiming at the gesture to the rendering picture, and returns the superimposed rendering picture to the VR glasses;
VR glasses receive the rendering pictures sent by the vehicle;
and the VR glasses display the rendering picture in the VR glasses screen.
According to a third aspect of the present disclosure, there is provided an apparatus of an in-vehicle VR gesture pose for a vehicle, comprising:
the receiving unit is used for receiving the identification information of the fingerstall and the first gesture data of the fingerstall, which are sent by the fingerstall acquisition equipment, wherein the fingerstall acquisition equipment comprises at least three fingerstalls, and each fingerstall corresponds to one identification information;
the judging unit is used for judging whether the first gesture data need to be converted according to the current driving state of the vehicle;
the acquisition unit is used for acquiring second posture data of the vehicle in the current driving state when the first posture data are required to be converted;
the correcting unit is used for correcting the first posture data according to the second posture data to obtain relative posture data;
and the identification unit is used for identifying the gesture according to the relative gesture data.
Optionally, the correction unit is further configured to perform a difference between the second gesture data and the first gesture data to obtain the relative gesture data.
Optionally, the apparatus further includes:
the detection unit is used for detecting whether the relative gesture data contains a preset number of fingerstall identification information before the gesture is identified by the identification unit according to the relative gesture data;
a discarding unit, configured to discard the relative gesture data when the relative gesture data does not include the preset number of fingerstall identification information;
the recognition unit is further configured to perform gesture recognition on the relative gesture data at the same acquisition time when the relative gesture data includes the preset number of fingerstall identification information.
Optionally, the identifying unit includes:
the calculation module is used for calculating the pointing gesture and the acceleration according to the last relative gesture data and the current relative gesture data;
the first determining module is used for determining the pointing gesture of the palm according to the pointing gesture;
and the second determining module is used for determining the displacement characteristic of the hand according to the acceleration.
Optionally, the judging unit includes:
the third confirmation module is used for confirming the current driving state of the vehicle based on the gear and the current speed per hour of the vehicle;
and the fourth determining module is used for determining that the first gesture data need to be converted when the current driving state is determined to be the running state.
Optionally, the apparatus further includes:
and the determining unit is used for directly determining the first gesture data as the relative gesture data when the current driving state is determined to be a stop state.
Optionally, the apparatus further includes:
the rendering unit is used for rendering a corresponding picture according to the third posture information after the third posture information of the VR glasses is acquired;
and the superposition unit is used for superposing the recognition result aiming at the gesture to the rendering picture and returning the superposed rendering picture to the VR glasses.
According to a fifth aspect of the present disclosure, there is provided an apparatus for a vehicle-mounted VR gesture pose, the apparatus for a finger cuff collection device, comprising:
a transmitting unit for transmitting identification information of at least three started fingerstalls to a vehicle in response to communication establishment between a wireless connection network and the vehicle;
the acquisition unit is used for acquiring first gesture data of the finger based on the at least three started finger cuffs;
and the sending unit is used for sending the acquired first gesture information and the corresponding identification information to the vehicle.
According to a sixth aspect of the present disclosure, there is provided a vehicle comprising the recognition device of the on-board VR gesture as set forth in the fourth or fifth aspect.
According to a seventh aspect of the present disclosure, there is provided an electronic device comprising:
at least one processor; and
a memory communicatively coupled to the at least one processor; wherein,
the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the method of the first aspect.
According to an eighth aspect of the present disclosure, there is provided an electronic device comprising:
at least one processor; and
a memory communicatively coupled to the at least one processor; wherein,
the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the method of the second aspect.
According to the method, the device, the system and the electronic equipment for the gesture of the vehicle-mounted VR, the finger stall collecting equipment collects first gesture data of fingers, the first gesture data and identification information of the finger stall are sent to the vehicle, the vehicle judges whether the first gesture data need to be subjected to data correction according to the current driving state of the vehicle, if so, the first gesture data are corrected according to the current state data of the vehicle to obtain relative gesture data, if not, the first gesture data are directly converted into the relative gesture data, and the vehicle recognizes the gesture of a user based on the relative gesture data. Compared with the related art, the embodiment of the application collects the gesture information of the finger in real time by adopting the fingerstall, processes and identifies the gesture information of the finger in the vehicle, and realizes low-cost and accurate hand gesture identification on the basis of not using a vehicle camera.
It should be understood that the description in this section is not intended to identify key or critical features of the embodiments of the application or to delineate the scope of the application. Other features of the present application will become apparent from the description that follows.
Drawings
The drawings are for a better understanding of the present solution and are not to be construed as limiting the present disclosure. Wherein:
fig. 1 is a flow chart of a method for recognizing gesture of a vehicle VR gesture provided by an embodiment of the present disclosure;
fig. 2 is a schematic diagram of a positional relationship between a fingerstall collecting device and a vehicle according to an embodiment of the present disclosure;
fig. 3 is a flowchart of a method for detecting finger cuff data according to an embodiment of the disclosure;
fig. 4 is a flowchart of another method for recognizing a gesture of a vehicle-mounted VR according to an embodiment of the present disclosure;
FIG. 5 is a schematic diagram of a finger cuff according to an embodiment of the disclosure;
fig. 6 is a schematic flow chart of a vehicle-mounted VR gesture recognition system provided in an embodiment of the present disclosure;
fig. 7 is a schematic structural diagram of another device for recognizing a gesture of a vehicle-mounted VR according to an embodiment of the present disclosure;
fig. 8 is a schematic structural diagram of another device for recognizing a gesture of a vehicle-mounted VR according to an embodiment of the present disclosure;
Fig. 9 is a schematic structural diagram of another device for recognizing a gesture of a vehicle VR in accordance with an embodiment of the present disclosure;
fig. 10 is a block diagram of a recognition system for a gesture of a vehicle VR according to an embodiment of the present disclosure;
fig. 11 is a schematic block diagram of an example electronic device 800 provided by an embodiment of the disclosure.
Detailed Description
Exemplary embodiments of the present disclosure are described below in conjunction with the accompanying drawings, which include various details of the embodiments of the present disclosure to facilitate understanding, and should be considered as merely exemplary. Accordingly, one of ordinary skill in the art will recognize that various changes and modifications of the embodiments described herein can be made without departing from the scope and spirit of the present disclosure. Also, descriptions of well-known functions and constructions are omitted in the following description for clarity and conciseness.
The following describes a method, a device, a system and an electronic device for recognizing a gesture of a vehicle-mounted VR according to an embodiment of the present disclosure with reference to the accompanying drawings.
Fig. 1 is a flowchart of a method for recognizing gesture of a vehicle VR gesture provided in an embodiment of the present disclosure.
As shown in fig. 1, the method is for a vehicle, and comprises the steps of:
step 101, receiving identification information of a fingerstall and first posture data of the fingerstall sent by fingerstall collecting equipment, wherein the fingerstall collecting equipment comprises at least three fingerstalls, and each fingerstall corresponds to one piece of identification information.
Referring to fig. 2, fig. 2 is a schematic diagram of a position relationship between a finger stall collecting device and a vehicle according to an embodiment of the present application; the vehicle receives first gesture data and identification information sent by the fingerstall acquisition equipment based on the wireless communication module; and distinguishing the first gesture data based on the identification information of the fingerstall.
The fingerstall equipment can comprise a plurality of fingerstalls, and in an ideal state, a user should use all the fingerstalls to interact with the vehicle, but in practical application, the user can select a corresponding number of fingerstalls to wear to interact according to own needs, but too few first gesture data acquired by the fingerstalls can lead to inaccurate gesture gestures of the identified user, so that in order to ensure the accuracy of identification, the user needs to wear at least three fingerstall equipment to carry out man-machine interaction.
Step 102, judging whether the first gesture data need to be converted according to the current driving state of the vehicle.
The first posture data collected by the fingerstall is inertial measurement unit (Inertial measurement unit, imu) data of the fingerstall relative to the earth, when the vehicle is in a driving state, the position of the vehicle relative to the earth changes, and therefore the first posture data can be influenced by the vehicle, and at the moment, the first posture data is superposition of the posture change of the vehicle and the posture change of the finger, so that the first posture data needs to be converted, and the data of the fingerstall relative to the earth is converted into imu data of the fingerstall relative to the vehicle.
Step 103, if the first posture data needs to be converted, obtaining second posture data of the vehicle in the current driving state, and correcting the first posture data according to the second posture data to obtain relative posture data.
The second gesture is imu data of the vehicle relative to the earth, and the method for obtaining the imu data of the vehicle refers to any implementation manner in the prior art, and the embodiments of the present application are not described in detail herein.
And step 104, recognizing the gesture according to the relative gesture data.
The relative gesture data is imu data of the fingerstall relative to the vehicle, namely gesture changes of the fingers of the user, and the gesture changes of the user identify gestures of the user so as to confirm indication information corresponding to the gestures.
The method for gesture of the vehicle-mounted VR provided by the present disclosure mainly comprises the following steps: the fingerstall collects first posture data of fingers and sends the first posture data and identification information of the fingerstall to the vehicle, the vehicle judges whether the first posture data is required to be subjected to data correction according to the current driving state of the vehicle, if so, the first posture data is corrected according to the current state data of the vehicle to obtain relative posture data, if not, the first posture data is directly converted into the relative posture data, and the vehicle recognizes gesture postures of a user based on the relative posture data. Compared with the related art, the embodiment of the application collects the gesture information of the finger in real time by adopting the fingerstall, processes and identifies the gesture information of the finger in the vehicle, and realizes low-cost and accurate hand gesture identification on the basis of not using a vehicle camera.
As an extension to the above-mentioned application embodiment, when the first gesture data is corrected according to the second gesture data in step 103, the following method may be used, but is not limited to: and subtracting the second posture data from the first posture data, subtracting the first posture data from the second posture data to obtain the relative posture data, wherein the first posture information comprises the second posture information and the relative posture data, and subtracting the second posture data from the first posture data to obtain the relative posture data, namely imu data of the fingerstall relative to the vehicle.
As an extension to the embodiment of the foregoing application, after the vehicle receives the first gesture information transmitted by the fingerstall collecting device, the first gesture information is sent to the input of the system, so that other applications can call the first gesture information.
When recognizing gesture according to relative gesture data, if the transmission of the finger stall gesture data is delayed or the finger stall collecting equipment is not started, a gesture instruction recognized by a vehicle is possibly different from an instruction to be expressed by a passenger, so that the quantity of the finger stall data contained in gesture data needs to be detected when the gesture is recognized; referring to fig. 3, fig. 3 is a flowchart of a method for detecting finger cuff data according to an embodiment of the disclosure, including:
Step 201, detecting whether the relative gesture data contains a preset number of fingerstall identification information.
The fingerstall collecting device has ten fingerstalls in total, and in an ideal state, the first gesture data should have ten fingerstall identification information, namely ten fingerstalls, and the first gesture data is collected at the same time, but in practical application, a user can wear different numbers of fingerstalls according to own needs, information delay caused by factors such as a network may also exist, and when the number of fingerstalls participating in detecting the first gesture data is too small, the relative gesture data corresponding to the first gesture data may be incomplete data. In practical application, the preset number may be set to 3, and in the embodiment of the subsequent application, the explanation is performed with the preset number of fingerstall identification information being 3, but the explanation mode is not limited to a specific preset number, and in practical application, the preset number may be set by oneself according to the actual number of fingerstalls, habits of users, and the like, which is not limited in the embodiment of the present application.
Step 202, discarding the first gesture data if the relative gesture data does not include the preset number of fingerstall identification information.
When the relative gesture data contains two or less fingerstall identification information, the current relative gesture data is determined to be incomplete, and if gesture recognition is performed on the current relative gesture data, gesture recognition errors can be caused, so that discarding processing is performed on gesture data determined to be incomplete.
Step 203, if the relative gesture data includes the preset number of fingerstall identification information, gesture recognition is performed on the relative gesture data at the same acquisition time.
When the relative gesture data comprises 3 or more gesture data, the current relative gesture data can be recognized as complete, and gesture recognition is performed; calculating the pointing gesture and the acceleration according to the last relative gesture data and the current relative gesture data; according to the pointing gesture, determining the pointing gesture of the palm; determining the displacement characteristics of the hand according to the acceleration; taking the gesture and acceleration of each finger stall, the motion characteristics of the hand can be obtained, such as: the finger sleeves approach to the same direction, indicating grabbing; the finger sleeves are far away from each other in the outward direction by the circle center, which indicates that the palm is open; when the imu value of a plurality of fingerstalls in the relative gesture data is not changed, and only the imu value of one fingerstall is obviously changed, the current user can be considered to point by using one finger; when the imu values of the plurality of finger cuffs in the relative posture data are not changed and only the imu values of the two finger cuffs are significantly changed, it can be said that the current user performs the operations of pinching and enlarging with two fingers.
When two kinds of indication information exist in the gesture situation of the user, selecting one of the more finger stall gestures as the indication information; for example, the five finger cuffs on the left hand of the user all make the grabbing action, and the two finger cuffs on the right hand of the user make the amplifying action, then the left hand of the user is selected as the indication information.
As an extension to the embodiment of the present application, when judging the driving state of the current vehicle, the following method may be adopted: confirming the current driving state of the vehicle based on the gear and the current speed per hour of the vehicle; if the current driving state is determined to be an operation state, determining that the first gesture data need to be converted; and if the current driving state is determined to be a stop state, directly determining the first gesture data as the relative gesture data.
As an extension to the embodiment of the above application, after the indication information of the gesture is identified, when the virtual screen is rendered in the vehicle according to the indication information, the gesture action of the user needs to be added to the rendered screen, and the following method may be adopted: after the third posture information of the VR glasses is obtained, rendering a corresponding picture according to the third posture information; superposing the recognition result aiming at the gesture into a rendering picture, and returning the superposed rendering picture to the VR glasses; please refer to any implementation manner in the prior art for a method for performing image rendering according to the gesture information, and the embodiments of the present application are not described in detail herein.
Fig. 4 is a flowchart of a method for recognizing gesture of a vehicle-mounted VR, which is provided by an embodiment of the present disclosure, where the method is used in a fingerstall collecting device, and includes:
in step 301, in response to the establishment of communication between the wireless connection network and the vehicle, identification information of at least three activated finger cuffs is transmitted to the vehicle.
Please continue to refer to fig. 2; the fingerstall and the vehicle are connected in a non-inductive mode, the fingerstall is started to send a connection broadcast, and after receiving the connection broadcast, a wireless communication module in the vehicle establishes communication; the wireless link network may be bluetooth, 2.4g, etc., which is not limited in the embodiment of the present application.
Step 302, collecting first gesture data of the finger based on the at least three started finger cuffs.
Referring to fig. 5, fig. 5 is a schematic diagram of a finger sleeve structure according to an embodiment of the disclosure; after the fingerstall is started, the gesture data of the finger are collected based on the imu module at the top.
Step 303, sending the collected first gesture information and the corresponding identification information to the vehicle.
Binding the identification information of the fingerstall with the posture information acquired by the fingerstall, and transmitting the information to the vehicle through a wireless connection network.
In order to facilitate understanding of the interaction relationship among the vehicle, the finger stall collecting device and the VR head display in the embodiment of the present application, please refer to fig. 6, which includes:
In step 401, the finger cuff collection device transmits identification information of at least three activated finger cuffs to the vehicle in response to communication establishment of the wireless connection network with the vehicle.
Step 402, the vehicle receives identification information of at least three started finger cuffs sent by the finger cuff collection device.
Step 403, the finger stall collecting device collects first gesture data of fingers based on the at least three started finger stalls, and sends the collected first gesture information and corresponding identification information to the vehicle.
Step 404, the vehicle receives identification information of the fingerstall and first gesture data of the fingerstall, which are sent by the fingerstall acquisition device.
Step 405, the vehicle determines whether the first gesture data needs to be converted according to the current driving state of the vehicle.
Step 406, the vehicle converts the first posture data to be converted, acquires second posture data of the vehicle in the current driving state, and corrects the first posture data according to the second posture data to obtain relative posture data.
In step 407, the vehicle recognizes the gesture according to the relative gesture data.
At step 408, the vr glasses send third pose information to the vehicle.
Step 409, the vehicle acquires third gesture information of the VR glasses, and renders a corresponding picture according to the third gesture information.
In step 410, the vehicle superimposes the recognition result of the gesture to the rendering screen, and returns the superimposed rendering screen to the VR glasses.
In step 4011, vr glasses receive the rendered frame sent by the vehicle end.
Step 4012, the VR glasses display the rendered frame on the VR glasses screen.
The system of on-vehicle VR gesture that this disclosure provided, main technical scheme includes: the fingerstall collects first posture data of fingers and sends the first posture data and identification information of the fingerstall to the vehicle, the vehicle judges whether the first posture data is required to be subjected to data correction according to the current driving state of the vehicle, if so, the first posture data is corrected according to the current state data of the vehicle to obtain relative posture data, if not, the first posture data is directly converted into the relative posture data, and the vehicle recognizes gesture postures of a user based on the relative posture data. Compared with the related art, the embodiment of the application collects the gesture information of the finger in real time by adopting the fingerstall, processes and identifies the gesture information of the finger in the vehicle, and realizes low-cost and accurate hand gesture identification on the basis of not using a vehicle camera.
Corresponding to the vehicle-mounted VR gesture recognition method, the invention further provides a vehicle-mounted VR gesture recognition device. Since the device embodiment of the present invention corresponds to the above-mentioned method embodiment, details not disclosed in the device embodiment may refer to the above-mentioned method embodiment, and details are not described in detail in the present invention.
Fig. 7 is a schematic structural diagram of a device for recognizing gesture gestures of a vehicle-mounted VR according to an embodiment of the present disclosure, where, as shown in fig. 7, the device is used for a vehicle and includes:
a receiving unit 51, configured to receive identification information of a finger stall and first gesture data of the finger stall sent by a finger stall collecting device, where the finger stall collecting device includes at least three finger stalls, and each finger stall corresponds to one piece of identification information;
a judging unit 52, configured to judge whether the first posture data needs to be converted according to a current driving state of the vehicle;
an obtaining unit 53, configured to obtain second posture data of the vehicle in the current driving state when the first posture data needs to be converted;
a correction unit 54, configured to correct the first posture data according to the second posture data, so as to obtain relative posture data;
And the recognition unit 55 is used for recognizing the gesture according to the relative gesture data.
The device of on-vehicle VR gesture that this disclosure provided, main technical scheme includes: the fingerstall collects first posture data of fingers and sends the first posture data and identification information of the fingerstall to the vehicle, the vehicle judges whether the first posture data is required to be subjected to data correction according to the current driving state of the vehicle, if so, the first posture data is corrected according to the current state data of the vehicle to obtain relative posture data, if not, the first posture data is directly converted into the relative posture data, and the vehicle recognizes gesture postures of a user based on the relative posture data. Compared with the related art, the embodiment of the application collects the gesture information of the finger in real time by adopting the fingerstall, processes and identifies the gesture information of the finger in the vehicle, and realizes low-cost and accurate hand gesture identification on the basis of not using a vehicle camera.
Further, in a possible implementation manner of this embodiment, as shown in fig. 8, the correction unit 54 is further configured to perform a difference between the second gesture data and the first gesture data to obtain the relative gesture data.
Further, in a possible implementation manner of this embodiment, as shown in fig. 8, the apparatus further includes:
a detecting unit 56, configured to detect, before the identifying unit identifies the gesture according to the relative gesture data, whether the relative gesture data includes a preset number of fingerstall identification information;
a discarding unit 57 configured to discard the first posture data when the relative posture data does not contain the preset number of fingerstall identification information;
the recognition unit 55 is further configured to perform gesture recognition on the relative gesture data at the same acquisition time when the relative gesture data includes the preset number of fingerstall identification information.
Further, in one possible implementation manner of this embodiment, as shown in fig. 8, the identifying unit 55 includes:
the calculating module 551 is configured to calculate the pointing gesture and the acceleration according to the previous relative gesture data and the current relative gesture data;
a first determining module 552, configured to determine a pointing posture of the palm according to the pointing posture;
a second determining module 553 for determining a displacement characteristic of the hand based on the acceleration.
Further, in one possible implementation manner of this embodiment, as shown in fig. 8, the determining unit 52 includes:
a third confirmation module 521 for confirming a current driving state of the vehicle based on the gear of the vehicle and the current speed per hour;
a fourth determining module 522 is configured to determine that the first gesture data needs to be converted when the current driving state is determined to be a running state.
Further, in a possible implementation manner of this embodiment, as shown in fig. 8, the apparatus further includes:
a determining unit 58 for directly determining the first posture data as the relative posture data when it is determined that the current driving state is a stopped state.
Further, in a possible implementation manner of this embodiment, as shown in fig. 8, the apparatus further includes:
a rendering unit 59, configured to render a corresponding picture according to third posture information of VR glasses after obtaining the third posture information;
and the superimposing unit 510 is configured to superimpose the recognition result for the gesture on the rendering screen, and return the superimposed rendering screen to the VR glasses.
Fig. 9 is a schematic structural diagram of another device for recognizing gesture gestures of a vehicle-mounted VR, which is provided by an embodiment of the present disclosure, where the device is used for a finger stall collecting device, as shown in fig. 9, and includes:
A transmitting unit 61 for transmitting identification information of at least three started fingerstalls to a vehicle in response to communication establishment of a wireless connection network with the vehicle;
an acquisition unit 62 for acquiring first posture data of the finger based on the at least three started finger cuffs;
and a sending unit 63, configured to send the collected first gesture information and the corresponding identification information to the vehicle.
The device of on-vehicle VR gesture that this disclosure provided, main technical scheme includes: the fingerstall collects first posture data of fingers and sends the first posture data and identification information of the fingerstall to the vehicle, the vehicle judges whether the first posture data is required to be subjected to data correction according to the current driving state of the vehicle, if so, the first posture data is corrected according to the current state data of the vehicle to obtain relative posture data, if not, the first posture data is directly converted into the relative posture data, and the vehicle recognizes gesture postures of a user based on the relative posture data; compared with the related art, the embodiment of the application collects the gesture information of the finger in real time by adopting the fingerstall, processes and identifies the gesture information of the finger in the vehicle, and realizes low-cost and accurate hand gesture identification on the basis of not using a vehicle camera.
According to an embodiment of the disclosure, the disclosure further provides a system for recognizing the gesture of the vehicle-mounted VR.
Referring to fig. 10, fig. 10 is a system structure diagram of a vehicle-mounted VR gesture provided by an embodiment of the present application, where the system includes: a vehicle 71, a finger cuff collection device 72, wherein:
the finger stall data acquisition equipment is used for responding to the communication establishment between a wireless connection network and a vehicle and sending identification information of at least three started finger stalls to the vehicle;
the vehicle 71 is configured to receive identification information of at least three started finger cuffs sent by the finger cuff collection device 72;
the finger cuff data acquisition device 72 is further configured to acquire first gesture data of a finger based on the at least three started finger cuffs, and send the acquired first gesture information and corresponding identification information to the vehicle 71;
the vehicle 71 is configured to receive the identification information of the finger stall and the collected first gesture data sent by the finger stall collecting device 72, determine, according to a current driving state of the vehicle, whether the first gesture data needs to be converted, convert the first gesture data that needs to be converted, obtain second gesture data of the vehicle in the current driving state, correct the first gesture data according to the second gesture data, obtain relative gesture data, and identify a gesture according to the relative gesture data.
The system further comprises VR glasses 73, wherein the vehicle 71 is further configured to:
VR glasses 73 for transmitting third posture information to vehicle 71;
the vehicle 71 is further configured to obtain third pose information of the VR glasses, and render a corresponding frame according to the third pose information;
the vehicle 71 is further configured to superimpose the recognition result for the gesture on the rendered image, and return the superimposed rendered image to the VR glasses 73;
the VR glasses 73 are further configured to receive a rendering frame sent by the vehicle 71, and display the rendering frame on a VR glasses screen.
The foregoing explanation of the method embodiment is also applicable to the apparatus of this embodiment, and the principle is the same, and this embodiment is not limited thereto.
According to an embodiment of the disclosure, the disclosure further provides an electronic device.
Fig. 11 illustrates a schematic block diagram of an example electronic device 800 that can be used to implement embodiments of the present disclosure. Electronic devices are intended to represent various forms of digital computers, such as laptops, desktops, workstations, personal digital assistants, servers, blade servers, mainframes, and other appropriate computers. The electronic device may also represent various forms of mobile devices, such as personal digital processing, cellular telephones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions, are meant to be exemplary only, and are not meant to limit implementations of the disclosure described and/or claimed herein.
As shown in fig. 11, the apparatus 800 includes a computing unit 801 that can perform various appropriate actions and processes according to a computer program stored in a ROM (Read-Only Memory) 802 or a computer program loaded from a storage unit 808 into a RAM (Random Access Memory ) 803. In the RAM 803, various programs and data required for the operation of the device 800 can also be stored. The computing unit 801, the ROM 802, and the RAM 803 are connected to each other by a bus 804. An I/O (Input/Output) interface 805 is also connected to bus 804.
Various components in device 800 are connected to I/O interface 805, including: an input unit 806 such as a keyboard, mouse, etc.; an output unit 807 such as various types of displays, speakers, and the like; a storage unit 808, such as a magnetic disk, optical disk, etc.; and a communication unit 809, such as a network card, modem, wireless communication transceiver, or the like. The communication unit 809 allows the device 800 to exchange information/data with other devices via a computer network such as the internet and/or various telecommunication networks.
The computing unit 801 may be a variety of general and/or special purpose processing components having processing and computing capabilities. Some examples of computing unit 801 include, but are not limited to, a CPU (Central Processing Unit ), GPU (Graphic Processing Units, graphics processing unit), various dedicated AI (Artificial Intelligence ) computing chips, various computing units running machine learning model algorithms, DSPs (Digital Signal Processor, digital signal processors), and any suitable processors, controllers, microcontrollers, and the like. The computing unit 801 performs the various methods and processes described above, such as the recognition method of the vehicle-mounted VR gesture. For example, in some embodiments, the method of recognition of an in-vehicle VR gesture may be implemented as a computer software program tangibly embodied on a machine-readable medium, such as the storage unit 808. In some embodiments, part or all of the computer program may be loaded and/or installed onto device 800 via ROM 802 and/or communication unit 809. When a computer program is loaded into RAM 803 and executed by computing unit 801, one or more steps of the methods described above may be performed. Alternatively, in other embodiments, the computing unit 801 may be configured to perform the aforementioned recognition method of the vehicle-mounted VR gesture gestures in any other suitable manner (e.g., by means of firmware).
Various implementations of the systems and techniques described here above may be implemented in digital electronic circuitry, integrated circuit System, FPGA (Field Programmable Gate Array ), ASIC (Application-Specific Integrated Circuit, application-specific integrated circuit), ASSP (Application Specific Standard Product, special-purpose standard product), SOC (System On Chip ), CPLD (Complex Programmable Logic Device, complex programmable logic device), computer hardware, firmware, software, and/or combinations thereof. These various embodiments may include: implemented in one or more computer programs, the one or more computer programs may be executed and/or interpreted on a programmable system including at least one programmable processor, which may be a special purpose or general-purpose programmable processor, that may receive data and instructions from, and transmit data and instructions to, a storage system, at least one input device, and at least one output device.
Program code for carrying out methods of the present disclosure may be written in any combination of one or more programming languages. These program code may be provided to a processor or controller of a general purpose computer, special purpose computer, or other programmable data processing apparatus such that the program code, when executed by the processor or controller, causes the functions/operations specified in the flowchart and/or block diagram to be implemented. The program code may execute entirely on the machine, partly on the machine, as a stand-alone software package, partly on the machine and partly on a remote machine or entirely on the remote machine or server.
In the context of this disclosure, a machine-readable medium may be a tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. The machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. The machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of a machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer diskette, a hard disk, RAM, ROM, EPROM (Electrically Programmable Read-Only-Memory, erasable programmable read-Only Memory) or flash Memory, an optical fiber, a CD-ROM (Compact Disc Read-Only Memory), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
To provide for interaction with a user, the systems and techniques described here can be implemented on a computer having: a display device (e.g., CRT (Cathode-Ray Tube) or LCD (Liquid Crystal Display ) monitor) for displaying information to a user; and a keyboard and pointing device (e.g., a mouse or trackball) by which a user can provide input to the computer. Other kinds of devices may also be used to provide for interaction with a user; for example, feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form, including acoustic input, speech input, or tactile input.
The systems and techniques described here can be implemented in a computing system that includes a background component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front component (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such background, middleware, or front components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: LAN (Local Area Network ), WAN (Wide Area Network, wide area network), internet and blockchain networks.
The computer system may include a client and a server. The client and server are typically remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other. The server can be a cloud server, also called a cloud computing server or a cloud host, and is a host product in a cloud computing service system, so that the defects of high management difficulty and weak service expansibility in the traditional physical hosts and VPS service ("Virtual Private Server" or simply "VPS") are overcome. The server may also be a server of a distributed system or a server that incorporates a blockchain.
It should be noted that, artificial intelligence is a subject of studying a certain thought process and intelligent behavior (such as learning, reasoning, thinking, planning, etc.) of a computer to simulate a person, and has a technology at both hardware and software level. Artificial intelligence hardware technologies generally include technologies such as sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, big data processing, and the like; the artificial intelligence software technology mainly comprises a computer vision technology, a voice recognition technology, a natural language processing technology, a machine learning/deep learning technology, a big data processing technology, a knowledge graph technology and the like.
It should be appreciated that various forms of the flows shown above may be used to reorder, add, or delete steps. For example, the steps recited in the present disclosure may be performed in parallel or sequentially or in a different order, provided that the desired results of the technical solutions of the present disclosure are achieved, and are not limited herein.
The above detailed description should not be taken as limiting the scope of the present disclosure. It will be apparent to those skilled in the art that various modifications, combinations, sub-combinations and alternatives are possible, depending on design requirements and other factors. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present disclosure are intended to be included within the scope of the present disclosure.

Claims (14)

1. The vehicle-mounted VR gesture recognition method is characterized by comprising the following steps of:
receiving identification information of a fingerstall and first posture data of the fingerstall, which are sent by fingerstall acquisition equipment, wherein the fingerstall acquisition equipment comprises at least three fingerstalls, and each fingerstall corresponds to one piece of identification information;
judging whether the first gesture data need to be converted according to the current driving state of the vehicle;
if the first posture data are required to be converted, second posture data of the vehicle in the current driving state are obtained, and the first posture data are corrected according to the second posture data to obtain relative posture data;
and recognizing the gesture according to the relative gesture data.
2. The method of claim 1, wherein the correcting the first gesture data according to the second gesture data to obtain the relative gesture data includes:
and carrying out difference on the second gesture data and the first gesture data to obtain the relative gesture data.
3. The method of recognition according to claim 1, wherein prior to recognizing a gesture from the relative gesture data, the method further comprises:
Detecting whether the relative gesture data contains preset number of fingerstall identification information;
discarding the relative gesture data if the relative gesture data does not contain the preset number of fingerstall identification information;
if the relative gesture data contains the preset number of fingerstall identification information, gesture recognition is performed on the relative gesture data under the same acquisition time.
4. A method of recognition as claimed in claim 3, wherein said recognizing the relative gesture data gesture for the same acquisition time comprises:
calculating the pointing gesture and the acceleration according to the last relative gesture data and the current relative gesture data;
according to the pointing gesture, determining the pointing gesture of the palm;
and determining the displacement characteristic of the hand according to the acceleration.
5. The method according to claim 1, wherein the determining whether the first posture data needs to be converted according to the current driving state of the vehicle includes:
confirming the current driving state of the vehicle based on the gear and the current speed per hour of the vehicle;
and if the current driving state is determined to be the running state, determining that the first gesture data need to be converted.
6. The identification method of claim 5, wherein the method further comprises:
and if the current driving state is determined to be a stop state, directly determining the first gesture data as the relative gesture data.
7. The identification method according to any one of claims 1-6, characterized in that the method further comprises:
after the third posture information of the VR glasses is obtained, rendering a corresponding picture according to the third posture information;
and superposing the recognition result aiming at the gesture into a rendering picture, and returning the superposed rendering picture to the VR glasses.
8. The vehicle-mounted VR gesture recognition method is characterized by comprising the following steps of:
transmitting identification information of at least three started fingerstalls to a vehicle in response to communication establishment of a wireless connection network with the vehicle;
collecting first gesture data of the finger based on the at least three started finger cuffs;
and sending the acquired first posture information and the corresponding identification information to the vehicle.
9. An on-vehicle VR gesture recognition device, characterized in that includes:
the receiving unit is used for receiving the identification information of the fingerstall and the first gesture data of the fingerstall, which are sent by the fingerstall acquisition equipment, wherein the fingerstall acquisition equipment comprises at least three fingerstalls, and each fingerstall corresponds to one identification information;
The judging unit is used for judging whether the first gesture data need to be converted according to the current driving state of the vehicle;
the acquisition unit is used for acquiring second posture data of the vehicle in the current driving state when the first posture data are required to be converted;
the correcting unit is used for correcting the first posture data according to the second posture data to obtain relative posture data;
and the identification unit is used for identifying the gesture according to the relative gesture data.
10. An on-vehicle VR gesture recognition device, characterized in that includes:
a transmitting unit for transmitting identification information of at least three started fingerstalls to a vehicle in response to communication establishment between a wireless connection network and the vehicle;
the acquisition unit is used for acquiring first gesture data of the finger based on the at least three started finger cuffs;
and the sending unit is used for sending the acquired first gesture information and the corresponding identification information to the vehicle.
11. A system for recognizing a gesture of a vehicle VR, the system comprising: vehicle, dactylotheca collection device, wherein:
the finger stall data acquisition equipment is used for responding to the communication establishment between a wireless connection network and a vehicle and sending identification information of at least three started finger stalls to the vehicle;
The vehicle is used for receiving the identification information of at least three started fingerstalls sent by the fingerstall acquisition equipment;
the finger stall data acquisition equipment is also used for acquiring first gesture data of fingers based on the at least three started finger stalls and sending the acquired first gesture information and corresponding identification information to the vehicle;
the vehicle is used for receiving the identification information of the fingerstall and the first posture data of the fingerstall sent by the fingerstall collecting equipment, judging whether the first posture data need to be converted according to the current driving state of the vehicle, converting the first posture data needing to be converted, acquiring second posture data of the vehicle in the current driving state, correcting the first posture data according to the second posture data, obtaining relative posture data, and identifying gesture postures according to the relative posture data.
12. The system of claim 11, further comprising VR glasses, wherein the vehicle is further configured to:
VR glasses for transmitting the third pose information to the vehicle;
the vehicle is further used for acquiring third posture information of the VR glasses and rendering a corresponding picture according to the third posture information;
The vehicle is further used for superposing the recognition result aiming at the gesture in the rendering picture and returning the superposed rendering picture to the VR glasses;
the VR glasses are also used for receiving the rendering pictures sent by the vehicle and displaying the rendering pictures in the VR glasses screen.
13. A vehicle comprising the on-board VR gesture recognition apparatus of claim 9 or claim 10.
14. An electronic device, comprising:
at least one processor; and
a memory communicatively coupled to the at least one processor; wherein,
the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-7 or claim 8.
CN202211213036.9A 2022-09-30 2022-09-30 Vehicle-mounted VR gesture recognition method, device and system and electronic equipment Pending CN117055722A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202211213036.9A CN117055722A (en) 2022-09-30 2022-09-30 Vehicle-mounted VR gesture recognition method, device and system and electronic equipment

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202211213036.9A CN117055722A (en) 2022-09-30 2022-09-30 Vehicle-mounted VR gesture recognition method, device and system and electronic equipment

Publications (1)

Publication Number Publication Date
CN117055722A true CN117055722A (en) 2023-11-14

Family

ID=88668013

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202211213036.9A Pending CN117055722A (en) 2022-09-30 2022-09-30 Vehicle-mounted VR gesture recognition method, device and system and electronic equipment

Country Status (1)

Country Link
CN (1) CN117055722A (en)

Similar Documents

Publication Publication Date Title
US11961297B2 (en) Terminal device, information processing device, object identifying method, program, and object identifying system
CN111649739B (en) Positioning method and device, automatic driving vehicle, electronic equipment and storage medium
CN112528850A (en) Human body recognition method, device, equipment and storage medium
CN113642431A (en) Training method and device of target detection model, electronic equipment and storage medium
CN113378712B (en) Training method of object detection model, image detection method and device thereof
CN111721305B (en) Positioning method and apparatus, autonomous vehicle, electronic device, and storage medium
CN113835570B (en) Control method, device, equipment, storage medium and program for display screen in vehicle
JP2016195323A (en) Information processing apparatus, information processing method, and program
CN117055722A (en) Vehicle-mounted VR gesture recognition method, device and system and electronic equipment
CN115134523B (en) Remote control method, remote control device, operation terminal, control device and storage medium
CN114013431B (en) Automatic parking control method and system based on user intention
CN111913574B (en) Method, apparatus, electronic device, and computer-readable medium for controlling device
CN113838477A (en) Packet loss recovery method and device for audio data packet, electronic equipment and storage medium
CN114299192A (en) Method, device, equipment and medium for positioning and mapping
CN113465611B (en) Navigation method, navigation device, electronic equipment and readable storage medium
CN220730766U (en) Sign language acquisition and recognition intelligent glove and conversion system
CN117008714A (en) Gesture recognition method and device, electronic equipment and storage medium
CN117011880A (en) Gesture recognition method and device, vehicle, electronic equipment and storage medium
CN117055723A (en) Vehicle-mounted VR gesture recognition method and device, electronic equipment and storage medium
CN115049725A (en) Display method, display device, electronic equipment and storage medium
KR20220056834A (en) Display method, display apparatus, device, computer readable storage medium and, and computer program
CN116597213A (en) Target detection method, training device, electronic equipment and storage medium
CN116894894A (en) Method, apparatus, device and storage medium for determining motion of avatar
CN117059085A (en) Vehicle lane changing method and device, vehicle and storage medium
CN114937185A (en) Image sample acquisition method and device, electronic equipment and storage medium

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination