CN114924647A - Vehicle control method, device, control equipment and medium based on gesture recognition - Google Patents
Vehicle control method, device, control equipment and medium based on gesture recognition Download PDFInfo
- Publication number
- CN114924647A CN114924647A CN202210588028.6A CN202210588028A CN114924647A CN 114924647 A CN114924647 A CN 114924647A CN 202210588028 A CN202210588028 A CN 202210588028A CN 114924647 A CN114924647 A CN 114924647A
- Authority
- CN
- China
- Prior art keywords
- gesture
- vehicle
- user
- database
- intention
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/017—Gesture based interaction, e.g. based on a set of recognized hand gestures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/50—Information retrieval; Database structures therefor; File system structures therefor of still image data
- G06F16/58—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
- G06F16/583—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually using metadata automatically derived from the content
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/74—Image or video pattern matching; Proximity measures in feature spaces
- G06V10/761—Proximity, similarity or dissimilarity measures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/20—Movements or behaviour, e.g. gesture recognition
- G06V40/28—Recognition of hand or arm movements, e.g. recognition of deaf sign language
Abstract
The invention provides a vehicle control method based on gesture recognition, which is applied to a cloud platform and comprises the following steps: receiving a gesture to be recognized uploaded by a vehicle; comparing the characteristic similarity of the gesture to be recognized with a pre-stored personalized gesture database associated with the user to which the vehicle belongs; when the feature similarity comparison is successful, outputting a comparison result containing the gesture type of the gesture to be recognized and the user intention represented by the gesture; issuing the comparison result to a vehicle for the vehicle to control according to the user intention in the comparison result; the pre-stored personalized gesture database at least comprises a plurality of groups of first mapping relations, wherein one group of first mapping relations comprise a gesture type representing the intention of a user and a plurality of gestures with different forms belonging to the same gesture type, and the gestures with different forms are provided by a plurality of test subjects with different heights, weights, ages and sexes.
Description
Technical Field
The invention is used in the field of intelligent electric automobiles, and particularly relates to a vehicle control method, a vehicle control device, control equipment and a medium based on gesture recognition.
Background
The vehicle gesture recognition method is mainly applied to scenes such as video and audio entertainment and skylight control in a vehicle. In the prior art, a gesture recognition function is mounted on a vehicle end, and the following 2 methods are generally used for realizing the function, one of which is: pre-storing a database containing gesture types and user intentions at a vehicle end, and searching gestures in the pre-stored database after the vehicle acquires the gestures to analyze the user intentions; the other is as follows: after the vehicle owner purchases the vehicle, some gestures are input according to the preference of the vehicle owner according to system navigation, the vehicle forms a personalized database based on the gestures input by the user, and the vehicle is controlled by utilizing the database. These implementations described above have the following problems:
1. the accuracy of gesture recognition is related to the related hardware performance of the vehicle, the problem of poor recognition result exists when a gesture recognition module with low price is selected, and the hardware cost with good recognition result is very high;
2. after the vehicle end realizes gesture recognition, the vehicle end is difficult to continuously optimize a pre-stored database or a generated database, so that the personalization degree of the prior art is not high.
Disclosure of Invention
Aiming at the existing problems, the invention provides a vehicle control method, a vehicle control device and a vehicle control medium based on gesture recognition, which are used for improving the accuracy of controlling a vehicle by gesture recognition, reducing the cost of a single vehicle and improving the degree of individuation.
The invention is realized by the following technical scheme:
the invention provides a vehicle control method based on gesture recognition, which is applied to a cloud platform and comprises the following steps:
receiving a gesture to be recognized uploaded by a vehicle;
comparing the characteristic similarity of the gesture to be recognized with a pre-stored personalized gesture database associated with the user to which the vehicle belongs;
when the feature similarity comparison is successful, outputting a comparison result containing the gesture type of the gesture to be recognized and the user intention represented by the gesture;
issuing the comparison result to a vehicle for the vehicle to control according to the user intention in the comparison result;
the pre-stored personalized gesture database at least comprises a plurality of groups of first mapping relations, wherein one group of first mapping relations comprise a gesture type representing the intention of a user and a plurality of gestures with different forms belonging to the same gesture type, and the gestures with different forms are provided by a plurality of test subjects with different heights, weights, ages and sexes.
Preferably, the pre-stored personalized gesture database further comprises at least one set of second mapping relationships, the set of second mapping relationships comprising a gesture type characterizing a user's intention and at least one morphological gesture provided by a user to whom the vehicle belongs.
Preferably, the method further comprises:
receiving a control result fed back by the vehicle based on the identification result;
if the control result shows that the vehicle performs reverse operation according to the intention opposite to the user intention within the preset time after being controlled according to the user intention, the fact that the comparison result is wrong is confirmed, and the selected target form gesture is marked once when the feature similarity is compared;
deleting the target form gesture from a pre-stored personalized gesture database when the marking times of the target form gesture reach preset times;
the target morphological gesture refers to a morphological gesture with the feature similarity exceeding a preset similarity value.
Preferably, the method further comprises:
when the feature similarity comparison fails, feeding back feedback information of the failure comparison to the vehicle;
and in the preset time after the feedback information is sent, if the information that the vehicle executes the operation based on the user trigger is received, the operation executed by the vehicle is divided into a new user intention, the gesture to be recognized is divided into a new gesture type associated with the new user intention, and the new gesture type and the gesture to be recognized are recorded into a pre-stored personalized gesture database as a group of second mapping relation.
Preferably, the user intent is at least: the method is characterized in that the user's operation intention on a front hatch cover, a skylight, a back door, a vehicle door and a vehicle machine of the vehicle is represented.
Preferably, the method further comprises:
aiming at the same vehicle type, comparing pre-stored personalized gesture databases related to users to which all vehicles belong;
if the same form of gesture is deleted in the pre-stored personalized gesture databases exceeding the first preset proportion number, the same form of gesture is deleted in the pre-stored personalized gesture databases associated with the users belonging to all vehicles.
Preferably, the method further comprises:
if the same group of mapping relations containing the new gesture types and the new user intentions are added in the pre-stored personalized gesture databases with the number exceeding the second preset proportion, the group of mapping relations are added in the pre-stored personalized gesture databases associated with the users to which all the vehicles belong, and the gestures of the users to which all the vehicles belong are divided into a plurality of morphological gestures in the group of mapping relations.
The invention also provides a vehicle control device based on gesture recognition, which comprises:
the first receiving module is used for receiving the gestures to be recognized uploaded by the vehicle;
the comparison module is used for comparing the characteristic similarity of the gesture to be recognized with a pre-stored personalized gesture database associated with the user to which the vehicle belongs;
the output module is used for outputting a comparison result containing the gesture type of the gesture to be recognized and the user intention represented by the gesture when the feature similarity comparison is successful;
the issuing module is used for issuing the comparison result to a vehicle for the vehicle to control according to the user intention in the comparison result;
the pre-stored personalized gesture database at least comprises a plurality of groups of first mapping relations, wherein one group of first mapping relations comprise one gesture type representing user intention and a plurality of different-form gestures belonging to the same gesture type, and the plurality of different-form gestures are provided by a plurality of test objects with different heights, weights, ages and sexes.
Preferably, the pre-stored personalized gesture database further comprises at least one set of second mapping relations, wherein the set of second mapping relations comprises a gesture type representing the intention of a user and at least one form gesture provided by the user to which the vehicle belongs.
Preferably, the apparatus further comprises:
the second receiving module is used for receiving a control result fed back by the vehicle based on the identification result;
the marking module is used for confirming that the comparison result is wrong and marking the selected target form gesture once when the feature similarity is compared if the control result shows that the vehicle is controlled according to the user intention and then performs reverse operation according to the intention opposite to the user intention in the preset time;
the deleting module is used for deleting the target form gesture from a pre-stored personalized gesture database when the marking times of the target form gesture reach the preset times;
the target morphological gesture refers to a morphological gesture with the feature similarity exceeding a preset similarity value.
Preferably, the apparatus further comprises:
the feedback module is used for feeding back feedback information of the failed comparison to the vehicle when the feature similarity comparison fails;
and the newly-added module is used for dividing the operation executed by the vehicle into a new user intention and dividing the gesture to be recognized into a new gesture type associated with the new user intention if receiving the information that the vehicle executes the operation based on the user trigger within the preset time after the feedback information is sent, and then recording the new gesture type and the gesture to be recognized into a group of second mapping relations into a pre-stored personalized gesture database.
Preferably, the user intent is at least: and characterizing the operation intention of a user on a front hatch cover, a skylight, a back door, a vehicle door and a vehicle machine of the vehicle.
Preferably, the apparatus further comprises:
the comparison module is used for comparing pre-stored personalized gesture databases related to users belonging to all vehicles aiming at the same vehicle type;
and the first updating module is used for deleting the gestures in the same form in the pre-stored personalized gesture database which exceeds the first preset proportion number, and deleting the gestures in the pre-stored personalized gesture database which is associated with all the users to which the vehicles belong.
Preferably, the apparatus further comprises:
and the second updating module is used for adding the group of mapping relations in the pre-stored personalized gesture database associated with the users to which all vehicles belong if the same group of mapping relations containing the new gesture types and the new user intentions are added in the pre-stored personalized gesture database with the number exceeding the second preset proportion number, and dividing the gestures of the users to which all vehicles belong into a plurality of morphological gestures in the group of mapping relations.
The invention also provides a control device comprising a processor, a memory and a program or instructions stored on the memory and executable on the processor, the program or instructions when executed by the processor implementing the steps of the gesture recognition based vehicle control method as described above.
The invention also provides a readable storage medium on which a program or instructions are stored, which program or instructions, when executed by a processor, implement the steps of the gesture recognition based vehicle control method as described above.
The beneficial effects of the invention are as follows:
the original gesture recognition process applied to the vehicle end is realized by placing the gesture recognition process on the cloud platform end, and the vehicle end does not need to be provided with a controller with high computing capacity and a camera with high sensing precision, so that the hardware cost of the vehicle can be reduced; in addition, the communication real-time performance between the vehicle and the cloud platform is guaranteed based on the low-time-delay high-transmission-rate characteristic of 5G communication.
The database with large-scale samples does not need to be stored at the vehicle end, the personalized gesture database based on different vehicle types and different user groups is pre-placed at the cloud platform end, real acquisition samples of the personalized gesture database can be gradually accumulated along with the use of the vehicle, and the accuracy of gesture recognition on gestures provided by the vehicle based on the personalized gesture database is gradually improved along with the increase of the number of the real acquisition samples.
The cloud platform can perform vehicle gesture recognition self-learning based on big data, and when the user intention is inconsistent with the recognition result of gesture recognition, the cloud platform can analyze the user behavior according to the control result fed back by the vehicle to obtain a deviation point and complete self-learning, so that the personalized gesture database of the user to which the vehicle belongs is optimized, and finally a personalized gesture database unique to the user to which each vehicle belongs is formed. After each optimization, the accuracy of gesture recognition by using the optimized database is improved on the basis of the previous step; and by utilizing the self-learning optimization function, the mapping relation between the new gesture and the user intention of the user in a new scene during vehicle control execution by utilizing the gesture recognition can be self-learned, and the scene coverage of the function is increased.
Drawings
FIG. 1 is a flow chart of a vehicle method based on gesture recognition in an embodiment of the present invention;
FIG. 2 is a block diagram of a gesture recognition system implemented with a vehicle end and a cloud end;
fig. 3 is a flowchart of the gesture uploading performed by the vehicle end and the gesture recognition performed by the cloud end in this embodiment;
fig. 4 is a schematic diagram of a gesture database based on vehicle types obtained by a preliminary test in the present embodiment;
fig. 5 is a schematic diagram illustrating that the cloud platform updates all personalized gesture databases of the same vehicle type in this embodiment;
fig. 6 is a block diagram of a vehicle control device based on gesture recognition in the present embodiment.
Detailed Description
The invention is further described below. The principles and features of this invention are described below in conjunction with the following drawings, which are given by way of illustration only and not by way of limitation.
The invention provides a vehicle control method based on gesture recognition, in order to realize the method, a cloud end and a vehicle end need to participate together, as shown in fig. 2, a man-machine interaction module at a wheel end is connected with a camera module, the camera module is used for collecting and preprocessing a user gesture image, the man-machine interaction module is used for being connected with a cloud platform through a 5G communication module, and the man-machine interaction module is further used for executing corresponding control on a vehicle according to the cloud platform and instructions from other ECUs of the vehicle.
Referring to fig. 1 and 3, in this embodiment, the vehicle control method based on gesture recognition is applied to a cloud platform, and includes:
and step S101, receiving the gesture to be recognized uploaded by the vehicle.
For each vehicle, connection with a cloud platform needs to be established, a camera module is arranged at the vehicle end and is installed in each area of the vehicle, where user gestures need to be collected, such as the front side of a column A of the vehicle, the front bumper, the inner side of a windshield, a column C of the vehicle and the like, and the camera obtains the gesture change state of the user in a period of time by shooting videos. The camera module is an integrated device integrating an information acquisition unit and an information processing unit, the information processing unit detects the duration of user gesture video information acquired by the information acquisition unit to identify a scene triggered by a user by mistake, and when the duration of the user gesture video information is greater than or equal to 2s, the gesture input by the user is determined to be real user input; the information processing unit obtains a user gesture image meeting the requirements through image frame interception, and the information acquisition unit uploads the user gesture image to the cloud platform through a human-computer interaction module of the vehicle after receiving the user gesture image. The interpersonal interaction module establishes communication with the cloud platform based on the 5G communication technology.
For the cloud platform, the gesture to be recognized uploaded by each vehicle with which communication is established in advance can be acquired.
And S102, comparing the gesture to be recognized with the feature similarity of a pre-stored personalized gesture database associated with the vehicle user.
The pre-stored personalized gesture database at least comprises a plurality of groups of first mapping relations, wherein one group of first mapping relations comprise one gesture type representing user intention and a plurality of different-form gestures belonging to the same gesture type, and the plurality of different-form gestures are provided by a plurality of test objects with different heights, weights, ages and sexes.
The plurality of groups of first mapping relations can form a common part in the pre-stored personalized gesture database. Meanwhile, in a pre-stored personalized gesture database corresponding to the same vehicle type, the initial data of the common part is the same; in the pre-stored personalized gesture database corresponding to different vehicle types, the initial data of the common part is different.
The initial data of the common part is obtained by sample test, specifically, as shown in fig. 4, an engineer performs the sample test by a real vehicle or a bench, and the basic idea of the test is as follows: the method comprises the steps of dividing various vehicle types (such as large-scale, medium-scale and compact SUVs, large-scale, medium-scale and compact cars and other vehicles) with different parameters, carrying out the same gesture (such as OK, V, fist making, five-finger closing and opening and the like) through the crowd with different heights, weights, ages and sexes, collecting a gesture photo of the person and recording the information of a tester. The data of the common part may be subjected to update processing such as deletion, addition, and the like according to big data analysis, and the process of the deletion and addition processing is described in the following embodiments.
The specific process of comparing the feature similarity of the gesture to be recognized is to extract the features of the gesture to be recognized, compare the features with the features of various forms of gestures in the personalized gesture database, if the feature similarity of a certain form of gesture and the feature similarity of the gesture to be recognized exceed a preset value, determine the form of gesture to be the same as the gesture to be recognized, and determine the form of gesture to be a required target form of gesture.
In this embodiment, the specific principle of comparing the feature similarity can be implemented by using the related technology in the prior art.
Step S103, when the feature similarity comparison is successful, outputting a comparison result containing the gesture type of the gesture to be recognized and the user intention represented by the gesture.
The user intent as described above is at least: the method is characterized in that the user's operation intention on a front hatch cover, a skylight, a back door, a vehicle door and a vehicle machine of the vehicle is represented. In this embodiment, the user intention is not limited to the control of the above components, but may also be used in other situations on the vehicle that involve user manipulation, such as air conditioning control, vehicle start, shut down, and other controls.
And step S104, issuing the comparison result to a vehicle for the vehicle to control according to the user intention in the comparison result.
Through the steps S101 to S104 in this embodiment, when performing gesture recognition, the cloud platform can perform feature comparison on the same gesture and different form gestures corresponding to different people, so as to reduce the probability of unsuccessful comparison or wrong comparison caused by individual differences when different people draw the same gesture, thereby improving the accuracy of vehicle control. Meanwhile, the gesture recognition is applied to the cloud, gesture recognition is not needed at the vehicle end, a device for performing gesture recognition at the vehicle end can be omitted, and the hardware cost of the vehicle end is reduced.
In order to detect the accuracy of the vehicle after performing the comparison, the present embodiment further includes the following steps S105 to S107 on the basis of the number of kangaroos being less than S104 to S105.
And step S105, receiving a control result fed back by the vehicle based on the identification result.
The control result comprises two types, wherein the first control result is as follows: after the vehicle is operated according to the user intention in the comparison result, reverse operation is not performed according to the intention opposite to the user intention in preset time; the second control result is: and performing reverse operation for the vehicle according to the intention opposite to the user intention within a preset time after the vehicle is operated according to the user intention, wherein the user intention in the comparison result is that the user desires to open the electric front hatch, and the vehicle performs the action of closing the electric front hatch within 10s under the control of the user after the front hatch is opened.
The first operation result shows that the vehicle executes the operation as really expected by the user, and the comparison result is correct; the second kind of manipulation result indicates that the manipulation performed according to the user's intention in the comparison result is not really desired by the user, indicating that the comparison result is deviated or wrong.
And when the control result is the first control result, after the cloud platform reads the information of the result, ignoring the result and not performing any processing.
And step S106, if the control result shows that the vehicle is controlled according to the user intention and then carries out reverse operation according to the intention opposite to the user intention in the preset time, the comparison result is confirmed to be wrong, and the selected target form gesture is marked once when the feature similarity is compared.
The target form gesture refers to a form gesture, when the feature similarity comparison is performed, the feature similarity of the compared gesture and the gesture to be recognized exceeds a preset similarity value.
And S107, deleting the target form gesture from a pre-stored personalized gesture database when the marking times of the target form gesture reach the preset times.
The above steps S101 to S107 will be described by way of example with reference to fig. 5. For example, when a user of a certain vehicle type 1 makes a gesture a, and after the gesture characteristics are compared, the gesture a and the gesture type 1 in the vehicle type 1 are determined to have a characteristic similarity of more than 90%, and the gesture type 2 is taken as a target gesture to generate a comparison result. And after the vehicle executes the control according to the comparison result, feeding back the second control result. The cloud platform is receiving this second kind and is controlled the result after, and the affirmation is compared the failure, carries out once with this form gesture 2 and marks, if discover that the accumulative mark number of times of this form gesture 2 has reached 2, then affirms this form gesture and causes gesture A's misidentification easily, so carry out the deletion processing with this form gesture 2, avoid being qualified for the next time again when discerning this gesture A misidentification, improve gesture recognition accuracy, and then improve the vehicle and control the precision.
In conjunction with fig. 5, after performing deletion of one or more morphological gestures in a single personalized gesture database, the cloud platform further needs to perform:
step S201, aiming at the same vehicle type, comparing pre-stored personalized gesture databases related to users to which all vehicles belong.
In this step, the purpose of comparison is to compare all the gestures in each pre-stored personalized gesture database to screen out a pre-stored personalized gesture database for deleting the same gesture, and the gesture in the pre-stored personalized gesture databases is deleted, which indicates that the gesture in the form has a high probability of causing gesture recognition errors.
Step S202, if the same form of gesture is deleted from the pre-stored personalized gesture databases with the number exceeding the first preset proportion, the same form of gesture is deleted from the pre-stored personalized gesture databases associated with all the users of the vehicles.
In the step, the form gesture in all the pre-stored personalized gesture databases is deleted as a remedial measure, so that the gesture error recognition caused by the form gesture in the next time is avoided.
After the step S102 of comparing the feature similarity, the method further includes:
and step S108, when the feature similarity comparison fails, feeding back feedback information of the failure comparison to the vehicle.
The feedback information represents the target morphological gestures of which the feature similarity exceeds the preset similarity and which are not found and are to be recognized.
Step S109, in a preset time after the feedback information is sent, if information that the vehicle executes the operation based on the user trigger is received, the operation executed by the vehicle is divided into a new user intention, the gesture to be recognized is divided into a new gesture type associated with the new user intention, and the new gesture type and the gesture to be recognized are recorded into a pre-stored personalized gesture database as a group of second mapping relation.
The step S108 is performed for the purpose of, in some cases, regarding a particular gesture as a trend gesture within a period of time, which may never occur before, and thus the gesture may not be recorded in the original database of pre-stored personalized gestures, and for this purpose, the step S109 needs to be performed to add and update such gesture into the database of pre-stored personalized gestures.
In this embodiment, when the vehicle leaves the factory, each vehicle includes only the common part formed by the plurality of sets of the first mapping relationships. With the use of the vehicle, after the cloud platform increases the data by using the big data, the pre-stored personalized gesture database further comprises at least one group of second mapping relations, and the group of second mapping relations comprise a gesture type representing the intention of a user and at least one form gesture provided by the user to which the vehicle belongs. At least one group of second mapping relations form a private part in the pre-stored personalized gesture database.
After the actions of the steps S107 and S109, the mapping relationships stored in the pre-stored personalized gesture database of the user to which each vehicle belongs are not completely the same, and the corresponding form gestures under the same set of mapping relationships are not completely the same.
After the data of the pre-stored personalized gesture database is updated, the cloud platform carries out judgment on all the pre-stored personalized gesture databases, the same group of mapping relations are increased in the predictive personalized gesture recognition database with a certain proportion, the group of mapping relations on the surface are concerned by more users, at the moment, all the pre-stored personalized gesture databases can be updated, gesture recognition is carried out on the users who use the gesture for the first time and expect to carry out similar vehicle control, and user use experience is improved. For this reason, in this embodiment, it is further necessary to perform:
firstly, executing the step S201;
and step S203 is executed again, if the same set of mapping relationships including the new gesture type and the new user intention are added to the pre-stored personalized gesture databases exceeding the second predetermined proportion number, the set of mapping relationships is added to the pre-stored personalized gesture database associated with the users to which all the vehicles belong, and the gestures of the users to which all the vehicles belong are classified into a plurality of morphological gestures in the set of mapping relationships.
The specific values of the second predetermined ratio amount in step S203 and the first predetermined ratio amount in step S202 can be defined based on practical situations, such as setting to 10%, 30%, 80%, etc. For example, for a certain vehicle enterprise, 10 thousands of vehicles that have established connection with the cloud platform, and if the same set of mapping relationships is added to 30% of the vehicles, the same set of mapping relationships is added to the pre-stored personalized gesture database corresponding to the 10 thousands of vehicles. Thus, when the new gesture is first uploaded by the remaining 7 thousands of vehicles, the new set of mapping relationships can be used to identify the user's intention. Therefore, after the step S203 is executed, the set of mapping relationships is stored in the pre-stored personalized gesture database of the user to which each vehicle belongs, and the cloud platform can serve all vehicles by using the set of mapping relationships.
After the actions of steps S201 to S203, the mapping relationship groups of the common part in the pre-stored personalized gesture database of the user to which each vehicle belongs are completely the same, and the corresponding form gestures under the same mapping relationship group are also completely the same.
According to the technical scheme, the original gesture recognition process applied to the vehicle end is realized by placing the gesture recognition process on the cloud platform end, and the vehicle end does not need to be provided with a controller with high computing capacity and a camera with high sensing precision, so that the hardware cost of the vehicle can be reduced; in addition, the communication real-time performance between the vehicle and the cloud platform is guaranteed based on the low-time-delay high-transmission-rate characteristic of 5G communication.
The database with large-scale samples does not need to be stored at the vehicle end, the personalized gesture database based on different vehicle types and different user groups is pre-placed at the cloud platform end, real acquisition samples of the personalized gesture database can be gradually accumulated along with the use of the vehicle, and the accuracy of gesture recognition on gestures provided by the vehicle based on the personalized gesture database is gradually improved along with the increase of the number of the real acquisition samples.
The cloud platform can perform vehicle gesture recognition self-learning based on big data, and when the user intention is inconsistent with the recognition result of gesture recognition, the cloud platform can analyze the user behavior according to the control result fed back by the vehicle to obtain a deviation point and complete self-learning, so that the personalized gesture database of the user to which the vehicle belongs is optimized, and finally a personalized gesture database unique to the user to which each vehicle belongs is formed. After each optimization, the accuracy of gesture recognition by using the optimized database is improved on the basis of the previous step; and by utilizing the self-learning optimization function, the mapping relation between the new gesture and the user intention of the user in a new scene during vehicle control execution by utilizing the gesture recognition can be self-learned, and the scene coverage of the function is increased.
Referring to fig. 6, the present invention also provides a vehicle control apparatus based on gesture recognition, including:
the first receiving module 301 is used for receiving a gesture to be recognized uploaded by a vehicle;
a comparison module 302, configured to compare the feature similarity between the gesture to be recognized and a pre-stored personalized gesture database associated with the user to which the vehicle belongs;
the output module 303 is configured to output a comparison result including the gesture type of the gesture to be recognized and the user intention represented by the gesture when the feature similarity comparison is successful;
the issuing module 304 is configured to issue the comparison result to a vehicle for the vehicle to control according to the user intention in the comparison result;
the pre-stored personalized gesture database at least comprises a plurality of groups of first mapping relations, wherein one group of first mapping relations comprise one gesture type representing user intention and a plurality of different-form gestures belonging to the same gesture type, and the plurality of different-form gestures are provided by a plurality of test objects with different heights, weights, ages and sexes.
Preferably, the pre-stored personalized gesture database further comprises at least one set of second mapping relations, wherein the set of second mapping relations comprises a gesture type representing the intention of a user and at least one form gesture provided by the user to which the vehicle belongs.
Preferably, the apparatus further comprises:
a second receiving module 305, configured to receive a manipulation result fed back by the vehicle based on the identification result;
a marking module 306, configured to determine that a comparison result is incorrect and mark a selected target form gesture once when the feature similarity is compared if the control result indicates that the vehicle performs a reverse operation according to an intention opposite to the user intention within a predetermined time after being controlled according to the user intention;
a deleting module 307, configured to delete the target form gesture from the pre-stored personalized gesture database when the marking frequency of the target form gesture reaches a predetermined frequency;
the target morphological gesture refers to a morphological gesture with the feature similarity exceeding a preset similarity value.
Preferably, the apparatus further comprises:
the feedback module 308 is configured to feed back feedback information of the failed comparison to the vehicle when the feature similarity comparison fails;
and the adding module 309 is configured to, within a predetermined time after the feedback information is sent, if information that the vehicle executes the operation based on the user trigger is received, classify the operation executed by the vehicle as a new user intention, classify the gesture to be recognized as a new gesture type associated with the new user intention, and record the new gesture type and the gesture to be recognized as a group of second mapping relationships in a pre-stored personalized gesture database.
Preferably, the user intent is at least: and characterizing the operation intention of a user on a front hatch cover, a skylight, a back door, a vehicle door and a vehicle machine of the vehicle.
Preferably, the apparatus further comprises:
the comparison module is used for comparing pre-stored personalized gesture databases related to users belonging to all vehicles aiming at the same vehicle type;
and the first updating module is used for deleting the gestures in the same form in the pre-stored personalized gesture database which exceeds the first preset proportion number, and deleting the gestures in the pre-stored personalized gesture database which is associated with all the users to which the vehicles belong.
Preferably, the apparatus further comprises:
and the second updating module is used for adding the group of mapping relations in the pre-stored personalized gesture database associated with the users to which all vehicles belong if the same group of mapping relations containing the new gesture types and the new user intentions are added in the pre-stored personalized gesture database with the number exceeding the second preset proportion number, and dividing the gestures of the users to which all vehicles belong into a plurality of morphological gestures in the group of mapping relations.
The device of the invention has the same technical effects as the method. That is, in the above-mentioned scheme of this embodiment, the original gesture recognition process applied to the vehicle end is implemented by placing the gesture recognition process on the cloud platform end, and the vehicle end does not need to be provided with a controller with high computing power and a camera with high sensing precision, so that the hardware cost of the vehicle can be reduced; in addition, the communication real-time performance between the vehicle and the cloud platform is guaranteed based on the low-time-delay high-transmission-rate characteristic of 5G communication.
The database with large-scale samples does not need to be stored at the vehicle end, the personalized gesture database based on different vehicle types and different user groups is pre-placed at the cloud platform end, real acquisition samples of the personalized gesture database can be gradually accumulated along with the use of the vehicle, and the accuracy of gesture recognition of gestures provided by the vehicle based on the personalized gesture database is gradually improved along with the increase of the number of the real acquisition samples.
The cloud platform can perform vehicle gesture recognition self-learning based on big data, and when the user intention is inconsistent with the recognition result of the gesture recognition, the cloud platform can analyze the user behavior according to the control result fed back by the vehicle to obtain a deviation point and complete the self-learning, so that the personalized gesture database of the user to which the vehicle belongs is optimized, and finally a unique personalized gesture database for the user to which each vehicle belongs is formed. After each optimization, the accuracy of gesture recognition by using the optimized database is improved on the basis of the previous step; and by utilizing the self-learning optimization function, the mapping relation between the new gesture and the user intention of the user in a new scene during vehicle control execution by utilizing the gesture recognition can be self-learned, and the scene coverage of the function is increased.
The invention also provides a control device comprising a processor, a memory and a program or instructions stored on the memory and executable on the processor, the program or instructions, when executed by the processor, implementing the steps of the gesture recognition based vehicle control method as described above.
The invention also provides a readable storage medium on which a program or instructions are stored, which program or instructions, when executed by a processor, carry out the steps of the gesture recognition based vehicle control method as described above.
Claims (10)
1. A vehicle control method based on gesture recognition is applied to a cloud platform and is characterized by comprising the following steps:
receiving a gesture to be recognized uploaded by a vehicle;
comparing the characteristic similarity of the gesture to be recognized with a pre-stored personalized gesture database associated with the user to which the vehicle belongs;
when the feature similarity comparison is successful, outputting a comparison result containing the gesture type of the gesture to be recognized and the user intention represented by the gesture;
issuing the comparison result to a vehicle for the vehicle to control according to the user intention in the comparison result;
the pre-stored personalized gesture database at least comprises a plurality of groups of first mapping relations, wherein one group of first mapping relations comprise one gesture type representing user intention and a plurality of different-form gestures belonging to the same gesture type, and the plurality of different-form gestures are provided by a plurality of test objects with different heights, weights, ages and sexes.
2. The vehicle control method based on gesture recognition according to claim 1, wherein the pre-stored personalized gesture database further comprises at least one set of second mapping relationships, and the set of second mapping relationships comprises a gesture type representing a user intention and at least one form gesture provided by a user to which the vehicle belongs.
3. The gesture recognition based vehicle control method according to claim 1 or 2, characterized in that the method further comprises:
receiving a control result fed back by the vehicle based on the identification result;
if the control result shows that the vehicle performs reverse operation according to the intention opposite to the user intention within the preset time after being controlled according to the user intention, the fact that the comparison result is wrong is confirmed, and the selected target form gesture is marked once when the feature similarity is compared;
deleting the target form gesture from a pre-stored personalized gesture database when the marking times of the target form gesture reach preset times;
the target morphological gesture refers to a morphological gesture with the feature similarity exceeding a preset similarity value.
4. The gesture recognition based vehicle control method according to claim 1 or 2, characterized in that the method further comprises:
when the feature similarity comparison fails, feeding back feedback information of the failure comparison to the vehicle;
and in the preset time after the feedback information is sent, if information that the vehicle executes the operation based on the user trigger is received, the operation executed by the vehicle is divided into a new user intention, the gesture to be recognized is divided into a new gesture type associated with the new user intention, and then the new gesture type and the gesture to be recognized are recorded into a pre-stored personalized gesture database as a group of second mapping relation.
5. The gesture recognition based vehicle control method according to claim 1, wherein the user intention is at least: the method is characterized in that the user's operation intention on a front hatch cover, a skylight, a back door, a vehicle door and a vehicle machine of the vehicle is represented.
6. The gesture recognition based vehicle control method according to claim 1, further comprising:
aiming at the same vehicle type, comparing pre-stored personalized gesture databases associated with users belonging to all vehicles;
if the same form of gesture is deleted in the pre-stored personalized gesture databases exceeding the first preset proportion number, the same form of gesture is deleted in the pre-stored personalized gesture databases associated with the users belonging to all vehicles.
7. The method of gesture recognition based vehicle control of claim 6, further comprising:
if the same group of mapping relations containing the new gesture types and the new user intentions are added in the pre-stored personalized gesture databases with the number exceeding the second preset proportion, the group of mapping relations are added in the pre-stored personalized gesture databases associated with the users to which all the vehicles belong, and the gestures of the users to which all the vehicles belong are divided into a plurality of morphological gestures in the group of mapping relations.
8. A vehicle control apparatus based on gesture recognition, comprising:
the first receiving module is used for receiving the gestures to be recognized uploaded by the vehicle;
the comparison module is used for comparing the characteristic similarity of the gesture to be recognized with a pre-stored personalized gesture database associated with the user to which the vehicle belongs;
the output module is used for outputting a comparison result containing the gesture type of the gesture to be recognized and the user intention represented by the gesture when the feature similarity comparison is successful;
the issuing module is used for issuing the comparison result to a vehicle for the vehicle to control according to the user intention in the comparison result;
the pre-stored personalized gesture database at least comprises a plurality of groups of first mapping relations, wherein one group of first mapping relations comprise a gesture type representing the intention of a user and a plurality of gestures with different forms belonging to the same gesture type, and the gestures with different forms are provided by a plurality of test subjects with different heights, weights, ages and sexes.
9. A control device comprising a processor, a memory and a program or instructions stored on the memory and executable on the processor, the program or instructions when executed by the processor implementing the steps of the gesture recognition based vehicle control method according to any one of claims 1 to 7.
10. A readable storage medium, characterized in that a program or instructions are stored thereon, which program or instructions, when executed by a processor, carry out the steps of the gesture recognition based vehicle control method according to any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210588028.6A CN114924647A (en) | 2022-05-27 | 2022-05-27 | Vehicle control method, device, control equipment and medium based on gesture recognition |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210588028.6A CN114924647A (en) | 2022-05-27 | 2022-05-27 | Vehicle control method, device, control equipment and medium based on gesture recognition |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN114924647A true CN114924647A (en) | 2022-08-19 |
Family
ID=82811241
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210588028.6A Pending CN114924647A (en) | 2022-05-27 | 2022-05-27 | Vehicle control method, device, control equipment and medium based on gesture recognition |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114924647A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116279237A (en) * | 2023-02-21 | 2023-06-23 | 惠州市科宇汽车精密配件有限公司 | Vehicle-mounted non-contact switch control system and control method thereof |
-
2022
- 2022-05-27 CN CN202210588028.6A patent/CN114924647A/en active Pending
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116279237A (en) * | 2023-02-21 | 2023-06-23 | 惠州市科宇汽车精密配件有限公司 | Vehicle-mounted non-contact switch control system and control method thereof |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11605226B2 (en) | Video data processing method and apparatus, and readable storage medium | |
| US20160101783A1 (en) | Adaptive driver identification fusion | |
| CN108983979B (en) | Gesture tracking recognition method and device and intelligent equipment | |
| CN106601257B (en) | Voice recognition method and device and first electronic device | |
| WO2020253039A1 (en) | Road section characteristic model training method and apparatus, and computer device and storage medium | |
| CN114924647A (en) | Vehicle control method, device, control equipment and medium based on gesture recognition | |
| CN113990299B (en) | Voice interaction method and device, server and readable storage medium thereof | |
| CN110335495A (en) | Vehicle position information processing method, device, computer equipment and storage medium | |
| CN109613930A (en) | Control method, device, unmanned vehicle and the storage medium of unmanned vehicle | |
| CN110781381A (en) | Data verification method, device and equipment based on neural network and storage medium | |
| CN113990298B (en) | Voice interaction method and device, server and readable storage medium | |
| CN113696904A (en) | Processing method, device, equipment and medium for controlling vehicle based on gestures | |
| CN104598473B (en) | A kind of information processing method and electronic equipment | |
| CN107894882B (en) | Voice input method of mobile terminal | |
| WO2023082571A1 (en) | Vehicle remote control method, system and apparatus, and computer readable storage medium | |
| CN113111091A (en) | Maintenance information acquisition method and device, computer equipment and storage medium | |
| CN108009411A (en) | Method, apparatus and computing device based on recognition of face control automobile | |
| CN111368721A (en) | Identity recognition processing method and device, electronic equipment and storage medium | |
| CN113696849B (en) | Gesture-based vehicle control method, device and storage medium | |
| CN110925945A (en) | Air conditioner robot control method and device based on gesture recognition | |
| CN116340332A (en) | Method and device for updating scene library of vehicle-mounted intelligent system and vehicle | |
| CN112732379B (en) | Method for running application program on intelligent terminal, terminal and storage medium | |
| CN117590978B (en) | Vehicle-mounted multimedia integrated management system and method based on artificial intelligence | |
| CN110736223A (en) | Air conditioner control method and device | |
| CN116469279B (en) | Automobile click-to-read teaching method and system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |