CN118131893A - Data processing method, device, electronic equipment and computer readable storage medium - Google Patents
Data processing method, device, electronic equipment and computer readable storage medium Download PDFInfo
- Publication number
- CN118131893A CN118131893A CN202211543650.1A CN202211543650A CN118131893A CN 118131893 A CN118131893 A CN 118131893A CN 202211543650 A CN202211543650 A CN 202211543650A CN 118131893 A CN118131893 A CN 118131893A
- Authority
- CN
- China
- Prior art keywords
- gesture
- target
- image
- information
- target object
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000003672 processing method Methods 0.000 title abstract description 18
- 238000000034 method Methods 0.000 claims abstract description 32
- 238000012545 processing Methods 0.000 claims abstract description 24
- 230000006870 function Effects 0.000 claims description 43
- 230000001815 facial effect Effects 0.000 claims description 23
- 238000004590 computer program Methods 0.000 claims description 17
- 230000003068 static effect Effects 0.000 claims description 9
- 238000012795 verification Methods 0.000 claims description 7
- 238000012216 screening Methods 0.000 claims description 6
- 239000010410 layer Substances 0.000 description 10
- 238000013527 convolutional neural network Methods 0.000 description 9
- 230000004044 response Effects 0.000 description 7
- 230000002452 interceptive effect Effects 0.000 description 6
- 238000013528 artificial neural network Methods 0.000 description 3
- 238000010586 diagram Methods 0.000 description 3
- 230000003287 optical effect Effects 0.000 description 3
- 230000008859 change Effects 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 230000003993 interaction Effects 0.000 description 2
- 230000002093 peripheral effect Effects 0.000 description 2
- 238000011176 pooling Methods 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 238000013135 deep learning Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 230000004438 eyesight Effects 0.000 description 1
- 238000001914 filtration Methods 0.000 description 1
- 239000011229 interlayer Substances 0.000 description 1
- 230000033001 locomotion Effects 0.000 description 1
- 230000007246 mechanism Effects 0.000 description 1
- 238000003058 natural language processing Methods 0.000 description 1
- 230000006855 networking Effects 0.000 description 1
- 238000010606 normalization Methods 0.000 description 1
- 238000012549 training Methods 0.000 description 1
- 238000012546 transfer Methods 0.000 description 1
- 230000016776 visual perception Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/017—Gesture based interaction, e.g. based on a set of recognized hand gestures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/764—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using classification, e.g. of video objects
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/20—Movements or behaviour, e.g. gesture recognition
- G06V40/28—Recognition of hand or arm movements, e.g. recognition of deaf sign language
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Physics & Mathematics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Health & Medical Sciences (AREA)
- Multimedia (AREA)
- General Health & Medical Sciences (AREA)
- Evolutionary Computation (AREA)
- Medical Informatics (AREA)
- Artificial Intelligence (AREA)
- Software Systems (AREA)
- Computing Systems (AREA)
- Databases & Information Systems (AREA)
- Human Computer Interaction (AREA)
- General Engineering & Computer Science (AREA)
- Psychiatry (AREA)
- Social Psychology (AREA)
- User Interface Of Digital Computer (AREA)
Abstract
The embodiment of the application provides a data processing method, a data processing device, electronic equipment and a computer readable storage medium, and relates to the technical field of software. The method comprises the steps of determining an identity of a target object and gesture information associated with the identity; the gesture information comprises at least one preset gesture and a task corresponding to the gesture; receiving at least one gesture image of a target object, and identifying a target gesture corresponding to the at least one gesture image; and determining a target task corresponding to the target gesture based on the gesture information, and executing the target task. According to the embodiment of the application, different gesture information can be set for different target objects, so that personalized gesture control is realized.
Description
Technical Field
The present application relates to the field of software technologies, and in particular, to a data processing method, apparatus, electronic device, and computer readable storage medium.
Background
With the development of network communication technology, the era of intelligent networking has come, and the application of intelligent vehicle-mounted systems is also becoming wider and wider; the intelligent vehicle-mounted system can realize huge operation amount only by high-configuration software and hardware, key parts can be reduced based on the micro-electromechanical system, the operation efficiency is improved in a straight line, and the intelligent vehicle-mounted system can integrate an original independent video-audio system, a navigation system, a driving record, an active/passive driving safety system and the like, so that more and more diversified application functions are developed.
Currently, in a smart vehicle-mounted system, a user interaction function can be realized through voice or gesture recognition, wherein the gesture recognition can enable a user to control the smart vehicle-mounted system through simple face or hand motions; in the prior art, the interactive application of the intelligent vehicle-mounted system is generally performed based on the fixed matching of the preset gesture library and the control task, but along with the continuous upgrading of the vehicle-mounted system software version and the continuous change of gesture types and recognition modes, the situation that the corresponding relation between each gesture and each system task is forgotten when the gesture function is used by a user can occur, so that the flexibility and the accuracy of the gesture control function are insufficient.
Disclosure of Invention
The embodiment of the application provides a data processing method, a device, electronic equipment and a computer readable storage medium, which can solve the problem of low flexibility and accuracy of gesture control functions in the prior art. The technical scheme is as follows:
According to an aspect of an embodiment of the present application, there is provided a data processing method, including:
determining the identity of the target object and gesture information associated with the identity; the gesture information comprises at least one preset gesture and a task corresponding to the gesture;
Receiving at least one gesture image of a target object, and identifying a target gesture corresponding to the at least one gesture image;
and determining a target task corresponding to the target gesture based on the gesture information, and executing the target task.
In one possible implementation manner, the determining the identity of the target object includes:
Acquiring facial feature information of a target object based on a login request of the target object;
and when the facial feature information passes the verification, determining the identity of the target object.
In one possible implementation manner, the method further includes:
Receiving a gesture setting instruction of a target object;
determining gesture information according to the gesture setting instruction;
and establishing an association relation between the gesture information and the identity.
In one possible implementation manner, after determining the gesture information according to the gesture setting instruction, the method includes:
And when the gesture corresponding to the gesture information is the first setting, playing the gesture teaching animation corresponding to the gesture.
In another possible implementation manner, the receiving the at least one gesture image of the target object and identifying the target gesture corresponding to the at least one gesture image include:
Acquiring an acquisition image aiming at a target object;
identifying the acquired image, and screening at least one gesture image from the acquired image;
identifying a target gesture corresponding to at least one gesture image; when the target gesture indicates a static gesture, the number of gesture images is one; when the target gesture indicates a dynamic gesture, the number of gesture images is at least two.
In another possible implementation manner, the identifying the target gesture corresponding to the at least one gesture image includes:
Inputting at least one gesture image into a pre-trained recognition model, and obtaining a target gesture identifier through the following operations:
extracting gesture characteristic information of at least one gesture image;
carrying out feature enhancement on the gesture feature information to obtain image enhancement features;
Classifying based on the image enhancement features to obtain a target gesture corresponding to at least one gesture image.
In another possible implementation manner, the classifying based on the image enhancement features to obtain the target gesture corresponding to the at least one gesture image includes:
calculating the similarity probability of at least one gesture image and each gesture based on the image enhancement features;
and taking the gesture corresponding to the maximum value of the similarity probability as the target gesture.
In another possible implementation, the recognition model is trained based on the following manner:
Acquiring a sample gesture image and a sample gesture corresponding to the sample gesture image;
Inputting the sample gesture image into an initial model to obtain a tag gesture output by the initial model in real time;
And determining a difference value between the sample gesture and the label gesture based on a preset loss function, and adjusting parameters of the initial model based on the difference value until the loss function meets a convergence condition to obtain the identification model.
According to another aspect of an embodiment of the present application, there is provided a data processing apparatus including:
The determining module is used for determining the identity mark of the target object and gesture information associated with the identity mark; the gesture information comprises at least one preset gesture and a task corresponding to the gesture;
The recognition module is used for receiving at least one gesture image of the target object and recognizing a target gesture corresponding to the at least one gesture image;
and the execution module is used for determining a target task corresponding to the target gesture based on the gesture information and executing the target task.
In one possible implementation manner, the determining module is configured to, when determining the identity of the target object:
Acquiring facial feature information of a target object based on a login request of the target object;
and when the facial feature information passes the verification, determining the identity of the target object.
In one possible implementation manner, the apparatus further includes an association module, configured to:
Receiving a gesture setting instruction of a target object;
determining gesture information according to the gesture setting instruction;
and establishing an association relation between the gesture information and the identity.
In one possible implementation manner, the association module is configured to, after determining the gesture information according to the gesture setting instruction:
And when the gesture corresponding to the gesture information is the first setting, playing the gesture teaching animation corresponding to the gesture.
In another possible implementation manner, the identifying module is configured to, when receiving at least one gesture image of the target object and identifying a target gesture corresponding to the at least one gesture image:
Acquiring an acquisition image aiming at a target object;
identifying the acquired image, and screening at least one gesture image from the acquired image;
identifying a target gesture corresponding to at least one gesture image; when the target gesture indicates a static gesture, the number of gesture images is one; when the target gesture indicates a dynamic gesture, the number of gesture images is at least two.
In another possible implementation manner, the identifying module is configured to, when identifying a target gesture corresponding to at least one gesture image:
Inputting at least one gesture image into a pre-trained recognition model, and obtaining a target gesture identifier through the following operations:
extracting gesture characteristic information of at least one gesture image;
carrying out feature enhancement on the gesture feature information to obtain image enhancement features;
Classifying based on the image enhancement features to obtain a target gesture corresponding to at least one gesture image.
In another possible implementation manner, the identification module is configured to, when classifying based on the image enhancement feature, obtain a target gesture corresponding to at least one gesture image:
calculating the similarity probability of at least one gesture image and each gesture based on the image enhancement features;
and taking the gesture corresponding to the maximum value of the similarity probability as the target gesture.
In another possible implementation, the recognition model is trained based on the following manner:
Acquiring a sample gesture image and a sample gesture corresponding to the sample gesture image;
Inputting the sample gesture image into an initial model to obtain a tag gesture output by the initial model in real time;
And determining a difference value between the sample gesture and the label gesture based on a preset loss function, and adjusting parameters of the initial model based on the difference value until the loss function meets a convergence condition to obtain the identification model.
According to another aspect of an embodiment of the present application, there is provided an electronic apparatus including: a memory, a processor and a computer program stored on the memory, the processor executing the computer program to perform the steps of the method according to the first aspect of the embodiment of the application.
According to a further aspect of embodiments of the present application there is provided a computer readable storage medium having stored thereon a computer program which when executed by a processor performs the steps of the method of the first aspect of embodiments of the present application.
According to an aspect of an embodiment of the present application, there is provided a computer program product comprising a computer program which, when executed by a processor, implements the steps of the method of the first aspect of the embodiment of the present application.
The technical scheme provided by the embodiment of the application has the beneficial effects that:
According to the embodiment of the application, the identity of the target object is acquired, the gesture information associated with the identity is determined, then at least one gesture image of the target object is received, the target gesture corresponding to the at least one gesture image is identified, and further the target task corresponding to the target gesture is determined based on the gesture information, so that the target task is executed. According to the embodiment of the application, the special gesture control for the target object is realized through the association binding of the identity mark of the target object and the gesture information; different from the interactive function of realizing gesture control based on fixed matching of a preset gesture library and a control task in the prior art, the method and the device can set different gesture information aiming at different target objects, realize personalized gesture control, enhance the flexibility and the accuracy of the gesture control function and effectively improve the user experience.
Drawings
In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings that are required to be used in the description of the embodiments of the present application will be briefly described below.
Fig. 1 is a schematic diagram of an application scenario of a data processing method according to an embodiment of the present application;
FIG. 2 is a schematic flow chart of a data processing method according to an embodiment of the present application;
FIG. 3 is a schematic flow chart of determining an identity in a data processing method according to an embodiment of the present application;
FIG. 4 is a schematic flow chart of training an identification model in a data processing method according to an embodiment of the present application;
FIG. 5 is a flow chart of an exemplary data processing method according to an embodiment of the present application;
FIG. 6 is a schematic diagram of a data processing apparatus according to an embodiment of the present application;
Fig. 7 is a schematic structural diagram of a data processing electronic device according to an embodiment of the present application.
Detailed Description
Embodiments of the present application are described below with reference to the drawings in the present application. It should be understood that the embodiments described below with reference to the drawings are exemplary descriptions for explaining the technical solutions of the embodiments of the present application, and the technical solutions of the embodiments of the present application are not limited.
As used herein, the singular forms "a", "an", "the" and "the" are intended to include the plural forms as well, unless expressly stated otherwise, as understood by those skilled in the art. It will be further understood that the terms "comprises" and "comprising," when used in this specification, specify the presence of stated features, information, data, steps, operations, elements, and/or components, but do not preclude the presence or addition of other features, information, data, steps, operations, elements, components, and/or groups thereof, all of which may be included in the present specification. It will be understood that when an element is referred to as being "connected" or "coupled" to another element, it can be directly connected or coupled to the other element or intervening elements may be present. Further, "connected" or "coupled" as used herein may include wirelessly connected or wirelessly coupled. The term "and/or" as used herein indicates that at least one of the items defined by the term, e.g., "a and/or B" may be implemented as "a", or as "B", or as "a and B".
For the purpose of making the objects, technical solutions and advantages of the present application more apparent, the embodiments of the present application will be described in further detail with reference to the accompanying drawings.
The gesture control end in the intelligent vehicle-mounted system can emit light through the 3D camera above the center console, and the change of gestures can be analyzed according to the time difference between light emission and receiving equipment. The obtained data are transmitted to a control unit of the vehicle-mounted system, the control unit calls out the function corresponding to the recognized gesture, then the function is matched with the gesture preset in the system, and a corresponding operation instruction is issued.
In the prior art, different control functions are matched with each gesture in the gesture library, and along with the function upgrading of the intelligent vehicle-mounted system and the updating of the corresponding gesture library, a user possibly forgets the corresponding relation between the gesture and the control function when performing gesture control, so that the user experience is poor, and the accuracy of the gesture control function is insufficient.
The application provides a data processing method, a data processing device, electronic equipment and a computer readable storage medium, and aims to solve the technical problems in the prior art.
The technical solutions of the embodiments of the present application and technical effects produced by the technical solutions of the present application are described below by describing several exemplary embodiments. It should be noted that the following embodiments may be referred to, or combined with each other, and the description will not be repeated for the same terms, similar features, similar implementation steps, and the like in different embodiments.
As shown in fig. 1, the data processing method of the present application may be applied to the scenario shown in fig. 1, specifically, the user sends a login instruction to the server 102 based on the touch screen 101, and the server 102 determines the identity of the user and gesture information associated with the identity; meanwhile, the server 102 receives at least one gesture image acquired by the camera for the user, and recognizes a target gesture corresponding to the at least one gesture image; and determining a target task corresponding to the target gesture based on the gesture information so as to execute the target task.
In the scenario shown in fig. 1, the data processing method may be performed in a server, or in other scenarios, may be performed in a terminal.
As will be appreciated by those skilled in the art, the "terminal" as used herein may be a cell phone, tablet computer, PDA (Personal DIGITAL ASSISTANT ), MID (Mobile INTERNET DEVICE, mobile internet device), etc.; the "server" may be implemented as a stand-alone server or as a server cluster composed of a plurality of servers.
The embodiment of the application provides a data processing method, as shown in fig. 2, which can be applied to a server or a terminal for data processing, and the method comprises the following steps:
S201, determining the identity of the target object and gesture information associated with the identity.
The gesture information comprises at least one preset gesture and a task corresponding to the gesture. The gesture comprises a static gesture and a dynamic gesture, the target object can be a driver using the intelligent vehicle-mounted system, and the corresponding identity mark can be a user number or a user name of the target object logging in the intelligent vehicle-mounted system.
Specifically, the server or the terminal for performing data processing may acquire the identity of the target object and gesture information associated with the identity after the target object logs in the intelligent vehicle-mounted system.
S202, at least one gesture image of the target object is received, and a target gesture corresponding to the at least one gesture image is identified.
The gesture image may be an image acquired for a hand gesture of the target object.
Specifically, the server or the terminal for performing data processing can collect an collected image of a target object through a camera, detect the collected image, and take the collected image as a gesture image when detecting that the collected image comprises a complete hand shape of the target object; and then, based on a preset recognition model, recognizing the at least one gesture image to obtain a corresponding target gesture.
In the embodiment of the present application, when the target gesture is a dynamic gesture, the number of corresponding gesture images is at least two; when the target gesture is a static gesture, the number of corresponding gesture images is one.
S203, determining a target task corresponding to the target gesture based on the gesture information, and executing the target task.
The gesture information comprises a plurality of groups of gesture information and task information.
Specifically, the server or the terminal for performing data processing may search for a data pair corresponding to the target gesture from the gesture information based on the target gesture, so as to determine a target task corresponding to the target gesture according to the data pair, and execute the target task.
In an embodiment of the present application, the gesture information may include: the V-shaped gesture corresponds to a photographing task, the OK gesture continues or determines an operation instruction, the left hand waving gesture corresponds to a previous or last page operation instruction and the like. When the server or the terminal is used for carrying out data processing, the V-shaped gesture is identified, a corresponding data pair can be obtained by searching based on the gesture, and a photographing task corresponding to the V-shaped gesture is determined according to the data pair so as to start the camera to photograph.
According to the embodiment of the application, the identity of the target object is acquired, the gesture information associated with the identity is determined, then at least one gesture image of the target object is received, the target gesture corresponding to the at least one gesture image is identified, and further the target task corresponding to the target gesture is determined based on the gesture information, so that the target task is executed. According to the embodiment of the application, the special gesture control for the target object is realized through the association binding of the identity mark of the target object and the gesture information; different from the interactive function of realizing gesture control based on fixed matching of a preset gesture library and a control task in the prior art, the method and the device can set different gesture information aiming at different target objects, realize personalized gesture control, enhance the flexibility and the accuracy of the gesture control function and effectively improve the user experience.
In one possible implementation manner provided in the embodiment of the present application, as shown in fig. 3, the determining the identity of the target object includes:
S301, acquiring facial feature information of a target object based on a login request of the target object.
The login request of the target object may be generated in response to a triggering operation of the touch screen, or may be generated in response to a vehicle start notification sent by the central control system.
In some embodiments, the server or the terminal for performing data processing may generate a login request based on a triggering operation of a user on the touch screen, and start a preset camera to acquire and acquire facial feature information of the target object in response to the login request.
In other embodiments, the server or the terminal for performing data processing may generate a login request in response to a vehicle start notification acquired from the vehicle central control system, and start a preset camera to acquire and acquire facial feature information of the target object in response to the login request.
S302, when the facial feature information verification passes, the identity of the target object is determined.
Specifically, comparing the facial feature information with each feature information in a preset user database, and when feature information consistent with the facial feature information exists in the user database, verifying the facial feature information; and acquiring an identity corresponding to the characteristic information in the user database as the identity of the target object.
In the embodiment of the application, the identity and the characteristic information of at least one common user corresponding to the vehicle are recorded in the intelligent vehicle-mounted system in advance, when the user starts the vehicle, the central control system of the vehicle sends a vehicle starting notice to the intelligent vehicle-mounted system, the intelligent vehicle-mounted system responds to the starting notice to generate a user login request, a camera is started to collect facial characteristic information of the user, and when the facial characteristic information is consistent with a certain characteristic information recorded in advance, the user login is successful, and the identity of the user is determined.
The embodiment of the application determines the identity of the user, namely the driver and the passenger of the vehicle by identifying the facial feature information of the user; the intelligent vehicle-mounted system is not required to be logged in by a user through complicated touch operation, and the use convenience of the intelligent vehicle-mounted system is effectively improved.
The embodiment of the application provides a possible implementation manner, and the method further comprises the following steps:
S401, receiving a gesture setting instruction of a target object.
The gesture setting instruction may be determined according to a touch operation of the touch screen.
Further, the touch operation includes at least one of the following:
dragging or moving the gesture identification component to a preset range of the current interface;
Click or touch operations for the gesture identification component;
And performing input operation aiming at gesture identification in a preset input control.
S402, determining gesture information according to the gesture setting instruction.
In the embodiment of the application, each gesture identifier can be displayed in the terminal user interface, a user can drag the gesture identifier to a setting area of a corresponding functional task, or select the corresponding functional task in a selection box of the gesture identifier, and the corresponding functional task can be input in an input box of the gesture identifier so as to determine the corresponding relation between each gesture identifier and the system functional task.
S403, establishing the association relation between the gesture information and the identity mark.
Specifically, the server or the terminal for performing data processing may store the gesture information and the associated identity in a preset database in the form of key value pairs, so as to establish an association relationship between the gesture information and the identity of the user.
The embodiment of the present application provides a possible implementation manner, which includes:
And when the gesture corresponding to the gesture information is the first setting, playing the gesture teaching animation corresponding to the gesture.
In the embodiment of the application, after the user logs in successfully, the user can check the gesture list supported by the current intelligent vehicle-mounted system, and when the user clicks or sets a certain gesture for the first time, the server or the terminal for processing data can play the gesture teaching animation corresponding to the gesture in the display screen.
The embodiment of the present application provides a possible implementation manner, where the receiving at least one gesture image of the target object and identifying a target gesture corresponding to the at least one gesture image include:
S501, acquiring an acquisition image aiming at a target object, identifying the acquisition image, and screening at least one gesture image from the acquisition image.
Specifically, the server or the terminal for performing data processing may collect an collected image of the target object through the camera, detect the collected image, and when detecting that the collected image includes a complete hand shape of the target object, use the collected image as the gesture image.
S502, identifying a target gesture corresponding to at least one gesture image.
When the target gesture indicates a static gesture, the number of gesture images is one; when the target gesture indicates a dynamic gesture, the number of gesture images is at least two.
Specifically, the server or the terminal for performing data processing may identify the at least one gesture image based on a preset identification model, so as to obtain a corresponding target gesture; the steps of gesture image recognition will be described in detail below.
The embodiment of the present application provides a possible implementation manner, where the identifying a target gesture corresponding to at least one gesture image includes:
Inputting at least one gesture image into a pre-trained recognition model, and obtaining a target gesture identifier through the following operations:
extracting gesture characteristic information of at least one gesture image;
carrying out feature enhancement on the gesture feature information to obtain image enhancement features;
Classifying based on the image enhancement features to obtain a target gesture corresponding to at least one gesture image.
The identification model may be a convolutional neural network model. Convolutional neural networks (Convolutional Neural Networks, CNN) are a class of feedforward neural networks that contain convolutional computations and have a deep structure, which are one of the representative algorithms for deep learning. Convolutional neural networks have a characteristic learning capability and can perform translation-invariant classification on input information according to a hierarchical structure of the convolutional neural networks, so the convolutional neural networks are also called as 'translation-invariant artificial neural networks'. The visual perception mechanism of the imitation living beings of the neural network is constructed, and the supervised learning and the unsupervised learning can be performed, so that the convolutional neural network can perform latticed features with smaller calculation amount due to the fact that the convolutional kernel parameters in the hidden layer are shared and the sparsity of interlayer connection. Convolutional neural networks are widely used in the fields of computer vision, natural language processing, and the like.
In an embodiment of the application, the recognition model may include an input layer, a convolution layer, a pooling layer, and an output layer. Specifically, the input layer is configured to receive pixel values of the gesture image, and perform normalization processing on each pixel value to obtain input data. Then, the convolution layer carries out convolution processing on the input data through a plurality of convolution cores, and gesture characteristic information of the gesture image is extracted; and the pooling layer performs feature selection and information filtering on the gesture feature information to obtain image enhancement information. Then, the output layer classifies the image enhancement information through a logic function or a normalized exponential function (softmax function) to obtain a corresponding target gesture. Specific classification steps will be described in detail below.
In an embodiment of the present application, as shown in fig. 4, a possible implementation manner is provided, where the above-mentioned recognition model is trained based on the following manner:
Acquiring a sample gesture image and a sample gesture corresponding to the sample gesture image; inputting the sample gesture image into an initial model to obtain a tag gesture output by the initial model in real time; and determining a difference value between the sample gesture and the label gesture based on a preset loss function, and adjusting parameters of the initial model based on the difference value until the loss function meets a convergence condition to obtain the identification model.
Wherein, the loss function can be used for representing the difference degree between the sample gesture and the label gesture. Further, the greater the loss value calculated from the loss function, the greater the degree of the difference. The convergence condition of the loss function may include that the difference (i.e., the loss value) is smaller than a preset threshold or that the difference is unchanged, which is not particularly limited in the embodiment of the present application.
The embodiment of the present application provides a possible implementation manner, where the classifying based on the image enhancement features to obtain a target gesture corresponding to at least one gesture image includes:
And calculating the similarity probability of at least one gesture image and each gesture based on the image enhancement features, wherein the gesture corresponding to the maximum value of the similarity probability is used as the target gesture.
Specifically, the server or the terminal for performing data processing may calculate the similarity probability between the gesture image and each preset gesture based on the classification function of the output layer in the recognition model; and taking the gesture corresponding to the maximum value of the pixel probability as the target gesture. Wherein the classification function may comprise a logic function or a normalized exponential function.
According to the embodiment of the application, the gesture image is obtained by detecting the acquired image, and the gesture corresponding to the gesture image is obtained by identifying according to the pre-trained identification model, so that the efficiency of gesture identification is improved, and the accuracy of a gesture control function is further enhanced.
In order to better understand the above data processing method, an example of the data processing method of the present application is described in detail below with reference to fig. 5, and the method may be applied to an intelligent vehicle terminal, and includes the following steps:
S601, the intelligent vehicle-mounted terminal acquires facial feature information of the target object based on a login request of the target object.
The login request of the target object can be generated in response to a triggering operation of the touch screen of the intelligent vehicle-mounted terminal, and can also be generated in response to a vehicle starting notification sent by the vehicle central control system.
S602, when the facial feature information verification passes, determining the identity of the target object.
Specifically, the intelligent vehicle-mounted terminal compares the facial feature information with each feature information in a preset user database, and when feature information consistent with the facial feature information exists in the user database, the facial feature information passes verification; and acquiring an identity corresponding to the characteristic information in the user database as the identity of the target object.
S603, receiving a gesture setting instruction of the target object, determining gesture information according to the gesture setting instruction, and establishing an association relationship between the gesture information and the identity.
In the embodiment of the application, each gesture identifier can be displayed in the terminal user interface, a user can drag the gesture identifier to a setting area of a corresponding functional task, or select the corresponding functional task in a selection box of the gesture identifier, and the corresponding functional task can be input in an input box of the gesture identifier so as to determine the corresponding relation between each gesture identifier and the system functional task. Then, the intelligent vehicle-mounted terminal can store the gesture information and the associated identity mark into a preset database in the form of key value pairs so as to establish the association relation between the gesture information of the user and the identity mark.
S604, when the gesture corresponding to the gesture information is the first setting, playing the gesture teaching animation corresponding to the gesture.
In the embodiment of the application, after the user logs in successfully, the user can check the gesture list supported by the current intelligent vehicle-mounted system, and when the user clicks or sets a gesture for the first time, the intelligent vehicle-mounted terminal can play the gesture teaching animation corresponding to the gesture in the display screen.
S605, acquiring an acquisition image aiming at a target object, identifying the acquisition image, and screening at least one gesture image from the acquisition image.
Specifically, the intelligent vehicle-mounted terminal can collect the collected image of the target object through the camera, detect the collected image, and take the collected image as the gesture image when detecting that the collected image comprises the complete hand shape of the target object.
S606, inputting at least one gesture image into a pre-trained recognition model, and obtaining a target gesture identifier through the following operations:
extracting gesture characteristic information of at least one gesture image;
carrying out feature enhancement on the gesture feature information to obtain image enhancement features;
Classifying based on the image enhancement features to obtain a target gesture corresponding to at least one gesture image.
S607, determining a target task corresponding to the target gesture based on the gesture information, and executing the target task.
The gesture information comprises a plurality of groups of gesture information and task information.
Specifically, the intelligent vehicle-mounted terminal can search for a data pair corresponding to the target gesture from the gesture information based on the target gesture, so as to determine a target task corresponding to the target gesture according to the data pair, and execute the target task.
In an embodiment of the present application, the gesture information may include: the V-shaped gesture corresponds to a photographing task, the OK gesture continues or determines an operation instruction, the left hand waving gesture corresponds to a previous or last page operation instruction and the like. When the intelligent vehicle-mounted terminal recognizes and obtains the V-shaped gesture, a corresponding data pair can be obtained based on the gesture search, and a photographing task corresponding to the V-shaped gesture is determined according to the data pair so as to start the camera to photograph.
According to the embodiment of the application, the identity of the target object is acquired, the gesture information associated with the identity is determined, then at least one gesture image of the target object is received, the target gesture corresponding to the at least one gesture image is identified, and further the target task corresponding to the target gesture is determined based on the gesture information, so that the target task is executed. According to the embodiment of the application, the special gesture control for the target object is realized through the association binding of the identity mark of the target object and the gesture information; different from the interactive function of realizing gesture control based on fixed matching of a preset gesture library and a control task in the prior art, the method and the device can set different gesture information aiming at different target objects, realize personalized gesture control, enhance the flexibility and the accuracy of the gesture control function and effectively improve the user experience.
An embodiment of the present application provides a data processing apparatus, as shown in fig. 6, the data processing apparatus 600 may include: a determining module 601, an identifying module 602 and an executing module 603;
the determining module 601 is configured to determine an identity of a target object and gesture information associated with the identity; the gesture information comprises at least one preset gesture and a task corresponding to the gesture;
The recognition module 602 is configured to receive at least one gesture image of a target object, and recognize a target gesture corresponding to the at least one gesture image;
the execution module 603 is configured to determine a target task corresponding to the target gesture based on the gesture information, and execute the target task.
In one possible implementation manner provided in the embodiment of the present application, the determining module 601 is configured to, when determining the identity of the target object:
Acquiring facial feature information of a target object based on a login request of the target object;
and when the facial feature information passes the verification, determining the identity of the target object.
The embodiment of the application provides a possible implementation manner, and the device further comprises a correlation module for:
Receiving a gesture setting instruction of a target object;
determining gesture information according to the gesture setting instruction;
and establishing an association relation between the gesture information and the identity.
The embodiment of the application provides a possible implementation manner, and the association module is used for after determining gesture information according to the gesture setting instruction:
And when the gesture corresponding to the gesture information is the first setting, playing the gesture teaching animation corresponding to the gesture.
The embodiment of the application provides a possible implementation manner, and the identification module is used for, when receiving at least one gesture image of a target object and identifying a target gesture corresponding to the at least one gesture image:
Acquiring an acquisition image aiming at a target object;
identifying the acquired image, and screening at least one gesture image from the acquired image;
identifying a target gesture corresponding to at least one gesture image; when the target gesture indicates a static gesture, the number of gesture images is one; when the target gesture indicates a dynamic gesture, the number of gesture images is at least two.
In one possible implementation manner provided in the embodiment of the present application, the identifying module 602 is configured to, when identifying a target gesture corresponding to at least one gesture image:
Inputting at least one gesture image into a pre-trained recognition model, and obtaining a target gesture identifier through the following operations:
extracting gesture characteristic information of at least one gesture image;
carrying out feature enhancement on the gesture feature information to obtain image enhancement features;
Classifying based on the image enhancement features to obtain a target gesture corresponding to at least one gesture image.
In one possible implementation manner provided in the embodiment of the present application, the identification module 602 is configured to, when classifying based on the image enhancement feature to obtain a target gesture corresponding to at least one gesture image:
calculating the similarity probability of at least one gesture image and each gesture based on the image enhancement features;
and taking the gesture corresponding to the maximum value of the similarity probability as the target gesture.
The embodiment of the application provides a possible implementation mode, and the identification model is trained based on the following modes:
Acquiring a sample gesture image and a sample gesture corresponding to the sample gesture image;
Inputting the sample gesture image into an initial model to obtain a tag gesture output by the initial model in real time;
And determining a difference value between the sample gesture and the label gesture based on a preset loss function, and adjusting parameters of the initial model based on the difference value until the loss function meets a convergence condition to obtain the identification model.
The device of the embodiment of the present application may perform the method provided by the embodiment of the present application, and its implementation principle is similar, and actions performed by each module in the device of the embodiment of the present application correspond to steps in the method of the embodiment of the present application, and detailed functional descriptions of each module of the device may be referred to the descriptions in the corresponding methods shown in the foregoing, which are not repeated herein.
According to the embodiment of the application, the identity of the target object is acquired, the gesture information associated with the identity is determined, then at least one gesture image of the target object is received, the target gesture corresponding to the at least one gesture image is identified, and further the target task corresponding to the target gesture is determined based on the gesture information, so that the target task is executed. According to the embodiment of the application, the special gesture control for the target object is realized through the association binding of the identity mark of the target object and the gesture information; different from the interactive function of realizing gesture control based on fixed matching of a preset gesture library and a control task in the prior art, the method and the device can set different gesture information aiming at different target objects, realize personalized gesture control, enhance the flexibility and the accuracy of the gesture control function and effectively improve the user experience.
The embodiment of the application provides an electronic device, which comprises a memory, a processor and a computer program stored on the memory, wherein the processor executes the computer program to realize the steps of a data processing method, and compared with the related technology, the method can realize the following steps: according to the embodiment of the application, the identity of the target object is acquired, the gesture information associated with the identity is determined, then at least one gesture image of the target object is received, the target gesture corresponding to the at least one gesture image is identified, and further the target task corresponding to the target gesture is determined based on the gesture information, so that the target task is executed. According to the embodiment of the application, the special gesture control for the target object is realized through the association binding of the identity mark of the target object and the gesture information; different from the interactive function of realizing gesture control based on fixed matching of a preset gesture library and a control task in the prior art, the method and the device can set different gesture information aiming at different target objects, realize personalized gesture control, enhance the flexibility and the accuracy of the gesture control function and effectively improve the user experience.
In an alternative embodiment, an electronic device is provided, as shown in fig. 7, the electronic device 700 shown in fig. 7 includes: a processor 701 and a memory 703. The processor 701 is coupled to a memory 703, such as via a bus 702. Optionally, the electronic device 700 may further comprise a transceiver 704, the transceiver 704 may be used for data interaction between the electronic device and other electronic devices, such as transmission of data and/or reception of data, etc. It should be noted that, in practical applications, the transceiver 704 is not limited to one, and the structure of the electronic device 700 is not limited to the embodiment of the present application.
The Processor 701 may be a CPU (Central Processing Unit ), general purpose Processor, DSP (DIGITAL SIGNAL Processor, data signal Processor), ASIC (Application SPECIFIC INTEGRATED Circuit), FPGA (Field Programmable GATE ARRAY ) or other programmable logic device, transistor logic device, hardware component, or any combination thereof. Which may implement or perform the various exemplary logic blocks, modules and circuits described in connection with this disclosure. The processor 701 may also be a combination that performs computing functions, such as including one or more microprocessors, a combination of a DSP and a microprocessor, or the like.
Bus 702 may include a path to transfer information between the components. Bus 702 may be a PCI (PERIPHERAL COMPONENT INTERCONNECT, peripheral component interconnect standard) bus, or an EISA (Extended Industry Standard Architecture ) bus, or the like. Bus 702 may be divided into an address bus, a data bus, a control bus, and the like. For ease of illustration, only one thick line is shown in fig. 7, but not only one bus or one type of bus.
The Memory 703 may be, but is not limited to, ROM (Read Only Memory) or other type of static storage device that can store static information and instructions, RAM (Random Access Memory ) or other type of dynamic storage device that can store information and instructions, EEPROM (ELECTRICALLY ERASABLE PROGRAMMABLE READ ONLY MEMORY ), CD-ROM (Compact Disc Read Only Memory, compact disc Read Only Memory) or other optical disk storage, optical disk storage (including compact discs, laser discs, optical discs, digital versatile discs, blu-ray discs, etc.), magnetic disk storage media, other magnetic storage devices, or any other medium that can be used to carry or store a computer program and that can be Read by a computer.
The memory 703 is used for storing a computer program for executing an embodiment of the present application and is controlled to be executed by the processor 701. The processor 701 is arranged to execute a computer program stored in the memory 703 for carrying out the steps shown in the foregoing method embodiments.
Among them, electronic devices include, but are not limited to: mobile terminals such as mobile phones, notebook computers, PADs, etc., and stationary terminals such as digital TVs, desktop computers, etc.
Embodiments of the present application provide a computer readable storage medium having a computer program stored thereon, which when executed by a processor, implements the steps of the foregoing method embodiments and corresponding content.
Embodiments of the present application provide a computer program product or computer program comprising computer instructions stored in a computer readable storage medium. The processor of the computer device reads the computer instructions from the computer readable storage medium, and the processor executes the computer instructions such that the computer device performs:
determining the identity of the target object and gesture information associated with the identity; the gesture information comprises at least one preset gesture and a task corresponding to the gesture;
Receiving at least one gesture image of a target object, and identifying a target gesture corresponding to the at least one gesture image;
and determining a target task corresponding to the target gesture based on the gesture information, and executing the target task.
The terms "first," "second," "third," "fourth," "1," "2," and the like in the description and in the claims and in the above figures, if any, are used for distinguishing between similar objects and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used may be interchanged where appropriate, such that the embodiments of the application described herein may be implemented in other sequences than those illustrated or otherwise described.
It should be understood that, although various operation steps are indicated by arrows in the flowcharts of the embodiments of the present application, the order in which these steps are implemented is not limited to the order indicated by the arrows. In some implementations of embodiments of the application, the implementation steps in the flowcharts may be performed in other orders as desired, unless explicitly stated herein. Furthermore, some or all of the steps in the flowcharts may include multiple sub-steps or multiple stages based on the actual implementation scenario. Some or all of these sub-steps or phases may be performed at the same time, or each of these sub-steps or phases may be performed at different times, respectively. In the case of different execution time, the execution sequence of the sub-steps or stages can be flexibly configured according to the requirement, which is not limited by the embodiment of the present application.
The foregoing is merely an optional implementation manner of some of the implementation scenarios of the present application, and it should be noted that, for those skilled in the art, other similar implementation manners based on the technical ideas of the present application are adopted without departing from the technical ideas of the scheme of the present application, and the implementation manner is also within the protection scope of the embodiments of the present application.
Claims (11)
1. A method of data processing, the method comprising:
determining the identity of a target object and gesture information associated with the identity; the gesture information comprises at least one preset gesture and a task corresponding to the gesture;
Receiving at least one gesture image of a target object, and identifying a target gesture corresponding to the at least one gesture image;
And determining a target task corresponding to the target gesture based on the gesture information, and executing the target task.
2. The method of claim 1, wherein determining the identity of the target object comprises:
Acquiring facial feature information of the target object based on a login request of the target object;
and when the facial feature information passes the verification, determining the identity of the target object.
3. The method according to claim 1, characterized in that the method further comprises:
receiving a gesture setting instruction of the target object;
determining the gesture information according to the gesture setting instruction;
and establishing an association relation between the gesture information and the identity.
4. A method according to claim 3, wherein after said determining said gesture information according to said gesture setting instruction, comprising:
And when the gesture corresponding to the gesture information is set for the first time, playing the gesture teaching animation corresponding to the gesture.
5. The method of claim 1, wherein receiving at least one gesture image of the target object and identifying a target gesture corresponding to the at least one gesture image comprises:
Acquiring an acquisition image aiming at the target object;
identifying the acquired images, and screening at least one gesture image from the acquired images;
Identifying a target gesture corresponding to the at least one gesture image; when the target gesture indicates a static gesture, the number of gesture images is one; when the target gesture indicates a dynamic gesture, the number of gesture images is at least two.
6. The method of claim 1, wherein the identifying the target gesture corresponding to the at least one gesture image comprises:
inputting the at least one gesture image into a pre-trained recognition model, and obtaining the target gesture identification through the following operations:
extracting gesture characteristic information of the at least one gesture image;
Performing feature enhancement on the gesture feature information to obtain image enhancement features;
and classifying based on the image enhancement features to obtain a target gesture corresponding to the at least one gesture image.
7. The method of claim 6, wherein the classifying based on the image enhancement features to obtain the target gesture corresponding to the at least one gesture image comprises:
Calculating the similarity probability of the at least one gesture image and each gesture based on the image enhancement features;
and taking the gesture corresponding to the maximum value of the similarity probability as a target gesture.
8. The method of claim 6, wherein the recognition model is trained based on:
Acquiring a sample gesture image and a sample gesture corresponding to the sample gesture image;
Inputting the sample gesture image into an initial model to obtain a tag gesture output by the initial model in real time;
And determining a difference value between the sample gesture and the label gesture based on a preset loss function, and adjusting parameters of the initial model based on the difference value until the loss function meets a convergence condition to obtain an identification model.
9. A data processing apparatus, the apparatus comprising:
The determining module is used for determining the identity mark of the target object and gesture information associated with the identity mark; the gesture information comprises at least one preset gesture and a task corresponding to the gesture;
The recognition module is used for receiving at least one gesture image of the target object and recognizing a target gesture corresponding to the at least one gesture image;
And the execution module is used for determining a target task corresponding to the target gesture based on the gesture information and executing the target task.
10. An electronic device comprising a memory, a processor and a computer program stored on the memory, characterized in that the processor executes the computer program to carry out the steps of the method according to any one of claims 1 to 8.
11. A computer readable storage medium, on which a computer program is stored, characterized in that the computer program, when being executed by a processor, implements the steps of the method according to any one of claims 1 to 8.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211543650.1A CN118131893A (en) | 2022-12-01 | 2022-12-01 | Data processing method, device, electronic equipment and computer readable storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211543650.1A CN118131893A (en) | 2022-12-01 | 2022-12-01 | Data processing method, device, electronic equipment and computer readable storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN118131893A true CN118131893A (en) | 2024-06-04 |
Family
ID=91238353
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202211543650.1A Pending CN118131893A (en) | 2022-12-01 | 2022-12-01 | Data processing method, device, electronic equipment and computer readable storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN118131893A (en) |
-
2022
- 2022-12-01 CN CN202211543650.1A patent/CN118131893A/en active Pending
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111797893B (en) | Neural network training method, image classification system and related equipment | |
| CN111062871B (en) | Image processing method and device, computer equipment and readable storage medium | |
| CN110569377B (en) | Media file processing method and device | |
| CN108280477B (en) | Method and apparatus for clustering images | |
| EP3893125A1 (en) | Method and apparatus for searching video segment, device, medium and computer program product | |
| CN109034069B (en) | Method and apparatus for generating information | |
| CN107545241A (en) | Neural network model is trained and biopsy method, device and storage medium | |
| CN112381104B (en) | Image recognition method, device, computer equipment and storage medium | |
| CN113014566B (en) | Malicious registration detection method and device, computer readable medium and electronic device | |
| CN111652087A (en) | Car checking method and device, electronic equipment and storage medium | |
| WO2023173646A1 (en) | Expression recognition method and apparatus | |
| CN114419509B (en) | Multi-mode emotion analysis method and device and electronic equipment | |
| CN111898561B (en) | Face authentication method, device, equipment and medium | |
| CN112995757B (en) | Video clipping method and device | |
| CN114093022A (en) | Activity detection device, activity detection system, and activity detection method | |
| CN114550051A (en) | Vehicle loss detection method and device, computer equipment and storage medium | |
| CN112733645A (en) | Handwritten signature verification method and device, computer equipment and storage medium | |
| CN114219971A (en) | Data processing method, data processing equipment and computer readable storage medium | |
| CN111738199A (en) | Image information verification method, image information verification device, image information verification computing device and medium | |
| CN117132950A (en) | Vehicle tracking method, system, equipment and storage medium | |
| CN114373098B (en) | Image classification method, device, computer equipment and storage medium | |
| CN115880702A (en) | Data processing method, device, equipment, program product and storage medium | |
| CN115222047A (en) | Model training method, device, equipment and storage medium | |
| CN118131893A (en) | Data processing method, device, electronic equipment and computer readable storage medium | |
| CN112685588B (en) | Resource recommendation method, device, equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication |