CN114954321A - Vehicle control method and device and vehicle - Google Patents
Vehicle control method and device and vehicle Download PDFInfo
- Publication number
- CN114954321A CN114954321A CN202210479711.6A CN202210479711A CN114954321A CN 114954321 A CN114954321 A CN 114954321A CN 202210479711 A CN202210479711 A CN 202210479711A CN 114954321 A CN114954321 A CN 114954321A
- Authority
- CN
- China
- Prior art keywords
- human body
- vehicle
- information
- facial expression
- control
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60R—VEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
- B60R16/00—Electric or fluid circuits specially adapted for vehicles and not otherwise provided for; Arrangement of elements of electric or fluid circuits specially adapted for vehicles and not otherwise provided for
- B60R16/02—Electric or fluid circuits specially adapted for vehicles and not otherwise provided for; Arrangement of elements of electric or fluid circuits specially adapted for vehicles and not otherwise provided for electric constitutive elements
- B60R16/037—Electric or fluid circuits specially adapted for vehicles and not otherwise provided for; Arrangement of elements of electric or fluid circuits specially adapted for vehicles and not otherwise provided for electric constitutive elements for occupant comfort, e.g. for automatic adjustment of appliances according to personal settings, e.g. seats, mirrors, steering wheel
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60R—VEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
- B60R16/00—Electric or fluid circuits specially adapted for vehicles and not otherwise provided for; Arrangement of elements of electric or fluid circuits specially adapted for vehicles and not otherwise provided for
- B60R16/02—Electric or fluid circuits specially adapted for vehicles and not otherwise provided for; Arrangement of elements of electric or fluid circuits specially adapted for vehicles and not otherwise provided for electric constitutive elements
- B60R16/023—Electric or fluid circuits specially adapted for vehicles and not otherwise provided for; Arrangement of elements of electric or fluid circuits specially adapted for vehicles and not otherwise provided for electric constitutive elements for transmission of signals between vehicle parts or subsystems
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/10—Image acquisition
- G06V10/12—Details of acquisition arrangements; Constructional details thereof
- G06V10/14—Optical characteristics of the device performing the acquisition or on the illumination arrangements
- G06V10/147—Details of sensors, e.g. sensor lenses
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/764—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using classification, e.g. of video objects
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Multimedia (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- General Health & Medical Sciences (AREA)
- Health & Medical Sciences (AREA)
- Mechanical Engineering (AREA)
- Human Computer Interaction (AREA)
- Vascular Medicine (AREA)
- Artificial Intelligence (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Computing Systems (AREA)
- Databases & Information Systems (AREA)
- Evolutionary Computation (AREA)
- Medical Informatics (AREA)
- Software Systems (AREA)
- Image Analysis (AREA)
Abstract
The invention discloses a vehicle control method and device and a vehicle. The control method of the vehicle includes: acquiring three-dimensional image information detected by a three-dimensional camera mounted on the vehicle; identifying human body characteristic information in the three-dimensional image information; inputting the human body characteristic information into a preset human body posture classifier for matching to obtain a target human body posture and control parameters of a vehicle component controlled correspondingly; and controlling the vehicle component to operate according to the control parameter of the vehicle component. By adopting the technical means of the invention, parameter control of different components of the vehicle can be realized according to the human body posture of the user, the intellectualization of each component of the vehicle is improved, and the use experience of the user on the vehicle is effectively improved.
Description
Technical Field
The invention relates to the technical field of vehicle control, in particular to a vehicle control method and device and a vehicle.
Background
With the rapid development of science and technology and the gradual improvement of the living standard of people, automobiles become essential transportation tools in daily life of people. And with the diversification of the individuality of the automobile users, the requirements of the automobile users on the aspects of comfort level, attractiveness, technological sense and the like of the automobile are higher and higher.
However, the inventors found that the prior art has at least the following problems: in the prior art, functions and control methods of various components of an automobile are single, for example, a control method of a vehicle door is limited to be turned on or off according to the getting-on and getting-off requirements of a user, and a control method of an atmosphere lamp is limited to be turned on or off, so that the effect of the whole automobile is limited, and the requirements of the automobile user on quality and quality cannot be fully met.
Disclosure of Invention
The embodiment of the invention aims to provide a vehicle control method and device and a vehicle, which can realize parameter control on different components of the vehicle according to the human body posture of a user, improve the intellectualization of each component of the vehicle and effectively improve the use experience of the user on the vehicle.
In order to achieve the above object, an embodiment of the present invention provides a control method of a vehicle, including:
acquiring three-dimensional image information detected by a three-dimensional camera mounted on the vehicle;
identifying human body characteristic information in the three-dimensional image information;
inputting the human body characteristic information into a preset human body posture classifier for matching to obtain a target human body posture and control parameters of a vehicle component controlled correspondingly;
and controlling the vehicle component to operate according to the control parameter of the vehicle component.
As an improvement of the above scheme, the identifying human body feature information in the three-dimensional image information specifically includes:
determining a target human body area according to the three-dimensional image information;
identifying each preset bone joint point in the target human body region and three-dimensional coordinate information of the bone joint point;
and generating human skeleton characteristic information according to the three-dimensional coordinate information of each bone joint point so as to obtain the human characteristic information.
As an improvement of the above scheme, the human posture classifier comprises a human skeleton classifier;
the human body characteristic information is input into a preset human body posture classifier to be matched, and control parameters of a target human body posture and a vehicle component controlled correspondingly are obtained, and the method specifically comprises the following steps:
inputting the human body skeleton characteristic information into the human body skeleton classifier to judge whether a human body skeleton image formed by the human body skeleton characteristic information is a target human body skeleton image or not;
and when the human body skeleton image is judged to be the target human body skeleton image, acquiring the control parameters of the vehicle component correspondingly controlled by the target human body skeleton image.
As an improvement of the above scheme, the vehicle component correspondingly controlled by the target human body skeleton image comprises a vehicle door, the control parameter comprises an opening and closing state of the vehicle door, and the opening and closing state is an opening state or a closing state; the control parameter further includes an opening angle of the vehicle door in an open state.
As an improvement of the above scheme, the identifying human body feature information in the three-dimensional image information specifically includes:
determining a face region in the target human body region according to the three-dimensional image information;
identifying each facial expression motion unit, each preset facial feature point and three-dimensional coordinate information thereof in the facial area;
and generating facial expression feature information according to each facial expression motion unit, each facial feature point and the three-dimensional coordinate information thereof so as to obtain the human body feature information.
As an improvement of the above scheme, the human posture classifier further comprises a facial expression classifier;
the human body characteristic information is input into a preset human body posture classifier to be matched, and control parameters of a target human body posture and a vehicle component controlled correspondingly are obtained, and the method specifically comprises the following steps:
inputting the facial expression feature information into the facial expression classifier to judge whether a facial expression image formed by the facial expression feature information is a target facial expression image;
and when the facial expression image is judged to be the target facial expression image, acquiring control parameters of the vehicle component correspondingly controlled by the target facial expression image.
As an improvement of the above scheme, the vehicle component correspondingly controlled by the target facial expression image includes a vehicle-mounted display device, the vehicle-mounted display device includes a plurality of LED lamps, the control parameter includes an operating state of the LED lamp, and the operating state is an on state or an off state; the control parameters further include operating parameters of the LED lamp in an on state, and the operating parameters include at least one of a brightness level and a color.
An embodiment of the present invention further provides a control apparatus for a vehicle, including:
the three-dimensional image information acquisition module is used for acquiring three-dimensional image information detected by a three-dimensional camera installed on the vehicle;
the human body characteristic information identification module is used for identifying human body characteristic information in the three-dimensional image information;
the control parameter acquisition module is used for inputting the human body characteristic information into a preset human body posture classifier for matching to obtain a target human body posture and control parameters of a correspondingly controlled vehicle component;
a vehicle component control module to control the vehicle component to operate in accordance with the control parameter of the vehicle component.
An embodiment of the present invention further provides a control apparatus for a vehicle, including a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, where the processor implements a control method for a vehicle as described in any one of the above when executing the computer program.
The embodiment of the invention also provides a vehicle which comprises the control device of the vehicle.
Compared with the prior art, the vehicle control method and device and the vehicle are disclosed by the embodiment of the invention. Acquiring three-dimensional image information detected by a three-dimensional camera mounted on the vehicle; identifying human body characteristic information in the three-dimensional image information; inputting the human body characteristic information into a preset human body posture classifier for matching to obtain a target human body posture and control parameters of a vehicle component controlled correspondingly; and controlling the vehicle component to operate according to the control parameter of the vehicle component. By adopting the technical means of the embodiment of the invention, the human body posture of the user can be recognized, the parameter control of different components of the vehicle is realized through the human body posture, the user can swing different human body postures in front of the vehicle to control the operation of the vehicle components, the effect of dancing people and vehicles is realized, the intellectualization of each component of the vehicle is effectively improved, and the use experience of the user on the vehicle is improved.
Drawings
Fig. 1 is a schematic flow chart of a control method of a vehicle according to an embodiment of the present invention in a first implementation manner;
FIG. 2 is a schematic diagram of a three-dimensional camera detecting three-dimensional image information according to an embodiment of the present invention;
FIG. 3 is a schematic representation of a skeletal joint in an embodiment of the present invention;
FIG. 4 is a schematic diagram of training a human skeleton classifier according to an embodiment of the present invention;
fig. 5 is a schematic structural diagram of a control device of a vehicle according to an embodiment of the invention in a first implementation;
fig. 6 is a schematic structural diagram of a control device of a vehicle according to a second embodiment of the present invention.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
Referring to fig. 1, a schematic flow chart of a control method of a vehicle according to an embodiment of the present invention in a first implementation manner is shown. An embodiment of the present invention provides a control method of a vehicle, which may be executed by a controller provided in the vehicle, including steps S11 to S14:
s11, acquiring three-dimensional image information detected by a three-dimensional camera installed on the vehicle;
s12, identifying human body characteristic information in the three-dimensional image information;
s13, inputting the human body characteristic information into a preset human body posture classifier for matching to obtain a target human body posture and control parameters of a vehicle component controlled correspondingly;
and S14, controlling the vehicle component to operate according to the control parameters of the vehicle component.
In the embodiment of the invention, the vehicle is provided with the three-dimensional camera which is used for shooting three-dimensional image information in front of the mirror surface of the three-dimensional camera. The three-dimensional camera can be installed inside the vehicle or outside the vehicle, and a user can determine a specific installation position of the three-dimensional camera according to the requirement of the user, which is not limited herein.
Specifically, referring to fig. 2, the schematic diagram of the three-dimensional camera in the embodiment of the present invention detecting three-dimensional image information is shown, where the three-dimensional camera is a 3D DTOF depth camera, such as a Kinect camera, which can capture body movements and facial expressions of a human body in real time through a human body recognition and sensing technology, and the 3D motion sensing camera has a function of a depth camera, and acquires environmental three-dimensional position information in a field of view through a depth sensor by using an "optical coding" technology. Such depth data can be simply understood as an image captured by a special camera, but the data of each pixel thereof is not the pixel value of a normal color picture, but the position of the pixel is a distance from the camera, and therefore, the formed three-dimensional image information includes the distance depth data of each pixel.
The vehicle controller acquires the three-dimensional image information detected by the three-dimensional camera in real time, identifies and extracts the human body characteristic information, and inputs the human body characteristic information into a preset human body posture classifier for analysis and calculation after data preprocessing such as filtering and the like is carried out on the human body characteristic information, namely, the human body posture is identified.
It should be noted that the human posture classifier is a model which is constructed in advance and trained, and the training process of the human posture classifier is as follows: the method comprises the steps of collecting three-dimensional images containing various postures of a human body in advance, storing the three-dimensional images into an image database, exporting JSON or CSV files, filtering the data to enable the image to be more smoothly embodied, extracting human body characteristic values of the three-dimensional images as input characteristic vectors, constructing a model dictionary, classifying the data into a training data set and a testing data set, training a human body posture classifier through the data in the training data set, adjusting and optimizing through adjustment of a network structure according to model training comparison, selecting the most appropriate model, testing and further optimizing through the testing data set, and finally training to obtain the human body posture classifier. In practical application, the human body posture classifier can analyze the human body posture according to the input human body characteristic information.
Specifically, a static posture recognition result is obtained through calculation of the human posture classifier, and whether the target human posture is preset and labeled is judged. And if so, further acquiring control parameters of the vehicle component correspondingly controlled by the target human body posture, thereby realizing parameter control of the vehicle component.
Including but not limited to vehicle doors, windows, mood lights, seats, and stereo, etc. For example, when the human body posture classifier obtains the recognition result as a human body posture a and the human body posture a is preset to control the opening of the vehicle door, the controller controls the opening of the vehicle door according to the recognition result and the corresponding control parameter.
The embodiment of the invention provides a control method of a vehicle, which comprises the steps of obtaining three-dimensional image information detected by a three-dimensional camera arranged on the vehicle; identifying human body characteristic information in the three-dimensional image information; inputting the human body characteristic information into a preset human body posture classifier for matching to obtain a target human body posture and control parameters of a vehicle component controlled correspondingly; and controlling the vehicle component to operate according to the control parameter of the vehicle component. By adopting the technical means of the embodiment of the invention, the human body posture of the user can be recognized, the parameter control of different components of the vehicle can be realized through the human body posture, the user can put different human body postures in front of the vehicle to control the operation of the vehicle components, the effect of dancing people and vehicles is realized, the intellectualization of each component of the vehicle is effectively improved, and the use experience of the user on the vehicle is improved.
As a preferred implementation manner, in the embodiment of the present invention, step S12, that is, the identifying the human body characteristic information in the three-dimensional image information, specifically includes steps S121 to S123:
s121, determining a target human body area according to the three-dimensional image information;
s122, identifying each preset bone joint point in the target human body region and three-dimensional coordinate information of the bone joint point;
and S123, generating human skeleton characteristic information according to the three-dimensional coordinate information of each bone joint point to obtain the human characteristic information.
In the embodiment of the invention, the human body posture is determined mainly by identifying the human body skeletal joint points, so that the parameter control of the vehicle component is realized. Fig. 3 is a schematic diagram of a skeletal joint according to an embodiment of the present invention. The 20 key skeletal joint points of the human body are preset, including the head, the center of the shoulder, the left shoulder, the right shoulder, the left elbow, the right elbow, the left wrist, the right wrist, the left hand, the right hand, the spine, the center of the hip, the left hip, the right hip, the left knee, the right knee, the left ankle, the right ankle, the left foot and the right foot. Of course, the division of the human body bone joint points may be further refined or coarsened according to actual requirements, which is not specifically limited herein.
Specifically, the controller of the vehicle determines a target human body region according to the three-dimensional image information, and acquires three-dimensional coordinate information of each preset skeletal joint point through depth data included in the three-dimensional image information captured by the three-dimensional camera. The positions of the skeletal joint points are represented by (x, y, z) coordinates. The coordinate axes x, y, z are the spatial x, y, z coordinate axes of the depth sensor entity, which are expressed as skeleton space coordinates, which are right-handed spirals. For example, with respect to the three-dimensional camera Kinect, the Kinect camera is located at the origin, the z coordinate axis is consistent with the orientation sensed by the Kinect, the positive y-axis half extends upward, and the positive x-axis half extends leftward. And drawing a skeleton according to the acquired position data of each skeleton joint point to obtain human skeleton characteristic information as the human characteristic information.
Preferably, the human posture classifier comprises a human skeleton classifier; then, step S13, that is, the human body feature information is input into a preset human body posture classifier for matching, so as to obtain the target human body posture and the control parameters of the correspondingly controlled vehicle component, specifically including steps S131 to S132:
s131, inputting the human body skeleton characteristic information into the human body skeleton classifier to judge whether a human body skeleton image formed by the human body skeleton characteristic information is a target human body skeleton image;
s132, when the human body skeleton image is judged to be the target human body skeleton image, obtaining the control parameters of the vehicle component correspondingly controlled by the target human body skeleton image.
In the embodiment of the present invention, the human skeleton classifier is obtained by pre-training, and refer to fig. 4, which is a training schematic diagram of the human skeleton classifier in the embodiment of the present invention. The training data required to be obtained by the human skeleton classifier in the training process is a three-dimensional image containing various body motions of the human body, and the training process refers to the training process of the human posture classifier in the above embodiments, and is not described herein again.
Specifically, after the position information of each bone joint point is extracted from the three-dimensional image information to form human skeleton feature information, the human skeleton feature information is input into the human skeleton classifier and is matched with a trained target human skeleton image, so that whether the human skeleton feature information is the target human skeleton image or not is judged. And if so, acquiring control parameters of the vehicle component correspondingly controlled by the target human body skeleton image.
Preferably, the vehicle component correspondingly controlled by the target human body skeleton image comprises a vehicle door, the control parameter comprises an opening and closing state of the vehicle door, and the opening and closing state is an opening state or a closing state; the control parameter further includes an opening angle of the vehicle door in an open state.
According to the embodiment of the invention, the controller acquires the three-dimensional image information shot by the current three-dimensional camera in real time, analyzes and matches the morphological action of the user in real time, and can present the effect that the door of the vehicle swings according to the continuous action of the user when continuously outputting the corresponding control parameters to the door controller of the vehicle. As an example, the opening angle of the door in the open state is set according to the swing angle of the head of the user. When the human body skeleton classifier identifies different swing angles of the head of the current user, corresponding vehicle door control information can be continuously output to the vehicle door controller, and the vehicle door is controlled to be opened to the corresponding opening angle according to the different swing angles of the head of the user. In the actual presentation, when the user shakes his head with different amplitudes, the door will also correspondingly shake with different amplitudes.
By adopting the technical means of the embodiment of the invention, the opening and closing state of the vehicle door is controlled by identifying the morphological action formed by different skeletal joint points of the user, so that the intelligence of the vehicle door is improved, and the use experience of the user is enhanced.
As a preferred implementation manner, in the embodiment of the present invention, the step S12, that is, the identifying the human body feature information in the three-dimensional image information, specifically includes steps S124 to S126:
s124, determining a face area in the target human body area according to the three-dimensional image information;
s125, identifying each facial expression motion unit, each preset facial feature point and three-dimensional coordinate information of the facial expression motion unit in the facial area;
and S126, generating facial expression feature information according to each facial expression motion unit, each facial feature point and three-dimensional coordinate information of the facial expression motion units, so as to obtain the human body feature information.
In the embodiment of the invention, the human body posture is further determined mainly by recognizing the facial expression of the human body, so that the parameter control of the vehicle component is realized.
Specifically, after determining the target human body region according to the three-dimensional image information, the controller of the vehicle further determines a face region in the target human body region. And acquiring three-dimensional coordinate information of each facial expression action unit and each preset facial feature point in the facial area through depth data contained in three-dimensional image information shot by the three-dimensional camera.
It should be noted that the facial motion coding system (FACS) anatomically divides the human face into 44 facial motion units according to the motion of muscles, which are used to represent the motion of muscles in different areas of the human face. For example AU7 indicates whether the eyelids are tightened, AU23 indicates whether the lips are tightened, etc. Also, a plurality of facial feature points of the human body, such as the left eye, the right eye, the left eyebrow, the right eyebrow, the nose, the mouth, and the like, are set in advance. Of course, the division of the facial feature points may also be refined according to actual requirements, and is not specifically limited herein.
And generating facial expression feature information as the human body feature information according to the acquired facial expression motion units, the facial feature points and the three-dimensional coordinate information thereof.
Compared with the use of six basic expressions (anger, disgust, fear, happiness, sadness and surprise), the facial expression description method based on AU is more objective and finer in granularity, and can accurately analyze the facial expressions to understand the individual emotion, so that the recognition result of the facial expressions is more accurate.
Preferably, the human posture classifier further comprises a facial expression classifier; then, step S13, that is, inputting the human body feature information into a preset human body posture classifier for matching, to obtain a target human body posture and control parameters of a correspondingly controlled vehicle component, includes steps S133 to S134:
s133, inputting the facial expression feature information into the facial expression classifier to judge whether a facial expression image formed by the facial expression feature information is a target facial expression image;
and S134, when the facial expression image is judged to be the target facial expression image, acquiring the control parameters of the vehicle component correspondingly controlled by the target facial expression image.
In the embodiment of the present invention, the facial expression classifier is obtained by training in advance, the training data that needs to be obtained in the training process is a three-dimensional image including various facial expressions of a human body, and the training process of the training process refers to the training process of the human body posture classifier in the above embodiment, which is not described herein again.
Alternatively, the process of collecting facial expression image data is as follows: the performer makes expressions in a mode as standard as possible at a proper position in front of the three-dimensional camera Kinect, the performer expresses a certain preset target facial expression in a designated expression and facial posture, each expression is repeated for 10 times in each facial posture, then the recorded RGB images are evaluated, if the preset target facial expression is met, the corresponding RGB images, the three-dimensional coordinates of the facial expression motion units and the facial feature points are recorded, the performer is replaced to repeat the steps, repeated data are removed, a plurality of records of a plurality of performers are obtained, and the records are stored and classified.
Specifically, after the position information of each facial expression motion unit and each facial feature point in the facial area is extracted from the three-dimensional image information to form facial expression feature information, the facial expression feature information is input into the facial table classifier and is matched with a trained target facial expression image, and therefore whether the facial expression image is the target facial expression image or not is judged. If yes, control parameters of the vehicle component controlled correspondingly by the target facial expression image are obtained.
Preferably, the vehicle component correspondingly controlled by the target facial expression image comprises a vehicle-mounted display device, the vehicle-mounted display device comprises a plurality of LED lamps, the control parameter comprises an operating state of the LED lamps, and the operating state is an on state or an off state; the control parameters further include operating parameters of the LED lamp in an on state, the operating parameters including at least one of a brightness level and a color.
Specifically, the vehicle-mounted display device includes a vehicle-mounted atmosphere lamp, a vehicle exterior projection lamp, and the like. Optionally, the vehicle-mounted atmosphere lamp is an ISD intelligent interactive lamp provided with a plurality of LED lamps, and different shapes are displayed by lighting the corresponding LED lamps; the projection lamp outside the vehicle is provided with a plurality of LED lamps, and different shapes are projected by lighting the corresponding LED lamps.
According to the embodiment of the invention, the controller acquires the three-dimensional image information shot by the current three-dimensional camera in real time, analyzes and matches the facial expression of the user in real time, and continuously outputs the corresponding control parameters to the LED lamp controller of the vehicle to control the turning-on or turning-off of the LED lamp and the brightness and color in the turning-on state, so that the corresponding LED lamp on the IED intelligent interactive lamp or in the projection lamp can immediately present the effect of the graph corresponding to the facial expression of the user. By way of example, the expression graph formed after the LED lamp is turned on is correspondingly set according to the facial expression of the user. When the face expression that the user presents smile is recognized, the corresponding LED lamp is controlled to be lightened and present a smile expression graph, and the brightness or the color of the LED lamp can be changed according to the smile amplitude of the user.
By adopting the technical means of the embodiment of the invention, the control on the vehicle-mounted display equipment is realized by identifying different facial expressions of the user, the intellectualization of the vehicle-mounted display equipment is improved, and the use experience of the user is enhanced.
Preferably, after the three-dimensional image information captured by the three-dimensional camera is acquired, the human body posture recognition thread of the controller of the vehicle simultaneously performs human body posture recognition including recognition of the human body skeleton and facial expression. And determining a target human body region according to the three-dimensional image information, acquiring three-dimensional coordinate information of each preset skeleton joint point through depth data contained in the three-dimensional image information, generating human body skeleton characteristic information, inputting the human body skeleton characteristic information into the human body skeleton classifier, and matching the human body skeleton characteristic information with a trained target human body skeleton image so as to judge whether the human body skeleton characteristic information is the target human body skeleton image. And if so, acquiring control parameters of the vehicle door correspondingly controlled by the target human body skeleton image so as to control the opening or closing of the vehicle door and the opening angle in the opening state.
And after the controller determines the target human body area, further determining the facial area in the target human body area, acquiring the three-dimensional coordinate information of each facial expression action unit and each preset facial feature point in the facial area to form facial expression feature information, inputting the facial expression feature information into the facial table classifier, and matching the facial expression feature information with the trained target facial expression image to judge whether the facial expression image is the target facial expression image. If yes, obtaining control parameters of the LED lamp correspondingly controlled by the target facial expression image so as to control the on or off of the LED lamp of the vehicle-mounted display equipment and the color and/or the brightness of the LED lamp in the on state.
The main thread of the vehicle receives the recognition result from the human body posture recognition thread, two sub-threads of the vehicle door control thread and the LED lamp control thread are started simultaneously, the threads are asynchronously executed, the sub-threads receive and process the processed data information, the sub-threads analyze the commands again in order to ensure the data accuracy, when the analysis result is the correct posture action, the feature information is extracted from the Json data structure to form an operation signal to be sent out, and then the vehicle door and the LED vehicle lamp respectively make corresponding responses, so that the parameter control of the vehicle door and the parameter control of the LED lamp are realized.
By adopting the technical means of the embodiment of the invention, the opening and closing state of the vehicle door is controlled by identifying the morphological action formed by different skeletal joint points of the user, and the vehicle-mounted display equipment is controlled by identifying different facial expressions of the user, so that the intellectualization of the vehicle door and the vehicle-mounted display equipment is improved, and the use experience of the user is enhanced.
Fig. 5 is a schematic structural diagram of a control device of a vehicle according to an embodiment of the present invention in a first embodiment. An embodiment of the present invention provides a control device 20 for a vehicle, including: the system comprises a three-dimensional image information acquisition module 21, a human body characteristic information identification module 22, a control parameter acquisition module 23 and a vehicle component control module 24; wherein, the first and the second end of the pipe are connected with each other,
the three-dimensional image information acquisition module 21 is configured to acquire three-dimensional image information detected by a three-dimensional camera mounted on the vehicle;
the human body characteristic information identification module 22 is configured to identify human body characteristic information in the three-dimensional image information;
the control parameter obtaining module 23 is configured to input the human body feature information into a preset human body posture classifier for matching, so as to obtain a target human body posture and a control parameter of a correspondingly controlled vehicle component;
the vehicle component control module 24 is configured to control the vehicle component to operate according to the control parameter of the vehicle component.
By adopting the technical means of the embodiment of the invention, the human body posture of the user can be recognized, the parameter control of different components of the vehicle can be realized through the human body posture, the user can put different human body postures in front of the vehicle to control the operation of the vehicle components, the effect of dancing people and vehicles is realized, the intellectualization of each component of the vehicle is effectively improved, and the use experience of the user on the vehicle is improved.
As a preferred embodiment, the human body characteristic information identification module 22 is specifically configured to:
determining a target human body area according to the three-dimensional image information;
identifying each preset bone joint point in the target human body region and three-dimensional coordinate information of the bone joint point;
and generating human skeleton characteristic information according to the three-dimensional coordinate information of each bone joint point so as to obtain the human characteristic information.
Preferably, the human posture classifier comprises a human skeleton classifier; the control parameter obtaining module 23 is specifically configured to:
inputting the human body skeleton characteristic information into the human body skeleton classifier to judge whether a human body skeleton image formed by the human body skeleton characteristic information is a target human body skeleton image or not;
and when the human body skeleton image is judged to be the target human body skeleton image, acquiring the control parameters of the vehicle component correspondingly controlled by the target human body skeleton image.
Preferably, the vehicle component correspondingly controlled by the target human body skeleton image comprises a vehicle door, the control parameter comprises an opening and closing state of the vehicle door, and the opening and closing state is an opening state or a closing state; the control parameter further includes an opening angle of the vehicle door in an open state.
In a preferred embodiment, the human body feature information recognition module 22 is further configured to:
determining a face region in the target human body region according to the three-dimensional image information;
identifying each facial expression motion unit, each preset facial feature point and three-dimensional coordinate information thereof in the facial area;
and generating facial expression feature information according to each facial expression motion unit, each facial feature point and the three-dimensional coordinate information thereof so as to obtain the human body feature information.
Preferably, the human posture classifier further comprises a facial expression classifier; the control parameter obtaining module 23 is further configured to:
inputting the facial expression feature information into the facial expression classifier to judge whether a facial expression image formed by the facial expression feature information is a target facial expression image;
and when the facial expression image is judged to be the target facial expression image, acquiring the control parameters of the vehicle component correspondingly controlled by the target facial expression image.
Preferably, the vehicle component correspondingly controlled by the target facial expression image comprises a vehicle-mounted display device, the vehicle-mounted display device comprises a plurality of LED lamps, the control parameter comprises an operating state of the LED lamps, and the operating state is an on state or an off state; the control parameters further include operating parameters of the LED lamp in an on state, and the operating parameters include at least one of a brightness level and a color.
By adopting the technical means of the embodiment of the invention, the opening and closing state of the vehicle door is controlled by identifying the morphological action formed by different skeletal joint points of the user, and the vehicle-mounted display equipment is controlled by identifying different facial expressions of the user, so that the intellectualization of the vehicle door and the vehicle-mounted display equipment is improved, and the use experience of the user is enhanced.
Fig. 6 is a schematic structural diagram of a control device of a vehicle according to an embodiment of the present invention in a second embodiment. The embodiment of the present invention further provides a control device 30 of a vehicle, which includes a processor 31, a memory 32, and a computer program stored in the memory and configured to be executed by the processor, and when the processor executes the computer program, the processor implements the control method of the vehicle as provided in any of the above embodiments.
It should be noted that, the control device for a vehicle according to the embodiment of the present invention is used for executing all the process steps of the control method for a vehicle according to the embodiment, and the working principles and beneficial effects of the two are in one-to-one correspondence, so that details are not repeated.
It will be understood by those skilled in the art that all or part of the processes of the methods of the embodiments described above can be implemented by a computer program, which can be stored in a computer-readable storage medium, and when executed, can include the processes of the embodiments of the methods described above. The storage medium may be a magnetic disk, an optical disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), or the like.
The embodiment of the invention also provides a vehicle which comprises the control device of the vehicle provided by the embodiment.
While the foregoing is directed to the preferred embodiment of the present invention, it will be understood by those skilled in the art that various changes and modifications may be made without departing from the spirit and scope of the invention.
Claims (10)
1. A control method of a vehicle, characterized by comprising:
acquiring three-dimensional image information detected by a three-dimensional camera mounted on the vehicle;
identifying human body characteristic information in the three-dimensional image information;
inputting the human body characteristic information into a preset human body posture classifier for matching to obtain a target human body posture and control parameters of a vehicle component controlled correspondingly;
and controlling the vehicle component to operate according to the control parameter of the vehicle component.
2. The method for controlling a vehicle according to claim 1, wherein the identifying human body feature information in the three-dimensional image information specifically includes:
determining a target human body area according to the three-dimensional image information;
identifying each preset bone joint point in the target human body region and three-dimensional coordinate information of the bone joint point;
and generating human skeleton characteristic information according to the three-dimensional coordinate information of each bone joint point so as to obtain the human characteristic information.
3. The control method of a vehicle according to claim 2, wherein the human posture classifier includes a human skeleton classifier;
the human body characteristic information is input into a preset human body posture classifier to be matched, and control parameters of a target human body posture and a vehicle component controlled correspondingly are obtained, and the method specifically comprises the following steps:
inputting the human body skeleton characteristic information into the human body skeleton classifier to judge whether a human body skeleton image formed by the human body skeleton characteristic information is a target human body skeleton image or not;
and when the human body skeleton image is judged to be the target human body skeleton image, acquiring the control parameters of the vehicle component correspondingly controlled by the target human body skeleton image.
4. The vehicle control method according to claim 3, wherein the vehicle component controlled correspondingly by the target human skeleton image comprises a vehicle door, the control parameter comprises an opening and closing state of the vehicle door, and the opening and closing state is an opening state or a closing state; the control parameter further includes an opening angle of the vehicle door in an open state.
5. The method according to claim 1 or 2, wherein the identifying of the human body feature information in the three-dimensional image information specifically includes:
determining a face region in the target human body region according to the three-dimensional image information;
identifying each facial expression motion unit, each preset facial feature point and three-dimensional coordinate information thereof in the facial area;
and generating facial expression feature information according to each facial expression motion unit, each facial feature point and the three-dimensional coordinate information thereof so as to obtain the human body feature information.
6. The control method of a vehicle according to claim 5, wherein the human posture classifier further includes a facial expression classifier;
the method comprises the following steps of inputting the human body characteristic information into a preset human body posture classifier for matching, and obtaining a target human body posture and control parameters of a correspondingly controlled vehicle component, wherein the method specifically comprises the following steps:
inputting the facial expression feature information into the facial expression classifier to judge whether a facial expression image formed by the facial expression feature information is a target facial expression image;
and when the facial expression image is judged to be the target facial expression image, acquiring the control parameters of the vehicle component correspondingly controlled by the target facial expression image.
7. The control method of a vehicle according to claim 6, wherein the vehicle component controlled in correspondence with the target facial expression image includes an in-vehicle display device including a plurality of LED lamps, the control parameter includes an operation state of the LED lamps, and the operation state is an on state or an off state; the control parameters further include operating parameters of the LED lamp in an on state, and the operating parameters include at least one of a brightness level and a color.
8. A control apparatus of a vehicle, characterized by comprising:
the three-dimensional image information acquisition module is used for acquiring three-dimensional image information detected by a three-dimensional camera installed on the vehicle;
the human body characteristic information identification module is used for identifying human body characteristic information in the three-dimensional image information;
the control parameter acquisition module is used for inputting the human body characteristic information into a preset human body posture classifier for matching to obtain a target human body posture and control parameters of a correspondingly controlled vehicle component;
a vehicle component control module to control the vehicle component to operate in accordance with the control parameter of the vehicle component.
9. A control apparatus of a vehicle, characterized by comprising a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, the processor implementing the control method of the vehicle according to any one of claims 1 to 7 when executing the computer program.
10. A vehicle characterized by comprising the control apparatus of the vehicle according to claim 8 or 9.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210479711.6A CN114954321A (en) | 2022-05-05 | 2022-05-05 | Vehicle control method and device and vehicle |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210479711.6A CN114954321A (en) | 2022-05-05 | 2022-05-05 | Vehicle control method and device and vehicle |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN114954321A true CN114954321A (en) | 2022-08-30 |
Family
ID=82981226
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210479711.6A Pending CN114954321A (en) | 2022-05-05 | 2022-05-05 | Vehicle control method and device and vehicle |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114954321A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114347917A (en) * | 2021-12-28 | 2022-04-15 | 华人运通(江苏)技术有限公司 | Vehicle and vehicle-mounted camera system calibration method and device |
| CN116001725A (en) * | 2022-12-28 | 2023-04-25 | 阿维塔科技(重庆)有限公司 | Vehicle control method and device, vehicle and computer readable storage medium |
-
2022
- 2022-05-05 CN CN202210479711.6A patent/CN114954321A/en active Pending
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114347917A (en) * | 2021-12-28 | 2022-04-15 | 华人运通(江苏)技术有限公司 | Vehicle and vehicle-mounted camera system calibration method and device |
| CN114347917B (en) * | 2021-12-28 | 2023-11-10 | 华人运通(江苏)技术有限公司 | Calibration method and device for vehicle and vehicle-mounted camera system |
| CN116001725A (en) * | 2022-12-28 | 2023-04-25 | 阿维塔科技(重庆)有限公司 | Vehicle control method and device, vehicle and computer readable storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN114954321A (en) | Vehicle control method and device and vehicle | |
| US7689010B2 (en) | Facial feature analysis system | |
| Hunke et al. | Face locating and tracking for human-computer interaction | |
| Aran et al. | Signtutor: An interactive system for sign language tutoring | |
| JP7016522B2 (en) | Machine vision with dimensional data reduction | |
| CN107656613A (en) | A kind of man-machine interactive system and its method of work based on the dynamic tracking of eye | |
| KR20110139694A (en) | Method and system for gesture recognition | |
| KR20100086262A (en) | Robot and control method thereof | |
| JP2007257043A (en) | Occupant state estimating device and occupant state estimating method | |
| US20060148323A1 (en) | Facial feature analysis system | |
| CN109325408A (en) | A kind of gesture judging method and storage medium | |
| WO2010133661A1 (en) | Identifying facial expressions in acquired digital images | |
| KR20100066866A (en) | Portable facial expression training system and methods thereof | |
| CN109643153A (en) | Equipment for influencing the virtual objects of augmented reality | |
| Camurri et al. | Visual Gesture Recognition: from motion tracking to expressive gesture | |
| CN114967937B (en) | Virtual human motion generation method and system | |
| CN110825216A (en) | Method and system for man-machine interaction of driver during driving | |
| Patil et al. | Emotion recognition from 3D videos using optical flow method | |
| CN113128295A (en) | Method and device for identifying dangerous driving state of vehicle driver | |
| Boehme et al. | User localisation for visually-based human-machine-interaction | |
| CN110390311A (en) | A kind of video analysis algorithm based on attention and subtask pre-training | |
| EP1667049A2 (en) | Facial feature analysis system | |
| CN111881857A (en) | Electronic candle control method based on face detection | |
| Bhowmick et al. | A Framework for Eye-Based Human Machine Interface | |
| CN115402333A (en) | In-vehicle interaction control system and method based on driver emotion and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |