CN112215187A - Intelligent automobile door opening method and device, intelligent automobile and storage medium - Google Patents
Intelligent automobile door opening method and device, intelligent automobile and storage medium Download PDFInfo
- Publication number
- CN112215187A CN112215187A CN202011131926.6A CN202011131926A CN112215187A CN 112215187 A CN112215187 A CN 112215187A CN 202011131926 A CN202011131926 A CN 202011131926A CN 112215187 A CN112215187 A CN 112215187A
- Authority
- CN
- China
- Prior art keywords
- features
- image
- feature
- human face
- region
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 59
- 230000001815 facial effect Effects 0.000 claims abstract description 35
- 238000001514 detection method Methods 0.000 claims abstract description 33
- 238000011176 pooling Methods 0.000 claims description 43
- 238000000605 extraction Methods 0.000 claims description 25
- 238000012545 processing Methods 0.000 claims description 18
- 238000013527 convolutional neural network Methods 0.000 claims description 14
- 238000004590 computer program Methods 0.000 claims description 5
- 238000004891 communication Methods 0.000 abstract description 14
- 230000005540 biological transmission Effects 0.000 abstract description 7
- 230000008569 process Effects 0.000 abstract description 6
- 230000004044 response Effects 0.000 abstract description 6
- 238000005336 cracking Methods 0.000 abstract description 5
- 238000010586 diagram Methods 0.000 description 10
- 238000004364 calculation method Methods 0.000 description 7
- 230000006870 function Effects 0.000 description 7
- 238000013528 artificial neural network Methods 0.000 description 3
- 230000009286 beneficial effect Effects 0.000 description 2
- 238000004422 calculation algorithm Methods 0.000 description 2
- 230000008859 change Effects 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 239000000463 material Substances 0.000 description 2
- 238000013500 data storage Methods 0.000 description 1
- 238000003066 decision tree Methods 0.000 description 1
- 238000013135 deep learning Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000007689 inspection Methods 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 239000011159 matrix material Substances 0.000 description 1
- 238000010295 mobile communication Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 210000000056 organ Anatomy 0.000 description 1
- 230000008707 rearrangement Effects 0.000 description 1
- 239000007787 solid Substances 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 238000012549 training Methods 0.000 description 1
- 238000012546 transfer Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/16—Human faces, e.g. facial parts, sketches or expressions
- G06V40/161—Detection; Localisation; Normalisation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
- G06F18/2148—Generating training patterns; Bootstrap methods, e.g. bagging or boosting characterised by the process organisation or structure, e.g. boosting cascade
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
- G06V10/44—Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/16—Human faces, e.g. facial parts, sketches or expressions
- G06V40/161—Detection; Localisation; Normalisation
- G06V40/166—Detection; Localisation; Normalisation using acquisition arrangements
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/16—Human faces, e.g. facial parts, sketches or expressions
- G06V40/168—Feature extraction; Face representation
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Health & Medical Sciences (AREA)
- General Health & Medical Sciences (AREA)
- Data Mining & Analysis (AREA)
- Multimedia (AREA)
- Oral & Maxillofacial Surgery (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Artificial Intelligence (AREA)
- Evolutionary Computation (AREA)
- Life Sciences & Earth Sciences (AREA)
- Human Computer Interaction (AREA)
- General Engineering & Computer Science (AREA)
- Biomedical Technology (AREA)
- Computational Linguistics (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Biophysics (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Evolutionary Biology (AREA)
- Image Analysis (AREA)
Abstract
The invention discloses an intelligent automobile door opening method and device, an intelligent automobile and a storage medium. The method comprises the following steps: the method comprises the steps of obtaining a shooting image collected by a vehicle-mounted TOF camera, extracting a region image comprising a human face from the shooting image, extracting human face features from the region image, and controlling the opening of a vehicle door of the intelligent vehicle when the human face features are matched with the preset facial features of a vehicle owner. The user need not manual operation can unblock the door, has improved the convenience. In addition, the embodiment of the invention adopts the TOF camera to collect images, can better adapt to the low-light environment, and improves the identification distance, the identification precision and the response time. The facial features of the car owner are stored locally in the intelligent car, and communication with a remote server is not needed in the face detection and identification process, so that the problem of car owner information leakage caused by cracking in the data transmission process is avoided, and the safety of user information is ensured; meanwhile, the unlocking speed is improved because the communication with the server is not needed.
Description
Technical Field
The embodiment of the invention relates to the technical field of intelligent automobiles, in particular to an intelligent automobile door opening method and device, an intelligent automobile and a storage medium.
Background
Along with the continuous development of intelligent automobile technology, the user pays more and more attention to the consciousness of automobile safety and has great demand on the convenience of the intelligent automobile, so that the realization of convenient and safe functions through innovative technologies such as a camera and face recognition is a necessary trend.
At present, the unlocking and door opening of an automobile are mainly realized by remote unlocking and door opening, keyless entry and other modes. The remote control unlocking door opening needs manual door opening, and the door cannot be opened when two hands are inconvenient, so that great inconvenience is caused to a user. The keyless entry is relatively convenient, but the keyless entry is poor in safety and easy to crack.
Disclosure of Invention
The invention provides an intelligent automobile door opening method and device, an intelligent automobile and a storage medium, and aims to improve the convenience and safety of opening an intelligent automobile door.
In a first aspect, an embodiment of the present invention provides an intelligent automobile door opening method, including: acquiring a shot image acquired by a vehicle-mounted TOF camera;
extracting a region image including a human face from the shot image;
extracting human face features from the region image;
and when the face features are matched with the preset facial features of the car owner, controlling the car door of the intelligent car to be opened.
Optionally, the extracting a region image including a human face from the captured image includes:
carrying out gray level processing on the shot image to obtain a gray level image;
extracting Haar features from the gray-scale image, wherein the gray-scale image comprises a plurality of Haar features;
calculating a characteristic value of the Haar characteristic through an integral graph;
and inputting the Haar features and the feature values into an Adaboost cascade classifier, and determining the region image of the shot image including the human face.
Optionally, the extracting Haar features from the grayscale image includes:
traversing the gray level image through a preset detection window to obtain a plurality of Haar features.
Optionally, the Haar feature includes at least one first region and at least one second region, the first region and the second region have different pixel values, and the calculating the feature value of the Haar feature by an integral map includes:
respectively calculating pixel values of the first region and pixel values of the second region through the integral graph;
and calculating the difference value of the pixel value of the first area and the pixel value of the second area as the characteristic value of the Haar characteristic.
Optionally, extracting facial features from the region image includes:
and inputting the area image into a preset convolutional neural network for processing to obtain the human face characteristics.
Optionally, the convolutional neural network includes a first convolutional layer, a first pooling layer, a second convolutional layer, a second pooling layer, and a third convolutional layer, and the region image is input into a preset convolutional neural network for processing to obtain a face feature, including:
inputting the area image into the first convolution layer to carry out convolution operation to obtain a first characteristic;
inputting the first features into the first pooling layer to perform maximum pooling operation to obtain second features;
inputting the second feature into the second convolution layer for convolution operation to obtain a third feature;
inputting the third features into the second pooling layer for maximum pooling operation to obtain fourth features;
and inputting the fourth feature into the third convolution layer for convolution operation to obtain the human face feature.
Optionally, when the facial features match with the facial features of a preset vehicle owner, the door of the intelligent vehicle is controlled to be opened, including:
calculating the cosine similarity of the human face features and the facial features;
and controlling the opening of the door of the intelligent automobile based on the cosine similarity.
In a second aspect, an embodiment of the present invention further provides an intelligent automobile door opener, including:
the image acquisition module is used for acquiring a shot image acquired by the vehicle-mounted TOF camera;
the region image extraction module is used for extracting a region image comprising a human face from the shot image;
the face feature extraction module is used for extracting face features from the region image;
and the vehicle door control module is used for controlling the vehicle door of the intelligent vehicle to be opened when the human face characteristics are matched with the preset facial characteristics of the vehicle owner.
Optionally, the region image extracting module includes:
the gray processing submodule is used for carrying out gray processing on the shot image to obtain a gray image;
a Haar feature extraction submodule, configured to extract a Haar feature from the grayscale image, where the grayscale image includes a plurality of the Haar features;
the eigenvalue operator module is used for calculating the eigenvalue of the Haar characteristic through an integrogram;
and the area image determining submodule is used for inputting the Haar features and the feature values into an Adaboost cascade classifier and determining the area image of the shot image including the human face.
Optionally, the Haar feature extraction sub-module includes:
and the traversing unit is used for traversing the gray level image through a preset detection window to obtain a plurality of Haar features.
Optionally, the Haar features include at least one first region and at least one second region, the first region and the second region having different pixel values, and the feature value operator module includes:
an integral graph unit for calculating pixel values of the first region and pixel values of the second region respectively through the integral graph;
and the characteristic value calculation unit is used for calculating the difference value of the pixel value of the first region and the pixel value of the second region as the characteristic value of the Haar characteristic.
Optionally, the facial feature extraction module includes:
and the convolution submodule is used for inputting the area image into a preset convolution neural network for processing to obtain the human face characteristics.
Optionally, the convolutional neural network includes a first convolutional layer, a first pooling layer, a second convolutional layer, a second pooling layer, and a third convolutional layer, and the convolutional sub-module includes:
the first feature extraction unit is used for inputting the area image into the first convolution layer to carry out convolution operation to obtain a first feature;
the second feature extraction unit is used for inputting the first features into the first pooling layer to carry out maximum pooling operation to obtain second features;
a third feature extraction unit, configured to input the second feature into the second convolution layer for convolution operation to obtain a third feature;
a fourth feature extraction unit, configured to input the third feature into the second pooling layer to perform maximum pooling operation, so as to obtain a fourth feature;
and the face feature extraction unit is used for inputting the fourth feature into the third convolution layer to carry out convolution operation so as to obtain the face feature.
Optionally, the door control module includes:
the similarity operator module is used for calculating the cosine similarity between the human face features and the facial features;
and the vehicle door control submodule is used for controlling the opening of the vehicle door of the intelligent vehicle based on the cosine similarity.
In a third aspect, an embodiment of the present invention further provides an intelligent vehicle, including:
one or more processors;
storage means for storing one or more programs;
when the one or more programs are executed by the one or more processors, the one or more processors implement the intelligent automobile door opening method according to the first aspect of the present invention.
In a fourth aspect, the embodiment of the present invention further provides a computer-readable storage medium, on which a computer program is stored, where the computer program, when executed by a processor, implements the intelligent automobile door opening method according to the first aspect of the present invention.
The intelligent automobile door opening method provided by the embodiment of the invention comprises the following steps: the method comprises the steps of obtaining a shooting image collected by a vehicle-mounted TOF camera, extracting a region image comprising a human face from the shooting image, extracting human face features from the region image, and controlling the opening of a vehicle door of the intelligent vehicle when the human face features are matched with the preset facial features of a vehicle owner. The user need not manual operation can unblock the door, has improved the convenience. In addition, the embodiment of the invention adopts the TOF camera to collect images, can better adapt to the low-light environment, and improves the identification distance, the identification precision and the response time. The facial features of the car owner are stored locally in the intelligent car, and communication with a remote server is not needed in the face detection and identification process, so that the problem of car owner information leakage caused by cracking in the data transmission process is avoided, and the safety of user information is ensured; meanwhile, the unlocking speed is improved because the communication with the server is not needed.
Drawings
Fig. 1 is a flowchart of an intelligent vehicle door opening method according to an embodiment of the present invention;
fig. 2A is a flowchart of an intelligent vehicle door opening method according to a second embodiment of the present invention;
FIG. 2B is a schematic diagram of computing Haar features according to a second embodiment of the present invention;
fig. 2C is a schematic structural diagram of an Adaboost cascade classifier according to a second embodiment of the present invention;
fig. 2D is a schematic structural diagram of a convolutional neural network according to an embodiment of the present invention;
fig. 3 is a schematic structural diagram of an intelligent automobile door opener according to a third embodiment of the present invention;
fig. 4 is a schematic structural diagram of an intelligent vehicle according to a fourth embodiment of the present invention.
Detailed Description
The present invention will be described in further detail with reference to the accompanying drawings and examples. It is to be understood that the specific embodiments described herein are merely illustrative of the invention and are not limiting of the invention. It should be further noted that, for the convenience of description, only some of the structures related to the present invention are shown in the drawings, not all of the structures.
Example one
Fig. 1 is a flowchart of an intelligent automobile door opening method according to an embodiment of the present invention, where this embodiment is applicable to a situation where an intelligent automobile safely and conveniently opens a door, and the method may be executed by an intelligent automobile door opening device according to an embodiment of the present invention, where the device may be implemented by software and/or hardware, and is generally configured in an intelligent automobile, as shown in fig. 1, where the method specifically includes the following steps:
and S101, acquiring a shot image acquired by the vehicle-mounted TOF camera.
Specifically, a TOF (Time of Flight) camera transmits modulated near-infrared light, reflects the light after encountering an object, is received by the camera, calculates a Time difference or a phase difference between light transmission and reflection, and collects data to form a group of distance depth data, thereby obtaining a 3D image. The TOF camera has the advantages of long-distance identification, high identification precision, strong ambient light adaptability, quick response time and the like, can better adapt to a weak light environment, and improves identification distance, identification precision and response time. The TOF camera can be arranged on an A column, close to a driving seat, of the intelligent automobile and is connected with a controller of the intelligent automobile.
The captured image acquired by the TOF camera may be a video stream or a picture, which is not limited herein. When a video stream is acquired, one or more frames of pictures can be cut from the video stream for subsequent face detection and recognition.
It should be noted that, in the foregoing embodiment, a TOF camera is taken as an example of the image acquisition device to describe the embodiment of the present invention, in other embodiments of the present invention, other infrared or near-infrared cameras may also be taken as the image acquisition device, and the embodiment of the present invention is not limited herein.
S102, extracting an area image comprising a human face from the shot image.
Specifically, the feature (for example, Haar feature) conforming to the face in the window may be detected through the detection window, and the area image including the face in the captured image may be determined by continuously changing the position of the detection window and the size of the detection window, or continuously changing the position of the detection window and the size of the captured image. In general, a region including a human face is framed out of a captured image using a rectangular frame as a region image.
It should be noted that, in the foregoing embodiment, the embodiment of the present invention is described by taking a detection window to detect a face as an example, in other embodiments of the present invention, face detection may also be performed based on deep learning, for example, dpm (robust Part model), Cascade CNN, and the like, which is not limited herein.
And S103, extracting the face features from the region image.
Specifically, the region image obtained in the above steps is processed, for example, convolved, pooled, and the like, to obtain a face feature for characterizing a face organ of a human face.
And S104, controlling the opening of the door of the intelligent automobile when the face characteristics are matched with the preset facial characteristics of the automobile owner.
Specifically, the internal storage of intelligent automobile stores the facial characteristics of the automobile owner, the collected facial characteristics are compared with the facial characteristics of the automobile owner, and when the facial characteristics are matched with the preset facial characteristics of the automobile owner, the door of the intelligent automobile is controlled to be opened.
In the embodiment of the invention, the facial features of the car owner are stored locally in the intelligent car, and the communication with a remote server is not needed in the process of face detection and identification, so that the problem of car owner information leakage caused by cracking in the data transmission process is avoided, and the safety of user information is ensured; meanwhile, the unlocking speed is improved because the communication with the server is not needed.
The intelligent automobile door opening method provided by the embodiment of the invention comprises the following steps: the method comprises the steps of obtaining a shooting image collected by a vehicle-mounted TOF camera, extracting a region image comprising a human face from the shooting image, extracting human face features from the region image, and controlling the opening of a vehicle door of the intelligent vehicle when the human face features are matched with the preset facial features of a vehicle owner. The user need not manual operation can unblock the door, has improved the convenience. In addition, the embodiment of the invention adopts the TOF camera to collect images, can better adapt to the low-light environment, and improves the identification distance, the identification precision and the response time. The facial features of the car owner are stored locally in the intelligent car, and communication with a remote server is not needed in the face detection and identification process, so that the problem of car owner information leakage caused by cracking in the data transmission process is avoided, and the safety of user information is ensured; meanwhile, the unlocking speed is improved because the communication with the server is not needed.
Example two
An embodiment of the present invention provides an intelligent automobile door opening method, and fig. 2A is a flowchart of the intelligent automobile door opening method provided by the embodiment of the present invention, and this embodiment performs refinement on the basis of the embodiment one described above, and describes in detail an extraction process of a region image and an extraction process of a face feature, as shown in fig. 2A, the method includes:
s201, acquiring a shot image collected by the vehicle-mounted TOF camera.
Specifically, the TOF camera can be arranged on a column a of the intelligent automobile close to the driving seat and connected with a controller of the intelligent automobile. The captured image acquired by the TOF camera may be a video stream or a picture, which is not limited herein.
And S202, carrying out gray scale processing on the shot image to obtain a gray scale image.
In particular, the picture taken is typically a color picture comprising R, G, B three color channels, each pixel having three color components. In order to reduce the data processing amount, a color picture is usually subjected to gradation processing and converted into a gradation picture. It is for each pixel in the pixel matrix to satisfy the following relationship: R-G-B (i.e., the value of the red component, the value of the green component, and the value of the blue component, which are equal), this value is called the gray value.
Specifically, the gradation processing of the image may be realized by a component method, a maximum value method, an average value method, a weighted average method, or the like.
Illustratively, in a specific embodiment of the present invention, graying of the image is achieved by a weighted average method. The gray value of the pixel is obtained by respectively giving different weights to the red component, the green component and the blue component of the pixel at any position (i, j) and summing the weights. Specifically, the calculation formula of the weighted average method is as follows:
Gray(i,j)＝0.299R(i,j)+0.578G(i,j)+0.114B(i,j)
and S203, extracting Haar features from the gray level image, wherein the gray level image comprises a plurality of Haar features.
Specifically, the gray level image is traversed through a preset detection window, and a plurality of Haar features are obtained. The detection window may be a 24 x 24 pixel sized sliding window, the window including a feature template having both white and black rectangles therein and defining a feature value of the template as the sum of the white rectangle pixel values minus the sum of the black rectangle pixel values. The feature templates may include edge feature templates, linear feature templates, center feature templates, and diagonal feature templates. By varying the size and position of the feature template, a large number of features may be exhausted in the image sub-window. Features obtained by expanding (translating and stretching) the feature template in the detection window are called Haar features or rectangular features. The Haar characteristic value reflects the gray level change condition of the image. For example: some features of the face can be described simply by rectangular features, such as: the eyes are darker than the cheeks, the sides of the bridge of the nose are darker than the bridge of the nose, the mouth is darker than the surroundings, etc.
The Haar feature comprises at least one first region (white region) and at least one second region (black region), the first and second regions having different pixel values. The pixel value is a value given by a computer when an image of an original is digitized, and represents average luminance information of a certain small block of the original, or average reflection (transmission) density information of the small block. If for an image that is itself gray scale, its pixel value is its gray scale value.
The Haar features can be located at any position of the image, and the size can be changed at will, so that the change of the category, the size and the position causes a very small detection window to contain a great number of Haar features.
And S204, calculating a characteristic value of the Haar characteristic through the integral graph.
The characteristic value of the Haar feature is the sum of white rectangular pixel values minus the sum of black rectangular pixel values in the Haar feature. As mentioned above, the Haar features can be located at any position of the image, and the size of the Haar features can be changed at will, so that a very small detection window contains a great number of Haar features, and the feature values of the Haar features are functions of three factors, namely the rectangular template type, the rectangular position and the rectangular size, so that the feature value calculation of the Haar features is a very huge project.
Therefore, in the embodiment of the invention, the characteristic value of the Haar characteristic is calculated through the integral graph, and the integral graph is a quick algorithm which can work out the pixel sum of all areas in the image only by traversing the image once, so that the efficiency of calculating the characteristic value of the image is greatly improved. The integral graph has the main idea that the sum of pixels of rectangular areas formed by the image from a starting point to each point is stored in a memory as an element of an array, when the pixel sum of a certain area needs to be calculated, the element of the array can be directly indexed, the pixel sum of the area does not need to be recalculated, and therefore calculation is accelerated (which is called as a dynamic programming algorithm). The integral map can use the same time (constant time) to compute different features at multiple scales, thus greatly improving the detection speed.
The integral map is constructed in such a way that the value ii (i, j) at position (i, j) is the sum of all pixel values f (k, l) in the upper left-hand corner direction of the original image (i, j), expressed mathematically as:
fig. 2B is a schematic diagram of computing a Haar feature according to the second embodiment of the present invention, and exemplarily, as shown in fig. 2B, the sum of pixel values in the region AB and a for the Haar feature is:
ii(5)+ii(1)-ii(2)-ii(4)
the sum of the pixel values of the B region is:
ii(6)+ii(2)-ii(5)-ii(3)
the eigenvalues of the Haar signature are:
ii(5)+ii(1)-ii(2)-ii(4)-[ii(6)+ii(2)-ii(5)-ii(3)]
wherein ii (5) represents the sum of all pixel values in the upper left corner direction of the coordinate point 5 in the figure, and the others are similar. Therefore, the eigenvalues of the Haar features are related only to the integrogram of the end points of the Haar features and not to the coordinates of the image. By calculating the integral graph of the end point of the Haar characteristic and then performing simple addition and subtraction operation, the characteristic value can be obtained, and therefore, the calculation speed of the characteristic value is greatly increased, and the detection speed of the target is also increased.
And S205, inputting the Haar features and the feature values into an Adaboost cascade classifier, and determining a region image including the human face in the shot image.
Fig. 2C is a schematic structural diagram of an Adaboost cascade classifier according to a second embodiment of the present invention, for example, as shown in fig. 2C, the Adaboost cascade classifier includes a plurality of cascade strong classifiers, and each strong classifier includes a plurality of weak classifiers in a branch structure. When the Haar features and feature values pass through all the strong classifiers, the detection window is determined as a region image including the human face. Since the discrimination accuracy of each strong classifier on the negative samples is very high, once the detected target bit negative sample is found, the following strong classifiers are not called continuously, and much detection time is reduced. Since many areas to be detected in one image are negative samples, the cascade classifier abandons the complex detection of many negative samples in the early stage of the classifier, so the speed of the cascade classifier is very high; only the positive samples are sent to the next strong classifier for secondary inspection, so that the probability of false positive (false positive) of the finally output positive samples is very low, and the detection accuracy is improved.
One weak classifier is a simple decision tree, each Haar feature corresponds to one weak classifier, and the mathematical expression of the weak classifier is as follows:
wherein f is a Haar feature, θ is a threshold, p indicates the direction of the unequal sign, and x represents a detection window.
The mathematical expression of the strong classifier is as follows:
wherein,
εtis the classification error of the weak classifier.
In the embodiment of the invention, a detection window is continuously enlarged, searching is carried out, Haar features and characteristic values in the window are input into an Adaboost cascade classifier, the Adaboost cascade classifier continuously screens the Haar features and the characteristic values, and the Haar features and the characteristic values are discarded or passed, so that whether the detection window is a regional image including a human face or not is finally determined.
And S206, inputting the area image into a preset convolutional neural network for processing to obtain the human face characteristics.
Fig. 2D is a schematic structural diagram of a convolutional neural network according to an embodiment of the present invention, and for example, as shown in fig. 2D, the convolutional neural network includes a first convolutional layer, a first pooling layer, a second convolutional layer, a second pooling layer, and a third convolutional layer.
The convolution neural network processes the area image as follows:
1. and inputting the area image into the first convolution layer for convolution operation to obtain a first characteristic.
Specifically, the convolution kernel size of the first convolution layer is 5 × 5, and the convolution step size is 2 × 2. The first convolution layer performs convolution operation on the input area image according to the parameters to obtain a first characteristic.
2. And inputting the first characteristics into the first pooling layer to carry out maximum pooling operation to obtain second characteristics.
Specifically, the pooling window size of the first pooling layer is 3 × 3, and the step size is 2 × 2. And the first pooling layer receives the first characteristics and performs maximum pooling operation according to the parameters to obtain second characteristics. And the maximum pooling operation is that the maximum number in the local area of the first feature selected in the pooling window represents the area, and the maximum feature in each channel data of the first feature is reserved to obtain a second feature. The pooling is to reduce the number of training parameters, reduce the dimensionality of the first feature output by the first convolution layer, reduce the over-fitting phenomenon, only retain the most useful feature information, and reduce the transfer of noise. In other embodiments of the present invention, the first pooling layer may also perform an average pooling operation on the first features to obtain the second features, which is not limited herein.
3. And inputting the second characteristic into a second convolution layer for convolution operation to obtain a third characteristic.
Specifically, the convolution kernel size of the second convolution layer is 3 × 3, and the convolution step size is 1 × 1. And the second convolution layer performs convolution operation on the input second characteristic according to the parameters to obtain a third characteristic.
4. And inputting the third features into the second pooling layer for maximum pooling operation to obtain fourth features.
Specifically, the pooling window size of the second pooling layer is 3 × 3, and the step size is 2 × 2. And the second pooling layer receives the third characteristic, and performs maximum pooling operation according to the parameters to obtain a fourth characteristic.
5. And inputting the fourth feature into the third convolution layer for convolution operation to obtain the human face feature.
Specifically, the convolution kernel size of the third convolution layer is 1 × 1, and the convolution step size is 1 × 1. And the third convolution layer performs convolution operation on the input fourth feature according to the parameters to obtain a one-dimensional face feature.
It should be noted that, in the foregoing embodiment, the convolutional neural network includes three convolutional layers as an example, and the present invention is exemplarily described, in other embodiments of the present invention, the number of convolutional layers in the convolutional neural network may be set according to needs, and the embodiments of the present invention are not limited herein.
And S207, calculating the cosine similarity between the human face features and the facial features.
Specifically, the internal storage of intelligent automobile stores the facial characteristics of the automobile owner, the collected facial characteristics are compared with the facial characteristics of the automobile owner, and when the facial characteristics are matched with the preset facial characteristics of the automobile owner, the door of the intelligent automobile is controlled to be opened.
Illustratively, in the embodiment of the present invention, the cosine similarity between the face feature and the face feature is calculated, and specifically, the calculation formula of the cosine similarity is as follows:
wherein A is the face feature, B is the face feature, cos (A, B) is the cosine similarity of the face feature and the face feature.
And S208, controlling the opening of the door of the intelligent automobile based on the cosine similarity.
Specifically, after the cosine similarity is obtained through the steps, the cosine similarity is compared with a preset value, if the cosine similarity is larger than the preset value, the face feature is determined to be matched with the preset face feature of the car owner, and at the moment, the opening of the car door of the intelligent car is controlled.
According to the intelligent automobile door opening method provided by the embodiment of the invention, a user can unlock the automobile door without manual operation, so that the convenience is improved. Adopt the TOF camera to gather the image, can adapt to the low light environment better to and improve discernment distance, discernment precision and response time. The facial features of the car owner are stored locally in the intelligent car, and communication with a remote server is not needed in the face detection and identification process, so that the problem of car owner information leakage caused by cracking in the data transmission process is avoided, and the safety of user information is ensured; meanwhile, the unlocking speed is improved because the communication with the server is not needed. The human face detection is carried out through the Adaboost cascade classifier, so that the detection accuracy and the detection speed are improved.
EXAMPLE III
Fig. 3 is a schematic structural diagram of an intelligent automobile door opener provided in a third embodiment of the present invention, and as shown in fig. 3, the intelligent automobile door opener includes:
the image acquisition module 301 is used for acquiring a shot image acquired by the vehicle-mounted TOF camera;
a region image extraction module 302, configured to extract a region image including a human face from the captured image;
a face feature extraction module 303, configured to extract face features from the region image;
and the vehicle door control module 304 is configured to control a vehicle door of the intelligent vehicle to open when the human face features are matched with preset facial features of a vehicle owner.
In some embodiments of the present invention, the region image extraction module 302 comprises:
the gray processing submodule is used for carrying out gray processing on the shot image to obtain a gray image;
a Haar feature extraction submodule, configured to extract a Haar feature from the grayscale image, where the grayscale image includes a plurality of the Haar features;
the eigenvalue operator module is used for calculating the eigenvalue of the Haar characteristic through an integrogram;
and the area image determining submodule is used for inputting the Haar features and the feature values into an Adaboost cascade classifier and determining the area image of the shot image including the human face.
In some embodiments of the invention, the Haar feature extraction sub-module comprises:
and the traversing unit is used for traversing the gray level image through a preset detection window to obtain a plurality of Haar features.
In some embodiments of the invention, the Haar features comprise at least one first region and at least one second region, the first region and the second region having different pixel values, the feature value operator module comprising:
an integral graph unit for calculating pixel values of the first region and pixel values of the second region respectively through the integral graph;
and the characteristic value calculation unit is used for calculating the difference value of the pixel value of the first region and the pixel value of the second region as the characteristic value of the Haar characteristic.
In some embodiments of the present invention, the facial feature extraction module 303 includes:
and the convolution submodule is used for inputting the area image into a preset convolution neural network for processing to obtain the human face characteristics.
In some embodiments of the invention, the convolutional neural network comprises a first convolutional layer, a first pooling layer, a second convolutional layer, a second pooling layer, and a third convolutional layer, the convolutional submodule comprises:
the first feature extraction unit is used for inputting the area image into the first convolution layer to carry out convolution operation to obtain a first feature;
the second feature extraction unit is used for inputting the first features into the first pooling layer to carry out maximum pooling operation to obtain second features;
a third feature extraction unit, configured to input the second feature into the second convolution layer for convolution operation to obtain a third feature;
a fourth feature extraction unit, configured to input the third feature into the second pooling layer to perform maximum pooling operation, so as to obtain a fourth feature;
and the face feature extraction unit is used for inputting the fourth feature into the third convolution layer to carry out convolution operation so as to obtain the face feature.
In some embodiments of the present invention, the door control module 304 comprises:
the similarity operator module is used for calculating the cosine similarity between the human face features and the facial features;
and the vehicle door control submodule is used for controlling the opening of the vehicle door of the intelligent vehicle based on the cosine similarity.
The intelligent automobile door opener can execute the method provided by any embodiment of the invention, and has the corresponding functional modules and beneficial effects of the execution method.
Example four
An embodiment of the present invention fourth provides an intelligent vehicle, fig. 4 is a schematic structural diagram of the intelligent vehicle according to the fourth embodiment of the present invention, and as shown in fig. 4, the intelligent vehicle includes:
a processor 401, a memory 402, a communication module 403, an input device 404, and an output device 405; the number of the processors 401 in the intelligent automobile can be one or more, and one processor 401 is taken as an example in fig. 4; the processor 401, the memory 402, the communication module 403, the input device 404 and the output device 405 in the smart car may be connected by a bus or other means, and the bus connection is exemplified in fig. 4. The processor 401, memory 402, communication module 403, input device 404, and output device 405 described above may be integrated on a smart car.
The memory 402 is a computer-readable storage medium, and can be used for storing software programs, computer-executable programs, and modules, such as the modules corresponding to the intelligent automobile door opening method in the foregoing embodiments. The processor 401 executes various functional applications and data processing of the intelligent automobile by running software programs, instructions and modules stored in the memory 402, so as to implement the intelligent automobile door opening method.
The memory 402 may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application program required for at least one function; the storage data area may store data created according to use of the microcomputer, and the like. Further, the memory 402 may include high speed random access memory, and may also include non-volatile memory, such as at least one magnetic disk storage device, flash memory device, or other non-volatile solid state storage device. In some examples, the memory 402 may further include memory located remotely from the processor 401, which may be connected to an electronic device through a network. Examples of such networks include, but are not limited to, the internet, intranets, local area networks, mobile communication networks, and combinations thereof.
And a communication module 403, configured to establish a connection with an external device (e.g., an intelligent terminal), and implement data interaction with the external device. The input device 404 may be used to receive input numeric or character information and generate key signal inputs related to user settings and function control.
The intelligent automobile provided by the embodiment can execute the intelligent automobile door opening method provided by the first embodiment and the second embodiment of the invention, and has corresponding functions and beneficial effects.
EXAMPLE five
The fifth embodiment of the present invention provides a storage medium containing computer-executable instructions, where a computer program is stored on the storage medium, and when the computer program is executed by a processor, the method for opening a door of an intelligent vehicle according to any of the above embodiments of the present invention is implemented.
Of course, the storage medium containing the computer-executable instructions provided by the embodiments of the present invention is not limited to the method operations described above, and may also perform related operations in the intelligent vehicle door opening method provided by the embodiments of the present invention.
It should be noted that, for the device, the smart car and the storage medium embodiment, since they are basically similar to the method embodiment, the description is relatively simple, and the relevant points can be referred to the partial description of the method embodiment.
From the above description of the embodiments, it is obvious for those skilled in the art that the present invention can be implemented by software and necessary general hardware, and certainly, can also be implemented by hardware, but the former is a better embodiment in many cases. Based on such understanding, the technical solution of the present invention may be embodied in the form of a software product, and the computer software product may be stored in a computer-readable storage medium, such as a floppy disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a FLASH Memory (FLASH), a hard disk or an optical disk of a computer, and includes several instructions for enabling an intelligent vehicle to execute the intelligent vehicle door opening method according to any embodiment of the present invention.
It should be noted that, in the above apparatus, each included module and unit are merely divided according to functional logic, but are not limited to the above division as long as the corresponding functions can be implemented; in addition, the specific names of the functional modules are only for convenience of distinguishing from each other and are not used for limiting the protection scope of the present invention.
It should be understood that portions of the present invention may be implemented in hardware, software, firmware, or a combination thereof. In the above embodiments, the various steps or methods may be implemented in software or firmware stored in memory and executed by suitable instruction execution devices. For example, if implemented in hardware, as in another embodiment, any one or combination of the following techniques, which are known in the art, may be used: a discrete logic circuit having a logic gate circuit for implementing a logic function on a data signal, an application specific integrated circuit having an appropriate combinational logic gate circuit, a Programmable Gate Array (PGA), a Field Programmable Gate Array (FPGA), or the like.
In the description herein, references to the description of the term "one embodiment," "some embodiments," "an example," "a specific example," or "some examples," etc., mean that a particular feature, structure, material, or characteristic described in connection with the embodiment or example is included in at least one embodiment or example of the invention. In this specification, the schematic representations of the terms used above do not necessarily refer to the same embodiment or example. Furthermore, the particular features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples.
It is to be noted that the foregoing is only illustrative of the preferred embodiments of the present invention and the technical principles employed. It will be understood by those skilled in the art that the present invention is not limited to the particular embodiments described herein, but is capable of various obvious changes, rearrangements and substitutions as will now become apparent to those skilled in the art without departing from the scope of the invention. Therefore, although the present invention has been described in greater detail by the above embodiments, the present invention is not limited to the above embodiments, and may include other equivalent embodiments without departing from the spirit of the present invention, and the scope of the present invention is determined by the scope of the appended claims.
Claims (10)
1. An intelligent automobile door opening method is characterized by comprising the following steps:
acquiring a shot image acquired by a vehicle-mounted TOF camera;
extracting a region image including a human face from the shot image;
extracting human face features from the region image;
and when the face features are matched with the preset facial features of the car owner, controlling the car door of the intelligent car to be opened.
2. The intelligent automobile door opening method according to claim 1, wherein the extracting of the area image including the human face from the shot image comprises:
carrying out gray level processing on the shot image to obtain a gray level image;
extracting Haar features from the gray-scale image, wherein the gray-scale image comprises a plurality of Haar features;
calculating a characteristic value of the Haar characteristic through an integral graph;
and inputting the Haar features and the feature values into an Adaboost cascade classifier, and determining the region image of the shot image including the human face.
3. The intelligent automobile door opening method according to claim 2, wherein the extracting Haar features from the grayscale image comprises:
traversing the gray level image through a preset detection window to obtain a plurality of Haar features.
4. The intelligent automobile door opening method according to claim 2, wherein the Haar feature comprises at least one first region and at least one second region, the first region and the second region have different pixel values, and the calculating the feature value of the Haar feature through an integral map comprises:
respectively calculating pixel values of the first region and pixel values of the second region through the integral graph;
and calculating the difference value of the pixel value of the first area and the pixel value of the second area as the characteristic value of the Haar characteristic.
5. The intelligent automobile door opening method according to any one of claims 1 to 4, wherein extracting human face features from the region image comprises:
and inputting the area image into a preset convolutional neural network for processing to obtain the human face characteristics.
6. The intelligent automobile door opening method according to claim 5, wherein the convolutional neural network comprises a first convolutional layer, a first pooling layer, a second convolutional layer, a second pooling layer and a third convolutional layer, and the region image is input into a preset convolutional neural network for processing to obtain the human face features, wherein the method comprises the following steps:
inputting the area image into the first convolution layer to carry out convolution operation to obtain a first characteristic;
inputting the first features into the first pooling layer to perform maximum pooling operation to obtain second features;
inputting the second feature into the second convolution layer for convolution operation to obtain a third feature;
inputting the third features into the second pooling layer for maximum pooling operation to obtain fourth features;
and inputting the fourth feature into the third convolution layer for convolution operation to obtain the human face feature.
7. The intelligent automobile door opening method according to any one of claims 1 to 4, wherein when the human face features are matched with preset facial features of an automobile owner, the method for controlling the door of the intelligent automobile to open comprises the following steps:
calculating the cosine similarity of the human face features and the facial features;
and controlling the opening of the door of the intelligent automobile based on the cosine similarity.
8. The utility model provides an intelligence car door opener which characterized in that includes:
the image acquisition module is used for acquiring a shot image acquired by the vehicle-mounted TOF camera;
the region image extraction module is used for extracting a region image comprising a human face from the shot image;
the face feature extraction module is used for extracting face features from the region image;
and the vehicle door control module is used for controlling the vehicle door of the intelligent vehicle to be opened when the human face characteristics are matched with the preset facial characteristics of the vehicle owner.
9. An intelligent automobile, comprising:
one or more processors;
storage means for storing one or more programs;
when executed by the one or more processors, cause the one or more processors to implement the intelligent automobile door opening method of any one of claims 1-7.
10. A computer-readable storage medium, on which a computer program is stored, which, when being executed by a processor, carries out the intelligent vehicle door opening method according to any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011131926.6A CN112215187A (en) | 2020-10-21 | 2020-10-21 | Intelligent automobile door opening method and device, intelligent automobile and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011131926.6A CN112215187A (en) | 2020-10-21 | 2020-10-21 | Intelligent automobile door opening method and device, intelligent automobile and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN112215187A true CN112215187A (en) | 2021-01-12 |
Family
ID=74056237
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011131926.6A Pending CN112215187A (en) | 2020-10-21 | 2020-10-21 | Intelligent automobile door opening method and device, intelligent automobile and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112215187A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114619993A (en) * | 2022-03-16 | 2022-06-14 | 上海齐感电子信息科技有限公司 | Automobile control method based on face recognition, system, equipment and storage medium thereof |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102975690A (en) * | 2011-09-02 | 2013-03-20 | 上海博泰悦臻电子设备制造有限公司 | Automobile locking system and method thereof |
| CN107066942A (en) * | 2017-03-03 | 2017-08-18 | 上海斐讯数据通信技术有限公司 | A kind of living body faces recognition methods and system |
| CN107423690A (en) * | 2017-06-26 | 2017-12-01 | 广东工业大学 | A kind of face identification method and device |
| CN109902560A (en) * | 2019-01-15 | 2019-06-18 | 浙江师范大学 | A kind of fatigue driving method for early warning based on deep learning |
| CN110163111A (en) * | 2019-04-24 | 2019-08-23 | 平安科技（深圳）有限公司 | Method, apparatus of calling out the numbers, electronic equipment and storage medium based on recognition of face |
| JP2020138702A (en) * | 2019-03-01 | 2020-09-03 | エスティーシー コーポレーション カンパニー リミテッド | Vehicle pre-heating control system with use of face recognition information and vehicle pre-heating control method with use of the same |
-
2020
- 2020-10-21 CN CN202011131926.6A patent/CN112215187A/en active Pending
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102975690A (en) * | 2011-09-02 | 2013-03-20 | 上海博泰悦臻电子设备制造有限公司 | Automobile locking system and method thereof |
| CN107066942A (en) * | 2017-03-03 | 2017-08-18 | 上海斐讯数据通信技术有限公司 | A kind of living body faces recognition methods and system |
| CN107423690A (en) * | 2017-06-26 | 2017-12-01 | 广东工业大学 | A kind of face identification method and device |
| CN109902560A (en) * | 2019-01-15 | 2019-06-18 | 浙江师范大学 | A kind of fatigue driving method for early warning based on deep learning |
| JP2020138702A (en) * | 2019-03-01 | 2020-09-03 | エスティーシー コーポレーション カンパニー リミテッド | Vehicle pre-heating control system with use of face recognition information and vehicle pre-heating control method with use of the same |
| CN110163111A (en) * | 2019-04-24 | 2019-08-23 | 平安科技（深圳）有限公司 | Method, apparatus of calling out the numbers, electronic equipment and storage medium based on recognition of face |
Non-Patent Citations (1)
| Title |
|---|
| （意）吉安卡洛•扎克尼，（德）礼萨•卡里姆著，连晓峰等译: "TensorFlow深度学习 原书第2版", 31 January 2020, 北京邮电大学出版社, pages: 102 - 104 * |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114619993A (en) * | 2022-03-16 | 2022-06-14 | 上海齐感电子信息科技有限公司 | Automobile control method based on face recognition, system, equipment and storage medium thereof |
| CN114619993B (en) * | 2022-03-16 | 2023-06-16 | 上海齐感电子信息科技有限公司 | Automobile control method based on face recognition, system, equipment and storage medium thereof |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111814902A (en) | Target detection model training method, target identification method, device and medium | |
| US20090304271A1 (en) | Object region extracting device | |
| CN111989689A (en) | Method for identifying objects within an image and mobile device for performing the method | |
| CN106909884B (en) | Hand region detection method and device based on layered structure and deformable part model | |
| CN110807384A (en) | Small target detection method and system under low visibility | |
| CN112614136A (en) | Infrared small target real-time instance segmentation method and device | |
| JP4658532B2 (en) | Method for detecting face and device for detecting face in image | |
| Hua et al. | Pedestrian-and vehicle-detection algorithm based on improved aggregated channel features | |
| CN107330354A (en) | Natural gesture recognition method | |
| CN111723822A (en) | RGBD image significance detection method and system based on multi-level fusion | |
| CN116721398A (en) | Yolov5 target detection method based on cross-stage route attention module and residual information fusion module | |
| CN115661720A (en) | Target tracking and identifying method and system for shielded vehicle | |
| CN110232381B (en) | License plate segmentation method, license plate segmentation device, computer equipment and computer readable storage medium | |
| Barodi et al. | An enhanced artificial intelligence-based approach applied to vehicular traffic signs detection and road safety enhancement | |
| CN116912541A (en) | Model training and image detection method and device, electronic equipment and storage medium | |
| CN112215187A (en) | Intelligent automobile door opening method and device, intelligent automobile and storage medium | |
| CN109165592B (en) | Real-time rotatable face detection method based on PICO algorithm | |
| WO2021189321A1 (en) | Image processing method and device | |
| KR100893086B1 (en) | Method for detecting face robust to illumination change | |
| CN113744141B (en) | Image enhancement method and device and automatic driving control method and device | |
| Jeong et al. | Homogeneity patch search method for voting-based efficient vehicle color classification using front-of-vehicle image | |
| Jourdheuil et al. | Heterogeneous adaboost with real-time constraints-application to the detection of pedestrians by stereovision | |
| CN111325209B (en) | License plate recognition method and system | |
| CN114296545A (en) | Unmanned aerial vehicle gesture control method based on vision | |
| Tsai et al. | Real-time automatic multilevel color video thresholding using a novel class-variance criterion |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |