CN110647813A - Human face real-time detection and identification method based on unmanned aerial vehicle aerial photography - Google Patents

Human face real-time detection and identification method based on unmanned aerial vehicle aerial photography Download PDF

Info

Publication number
CN110647813A
CN110647813A CN201910771567.1A CN201910771567A CN110647813A CN 110647813 A CN110647813 A CN 110647813A CN 201910771567 A CN201910771567 A CN 201910771567A CN 110647813 A CN110647813 A CN 110647813A
Authority
CN
China
Prior art keywords
data
value
method based
real
identification method
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN201910771567.1A
Other languages
Chinese (zh)
Inventor
刘洋
冉欢欢
李毅捷
谢雨峰
时翔
唐柯
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
CHENGDU SHINE TECHNOLOGY Co Ltd
Original Assignee
CHENGDU SHINE TECHNOLOGY Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by CHENGDU SHINE TECHNOLOGY Co Ltd filed Critical CHENGDU SHINE TECHNOLOGY Co Ltd
Priority to CN201910771567.1A priority Critical patent/CN110647813A/en
Publication of CN110647813A publication Critical patent/CN110647813A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/16Human faces, e.g. facial parts, sketches or expressions
    • G06V40/161Detection; Localisation; Normalisation
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00Arrangements for image or video recognition or understanding
    • G06V10/70Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/74Image or video pattern matching; Proximity measures in feature spaces
    • G06V10/75Organisation of the matching processes, e.g. simultaneous or sequential comparisons of image or video features; Coarse-fine approaches, e.g. multi-scale approaches; using context analysis; Selection of dictionaries
    • G06V10/751Comparing pixel values or logical combinations thereof, or feature values having positional relevance, e.g. template matching
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/16Human faces, e.g. facial parts, sketches or expressions
    • G06V40/168Feature extraction; Face representation
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/16Human faces, e.g. facial parts, sketches or expressions
    • G06V40/172Classification, e.g. identification

Abstract

The invention discloses a human face real-time detection and identification method based on unmanned aerial vehicle aerial photography, which comprises the following steps: loading a data processing model, acquiring data by a camera, and writing image data; preprocessing data; compressing the converted image data and storing the image data in a shared memory; the processor reads the image data; carrying out convolution processing on the data; the pooling layer performs dimensionality reduction on the data; carrying out nonlinear mapping on the data; carrying out classification processing on the weighted sum of the data; performing non-maximum suppression on the data; face matching is carried out on adjacent video frames, and a tracking template is detected and updated through a fixed frame number; calculating the correlation degree of the obtained target and the detected target to enhance the matching accuracy; superposing the face frames and counting the faces; and displaying the plug flow. The invention can realize real-time face detection and identification and greatly reduce the false detection rate.

Description

Human face real-time detection and identification method based on unmanned aerial vehicle aerial photography
Technical Field
The invention relates to the technical field of unmanned aerial vehicle monitoring, in particular to a human face real-time detection and identification method based on unmanned aerial vehicle aerial photography.
Background
The traditional face recognition is mainly applied to recognition scenes based on fixed cameras, the face recognition effect is poor, the flexibility is poor, and once personnel move or the cameras move, face recognition data are inaccurate; in the relative movement of the camera and the personnel, the real-time performance of the face recognition is poor.
Disclosure of Invention
The invention aims to solve the problems and provides a human face real-time detection and identification method based on unmanned aerial vehicle aerial photography, which comprises the following steps:
s1: the detection module loads a data processing model and waits for receiving image data;
s2, the camera collects data and writes the data into the memory;
s3, preprocessing data: converting the format of image data, normalizing the image data, and converting the data into RGB data;
s4, compressing the converted image data and storing the image data in a shared memory;
s5, the detection module reads the image data and processes the data;
s6, carrying out convolution processing on the convolution layer neural network through convolution check data;
s7, the pooling layer performs dimensionality reduction on the data, and selects the maximum pixel value in the convolution kernel as a dimensionality reduced value;
s8, the excitation layer carries out nonlinear mapping on the output data of the convolution processing: let alpha be the slope coefficient, x be the abscissa value, and g (x) be the result of the function, then
g(x)=max(αx,x);
S9, classifying the extracted feature data by the full connection layer through weighted summation: y is Wx, wherein W is a weight vector and x is a feature vector;
s10, performing non-maximum value inhibition on the detection result, and removing repeated detection of the same target;
s11, reading each frame data in the video, and setting a reference value of the frame spacing frame number according to the scene; judging whether the current frame number is a multiple of a reference value; if yes, carrying out face detection on the next frame, and updating a tracking template; if not, matching with the face detected in the front so as to track the face;
s12, judging the correlation degree of the targets in the front frame and the rear frame by a normalized cross-correlation matching algorithm, if the correlation degree is within a certain threshold value range, determining the targets as the same face targets, otherwise, determining the targets as different face targets, and achieving the purpose of face tracking;
s13, superposing the face frames and counting the faces; and displayed by plug flow.
The invention has the beneficial effects that: based on the view angle of the unmanned aerial vehicle, the invention can realize mobile shooting, carry out real-time deep learning network, carry out network optimization and training aiming at the data format of the camera, further reduce the network operation amount, realize real-time face detection and recognition, and simultaneously greatly reduce the false detection rate.
Drawings
FIG. 1 is a flow chart of the present invention.
Detailed Description
The invention will be further described with reference to the accompanying drawings in which:
as shown in fig. 1, the invention relates to a human face real-time detection and identification method based on unmanned aerial vehicle aerial photography, which comprises the following steps:
s1: the detection module loads a data processing model and waits for receiving image data;
s2, the camera collects data and writes the data into the memory;
s3, preprocessing data: converting the format of image data, normalizing the image data, and converting the data into RGB data;
s4, compressing the converted image data and storing the image data in a shared memory;
s5, the detection module reads the image data and processes the data;
s6, carrying out convolution processing on the convolution layer neural network through convolution check data;
s7, the pooling layer performs dimensionality reduction on the data, and selects the maximum pixel value in the convolution kernel as a dimensionality reduced value;
s8, the excitation layer carries out nonlinear mapping on the output data of the convolution processing: let alpha be the slope coefficient, x be the abscissa value, and g (x) be the result of the function, then
g(x)=max(αx,x);
S9, classifying the extracted feature data by the full connection layer through weighted summation: y is Wx, wherein W is a weight vector and x is a feature vector;
s10, performing non-maximum value inhibition on the detection result, and removing repeated detection of the same target;
s11, reading each frame data in the video, and setting a reference value of the frame spacing frame number according to the scene; judging whether the current frame number is a multiple of a reference value; if yes, carrying out face detection on the next frame, and updating a tracking template; if not, matching with the face detected in the front so as to track the face;
s12, judging the correlation degree of the targets in the front frame and the rear frame by a normalized cross-correlation matching algorithm, if the correlation degree is within a certain threshold value range, determining the targets as the same face targets, otherwise, determining the targets as different face targets, and achieving the purpose of face tracking;
s13, superposing the face frames and counting the faces; and displayed by plug flow.
Further, the normalizing the data in S3 specifically includes:
and setting x as the value of the current pixel value normalization, x as the current pixel value, and max as the maximum value of all pixels in the current frame, then: x is the number of*=log10(x)/log10(max)。
Further, the step S2 of converting the data format specifically includes: conversion of 1920 × 1080 × 2 uyvy data into 1920 × 1080 × 3 RGB data:
Figure BDA0002173714130000031
wherein: y is brightness, u and v denote chromaticity, R denotes a red channel, G denotes a green channel, and B denotes a blue channel.
Further, the convolution processing in S6 specifically includes:
Figure BDA0002173714130000041
where R (i, j) is the convolution value at the image (i, j), k is the convolution kernel of n × n, n is 2a, and f (i, j) is the pixel value at the image (i, j).
Further, the specific process of S7 is as follows:
y(u,v)=max{f(u+i,v+j),i∈(0,n),j∈(0,n)};
wherein: n denotes the size of the kernel and y (u, v) denotes the maximum pixel value in the neighborhood n x n in the image after max-firing.
Further, the specific process of S10 is as follows:
Figure BDA0002173714130000042
wherein, the interration is the intersection of two target frames, the Union is the Union of the two target frames, and if IoU exceeds a set threshold, the two target frames are the same target frame;
further, the performing face matching on adjacent frames in S11 specifically includes:
Figure BDA0002173714130000043
wherein: t is the matching template, S is the image matrix, and D is the matching result at (i, j).
Further, the normalization product correlation algorithm in S12 specifically processes:
Figure BDA0002173714130000044
wherein: e (S)i,j) The average gray scale value of the subgraph at (i, j), e (T) the average gray scale value of the template, M × N the matching template size, and T the matching template.
The specific process is as follows:
the detection module loads a data processing model and waits for receiving image data; the camera collects data and writes the data into the memory.
Pretreatment: the data that airborne camera gathered are fixed format, fixed resolution, and the data of gathering are the uyvy format, 1920x 1080's two-channel data, need carry out the preliminary treatment to data, including data format conversion and normalization, specifically include: and setting x as the value of the current pixel value normalization, x as the current pixel value, and max as the maximum value of all pixels in the current frame, then: x is the number of*=log10(x)/log10(max)。
After normalization, 1920 × 1080 × 2 uyvy data was converted into 1920 × 1080 × 3 rgb data:
Figure BDA0002173714130000051
wherein: y is brightness, u and v denote chromaticity, R denotes a red channel, G denotes a green channel, and B denotes a blue channel.
To reduce the amount of computation and maintain the accuracy of target detection, a 1920x1080 resolution is compressed to 640 x 640: let Q11、Q12、Q21And Q22The pixel coordinate positions of adjacent pixel points in the image are respectively (x)1,y1)、(x1,y2)、(x2,y1) And (x)2,y2) P is the point to be estimated, R1Is Q11And Q21Approximate evaluation point of, R2Is Q12And Q22P is R1And R2Then:
through the normalization processing and bilinear interpolation calculation, the original data is processed into a format suitable for a face detection algorithm and stored in a shared memory, as shown in a flow chart shown in fig. 1.
And the processor reads the image data, loads the data processing model and processes the data.
Convolution processing: carrying out convolution processing on data through convolution check in the neural network: and setting x as the value of the current pixel value normalization, x as the current pixel value, and max as the maximum value of all pixels in the current frame, then: x is the number of*=log10(x)/log10(max)。
Performing pooling treatment: the pooling layer performs dimensionality reduction on the data, a max-posing mode is selected for dimensionality reduction, the calculated amount can be reduced, and the maximum value of the data in the convolution kernel is selected as the value after dimensionality reduction: setting: n represents the size of the kernel, y (u, v) represents the maximum pixel value in the neighborhood n x n in the image (u, v) as a result after max-posing, then: y (u, v) ═ max { f (u + i, v + j), i ∈ (0, n), j ∈ (0, n) }.
Excitation: non-linear mapping of the convolution processed output: assuming α is the slope coefficient, x is the abscissa parameter, and g (x) is the result of the function, then: g (x) max (α x, x).
The full connection layer classifies the extracted feature data through weighted summation: and y is Wx, wherein W is a weight vector and x is a feature vector.
By the above-mentioned feature extraction of the data, and the target detection and classification, the same target may be detected many times in the result, and in order to eliminate the phenomenon, the result is subjected to non-maximum suppression, and the process is as follows:
the intersections is the intersection of two target frames, the Union is the Union of the two target frames, the larger the IoU is, the higher the overlapping degree is, if the overlapping degree exceeds a certain threshold value, the same target frame is considered, and the selection is preferred.
Reading frame data, judging whether the frame number is a multiple of a reference value according to the reference value of the scene set frame number, if so, performing face matching on adjacent frames, updating a tracking template, otherwise, performing face matching on the adjacent frames, and performing face tracking;
further judging the absolute error and the target obtained by the algorithm and the detected target by normalizing the algorithm correlation coefficientAnd the correlation degree greatly reduces the false detection rate. The specific process is as follows: setting: e (S)i,j) (ii) the average gray value of the subgraph at (i, j), e (T) the average gray value of the template, M × N the size of the matching template, and T the matching template, then:
Figure BDA0002173714130000071
based on the view angle of the unmanned aerial vehicle, the invention can realize mobile shooting, carry out real-time deep learning network, carry out network optimization and training aiming at the data format of the camera, further reduce the network operation amount, realize real-time face detection and recognition, and simultaneously greatly reduce the false detection rate.
The technical solution of the present invention is not limited to the limitations of the above specific embodiments, and all technical modifications made according to the technical solution of the present invention fall within the protection scope of the present invention.

Claims (8)

1. A real-time face detection and identification method based on unmanned aerial vehicle aerial photography is characterized by comprising the following steps:
s1: the detection module loads a data processing model and waits for receiving image data;
s2, the camera collects data and writes the data into the memory;
s3, preprocessing data: converting the format of image data, normalizing the image data, and converting the data into RGB data;
s4, compressing the converted image data and storing the image data in a shared memory;
s5, the detection module reads the image data and processes the data;
s6, carrying out convolution processing on the convolution layer neural network through convolution check data;
s7, the pooling layer performs dimensionality reduction on the data, and selects the maximum pixel value in the convolution kernel as a dimensionality reduced value;
s8, the excitation layer carries out nonlinear mapping on the output data of the convolution processing: let alpha be the slope coefficient, x be the abscissa value, and g (x) be the result of the function, then
g(x)=max(αx,x);
S9, classifying the extracted feature data by the full connection layer through weighted summation: y is Wx, wherein W is a weight vector and x is a feature vector;
s10, performing non-maximum value inhibition on the detection result, and removing repeated detection of the same target;
s11, reading each frame data in the video, and setting a reference value of the frame spacing frame number according to the scene; judging whether the current frame number is a multiple of a reference value; if yes, carrying out face detection on the next frame, and updating a tracking template; if not, matching with the face detected in the front so as to track the face;
s12, judging the correlation degree of the targets in the front frame and the rear frame by a normalized cross-correlation matching algorithm, if the correlation degree is within a certain threshold value range, determining the targets as the same face targets, otherwise, determining the targets as different face targets, and achieving the purpose of face tracking;
s13, superposing the face frames and counting the faces; and displayed by plug flow.
2. The real-time human face detection and identification method based on unmanned aerial vehicle aerial photography according to claim 1, wherein the normalization processing of data in S3 specifically comprises:
and setting x as the value of the current pixel value normalization, x as the current pixel value, and max as the maximum value of all pixels in the current frame, then: x is the number of*=log10(x)/log10(max)。
3. The real-time human face detection and identification method based on unmanned aerial vehicle aerial photography according to claim 1, wherein the step S3 of converting the data format specifically comprises: converting the 1920 × 1080 × 2 UYVY-formatted data into 1920 × 1080 × 3 RGB-formatted data:
Figure FDA0002173714120000021
wherein: y is brightness, U and V denote chromaticities, R denotes a red channel, G denotes a green channel, and B denotes a blue channel.
4. The real-time human face detection and identification method based on unmanned aerial vehicle aerial photography according to claim 1, wherein the convolution processing in S6 specifically comprises:
Figure FDA0002173714120000022
where R (i, j) is the convolution value at the image (i, j), k is the convolution kernel of n × n, n is 2a, and f (i, j) is the pixel value at the image (i, j).
5. The real-time human face detection and identification method based on unmanned aerial vehicle aerial photography according to claim 1, wherein the S7 specific process is as follows:
y(u,v)=max{f(u+i,v+j),i∈(0,n),j∈(0,n)};
wherein: n denotes the size of the kernel and y (u, v) denotes the maximum pixel value in the neighborhood n x n in the image after max-firing.
6. The real-time human face detection and identification method based on unmanned aerial vehicle aerial photography according to claim 1, wherein the S10 specific process is as follows:
Figure FDA0002173714120000031
wherein, the interration is the intersection of two target frames, the Union is the Union of the two target frames, and if IoU exceeds the set threshold, the same target frame is obtained.
7. The real-time human face detection and identification method based on unmanned aerial vehicle aerial photography according to claim 1, wherein the performing of human face matching of adjacent frames in S11 specifically comprises:
Figure FDA0002173714120000032
wherein: t is the matching template, S is the image matrix, and D is the matching result at (i, j).
8. The real-time human face detection and identification method based on unmanned aerial vehicle aerial photography according to claim 1, wherein the normalization product correlation algorithm in the step S12 is used for processing the specific process:
Figure FDA0002173714120000033
wherein: e (S)i,j) The average gray scale value of the subgraph at (i, j), e (T) the average gray scale value of the template, M × N the matching template size, and T the matching template.
CN201910771567.1A 2019-08-21 2019-08-21 Human face real-time detection and identification method based on unmanned aerial vehicle aerial photography Pending CN110647813A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201910771567.1A CN110647813A (en) 2019-08-21 2019-08-21 Human face real-time detection and identification method based on unmanned aerial vehicle aerial photography

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201910771567.1A CN110647813A (en) 2019-08-21 2019-08-21 Human face real-time detection and identification method based on unmanned aerial vehicle aerial photography

Publications (1)

Publication Number Publication Date
CN110647813A true CN110647813A (en) 2020-01-03

Family

ID=69009705

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201910771567.1A Pending CN110647813A (en) 2019-08-21 2019-08-21 Human face real-time detection and identification method based on unmanned aerial vehicle aerial photography

Country Status (1)

Country Link
CN (1) CN110647813A (en)

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112229845A (en) * 2020-10-12 2021-01-15 国网河南省电力公司濮阳供电公司 Unmanned aerial vehicle high-precision winding tower intelligent inspection method based on visual navigation technology
CN112418117A (en) * 2020-11-27 2021-02-26 北京工商大学 Small target detection method based on unmanned aerial vehicle image
CN114299596A (en) * 2022-03-09 2022-04-08 深圳联和智慧科技有限公司 Smart city face recognition matching method and system and cloud platform

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107390722A (en) * 2017-07-28 2017-11-24 上海瞬动科技有限公司合肥分公司 A kind of unmanned plane during flying angle intelligent control method
CN107784288A (en) * 2017-10-30 2018-03-09 华南理工大学 A kind of iteration positioning formula method for detecting human face based on deep neural network
CN109063593A (en) * 2018-07-13 2018-12-21 北京智芯原动科技有限公司 A kind of face tracking method and device
CN110008919A (en) * 2019-04-09 2019-07-12 南京工业大学 The quadrotor drone face identification system of view-based access control model

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107390722A (en) * 2017-07-28 2017-11-24 上海瞬动科技有限公司合肥分公司 A kind of unmanned plane during flying angle intelligent control method
CN107784288A (en) * 2017-10-30 2018-03-09 华南理工大学 A kind of iteration positioning formula method for detecting human face based on deep neural network
CN109063593A (en) * 2018-07-13 2018-12-21 北京智芯原动科技有限公司 A kind of face tracking method and device
CN110008919A (en) * 2019-04-09 2019-07-12 南京工业大学 The quadrotor drone face identification system of view-based access control model

Non-Patent Citations (3)

* Cited by examiner, † Cited by third party
Title
余丹: "基于卷积神经网络的图像识别方法研究", 《中国优秀硕士学位论文全文数据库信息科技辑》 *
李楠: "基于多尺度梯度相角的人脸图像对称性检测与定位", 《小型微型计算机系统》 *
沈沛意等: "《DAVINCI技术剖析及实战应用开发指南》", 30 April 2012 *

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112229845A (en) * 2020-10-12 2021-01-15 国网河南省电力公司濮阳供电公司 Unmanned aerial vehicle high-precision winding tower intelligent inspection method based on visual navigation technology
CN112418117A (en) * 2020-11-27 2021-02-26 北京工商大学 Small target detection method based on unmanned aerial vehicle image
CN112418117B (en) * 2020-11-27 2023-05-12 北京工商大学 Small target detection method based on unmanned aerial vehicle image
CN114299596A (en) * 2022-03-09 2022-04-08 深圳联和智慧科技有限公司 Smart city face recognition matching method and system and cloud platform

Similar Documents

Publication Publication Date Title
US11882357B2 (en) Image display method and device
CN107463949B (en) Video action classification processing method and device
CN109903331B (en) Convolutional neural network target detection method based on RGB-D camera
CN113065558A (en) Lightweight small target detection method combined with attention mechanism
CN110532970B (en) Age and gender attribute analysis method, system, equipment and medium for 2D images of human faces
CN107194317B (en) Violent behavior detection method based on grid clustering analysis
CN109740721B (en) Wheat ear counting method and device
CN110647813A (en) Human face real-time detection and identification method based on unmanned aerial vehicle aerial photography
WO2018082308A1 (en) Image processing method and terminal
CN111325051A (en) Face recognition method and device based on face image ROI selection
CN111738344A (en) Rapid target detection method based on multi-scale fusion
WO2022073282A1 (en) Motion recognition method based on feature interactive learning, and terminal device
CN112633185A (en) Image processing method and device
CN110991256A (en) System and method for carrying out age estimation and/or gender identification based on face features
CN113781421A (en) Underwater-based target identification method, device and system
CN111047618B (en) Multi-scale-based non-reference screen content image quality evaluation method
CN114463677A (en) Safety helmet wearing detection method based on global attention
WO2019228450A1 (en) Image processing method, device, and equipment, and readable medium
CN114581318A (en) Low-illumination image enhancement method and system
CN112101260B (en) Method, device, equipment and storage medium for identifying safety belt of operator
CN113435367A (en) Social distance evaluation method and device and storage medium
CN116778473A (en) Improved YOLOV 5-based mushroom offline real-time identification method and system
CN110766655A (en) Hyperspectral image significance analysis method based on abundance
CN115311520A (en) Passion fruit maturity detection and positioning method based on visual identification
CN107194406A (en) A kind of panorama machine vision target identification method based on CS characteristic values

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
RJ01 Rejection of invention patent application after publication

Application publication date: 20200103

RJ01 Rejection of invention patent application after publication