Disclosure of Invention
In order to overcome the problems of high network overhead and serious and uneven hardware overhead distribution of the monitoring system in the prior art, the invention aims to provide a face information collection method based on face recognition of a camera.
The method provided by the invention comprises the following steps:
a face information collection method based on camera face recognition comprises the following steps: in the process of acquiring video data by a camera, simultaneously carrying out face information identification on each frame of video data to obtain face images of all people in the current frame and quality parameters of the corresponding face images, finally determining people of which the face information and the quality parameters of the face images in all people in the current frame reach preset standards, and snapshotting face photos of the people.
The method for identifying the face information of each frame of video data to obtain the face images of all people in the current frame and the quality parameters of the corresponding face images comprises the following steps: the camera picks up face information according to the brightness distribution in one frame of the video data, and then determines the quality parameters of the face image according to the brightness and the definition of the face image in the face information and the matching degree of the face features under the front face.
The camera further comprises, after picking up face information according to the luminance distribution in one frame of the video data: the camera judges whether the current face image is a new person according to the comparison condition of the current face information and the face information picked up from a plurality of adjacent frames, if the current face image is determined to be the new person, a face ID is distributed to the current face image, otherwise, a corresponding relation is established between the current face image and the existing face ID.
The method for determining the figure of which the face information and the quality parameters of the face images in all the figures in the current frame reach the preset standard and snapshotting the figure with the face photo specifically comprises the following steps: and when the new person is determined and the quality parameter of the face image is greater than or equal to a preset threshold value, the camera takes a face picture and stores the face picture.
After the face information recognition is carried out on each frame of video data to obtain the face images of all people in the current frame and the quality parameters of the corresponding face images, the method further comprises the following steps: and when the new person is determined but the quality parameter of the face image is smaller than the preset threshold value, the camera adjusts the camera parameter according to the face information, simultaneously discards the face image and continuously judges the quality parameter of the face image generated by the subsequent video data until the quality parameter of the face image is larger than or equal to the preset threshold value.
Determining persons of which the face information and the quality parameters of the face images in all the persons in the current frame reach preset standards, and snapshotting a face photo of the persons, wherein the steps are as follows: and when the original face is determined and the quality parameter of the face image is greater than or equal to the preset threshold value, the camera judges that the quality parameter of the face image or the face front degree is greater than that of the original face photo, and finally the face photo is captured and replaced by the original face photo.
After the face information recognition is carried out on each frame of video data to obtain the face images of all people in the current frame and the quality parameters of the corresponding face images, the method further comprises the following steps: when the original face is determined and the quality parameter of the face image is smaller than the preset threshold value, the camera adjusts the camera parameter according to the face information, meanwhile, the face image is discarded, and the quality parameter of the face image generated by the subsequent video data is continuously judged until the quality parameter of the face image is larger than or equal to the preset threshold value.
When the quality parameter of the face image is greater than or equal to the preset threshold value, the method further comprises the following steps: the camera records exposure parameters when the quality parameters of the face image are larger than or equal to a threshold value, then judges that the camera does not detect face information within preset time, and adjusts the camera according to the exposure parameters recorded when the parameters of the face image at the last time are larger than or equal to the preset threshold value.
Determining the persons of which the face information and the quality parameters of the face images in all the persons in the current frame reach preset standards, and snapping face photos for the persons, wherein the method also comprises the following steps: when the camera detects that the storage capacity is less than the early warning value, part of the face photos are deleted, specifically: the camera determines the priority of the face ID according to the access state, the uploading state and the tracking state of the face photo corresponding to the face ID, adjusts the face ID in real time, deletes the face photo corresponding to the face ID according to the priority sequence and cancels the face ID, and finally detects that the storage capacity is higher than a preset threshold value and stops deleting work.
And snapping the face photo according to preset indexes reached by the face information and the quality parameters of the face image and uploading the face photo, specifically, selecting a snapping mode of the face photo by the camera according to the requirements of a remote server.
The method for collecting face information based on face recognition of camera according to claim 1, characterized in that: the method comprises the steps of snapping and uploading a face photo according to preset indexes reached by face information and quality parameters of a face image, and specifically, selecting an uploading strategy of the face photo by a camera according to the requirement of a remote server.
Compared with the prior art, the invention undertakes a large amount of simple face recognition work by adding the image operation chip, obtains and uploads the face photos according to the face recognition result by using the general processor, replaces the bandwidth waste caused by the fact that the original video code stream is directly uploaded for the server to carry out face recognition and the hardware resource waste of the server, and is suitable for simultaneously monitoring and tracking more faces and improving the monitoring effect.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more apparent, the present invention is described in further detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.
The embodiment of the invention provides a face recognition method based on a camera, which comprises the following steps: the method comprises the steps that a camera collects video data, face information identification is carried out on each frame of video data while the data are collected so as to obtain face information of a face image and quality parameters of the face image, the face image is determined to reach a preset index, and a face with the face reaching the preset index is photographed.
In order to realize the method concretely, the camera comprises a main controller used for controlling the operations of the camera, image coding and decoding, communication and the like, a corresponding main memory, an image operation chip used for carrying out face image recognition by using a face recognition algorithm, a corresponding memory and other external components, and the camera can be additionally provided with an external memory used for storing the pictures obtained by snapshot and reducing the storage pressure of the main memory. The method comprises the following specific steps on the basis that:
step 100: the main controller collects video data of the sensor and caches the video data to the main memory.
Step 200: and the main controller transmits the segmented video data to the image operation chip.
Step 300: the image operation chip analyzes the human face in the segmented video data, identifies the human face and returns human face information to the main controller.
Step 301: the image operation chip picks up the face information according to the brightness distribution of the face image in a certain frame in the received segmented video data.
Step 302: the image operation chip compares the face information detected at present with the face information of the face images detected in a plurality of frames adjacent to the previous frames, judges the similarity of the two, judges whether the face detected at present is a new person according to the similarity and a set threshold value, and corresponds the face information of the face image which has appeared with the corresponding face ID according to the judgment result or sets a new face ID for the new person.
Step 303; and the image operation chip gives the quality parameters of the current face image according to the brightness and the definition of the current face image and the matching degree of the face image under the front face.
Step 304: and transmitting the face image and the quality parameter information to the main controller.
Step 400: and the main controller adjusts the environment peripheral equipment according to the face information.
Step 401: and the main controller judges whether the exposure parameters and the required adjustment step length need to be adjusted according to the quality parameters of the face image and the brightness of the current face.
Step 402: when the quality parameter of the face image reaches a set threshold value, the main controller records the exposure parameter within a period of time.
Step 403: when the human face is not detected, the main controller sets the camera using the recorded exposure parameters.
Step 500: the main controller takes a snapshot of the face and stores it in the main memory or the external memory.
Step 501: and according to the face ID, when the quality parameter of the received new face image is higher than a preset threshold value, the main controller controls the camera to capture and store the face picture.
Specifically, the main controller controls the camera to capture a face image, a body image or an original image or a free combination of the face image, the body image and the original image according to the setting.
Step 502: according to the face ID, when the face image information of the stored face picture is received, the main controller judges that the current face picture is not a front face, the front face area of the current face picture is larger than that of the stored face picture, and the quality parameter of the current face picture is higher than a threshold value, or the quality parameter of the current face picture is higher than that of the stored face picture, the face picture is feared, the original face picture is stored and replaced, and the quality parameter of the face picture is recorded.
Step 503: according to the face ID, when the face image information of the stored face photo is received, the main controller judges that when the stored face photo is a front face, the quality parameter of the current face image is larger than that of the stored face, the face photo is captured in a snapping mode, the original face photo is stored and replaced, and the quality parameter of the face photo is recorded.
Step 600: and uploading and cleaning the face photos.
Step 601: and when the quality of the human face photos exceeds a set value, the main controller uploads the human face photos.
Specifically, the main controller preferentially uploads a picture with picture quality parameters meeting the uploading standard and a face picture with a shorter time interval with the current time interval, and all face pictures with higher quality parameters.
Specifically, the main controller controls the camera to upload in real time, upload after the person leaves and upload at intervals according to the setting, wherein the real-time uploading is to upload the face photos immediately after the face photos change, the uploading after the person leaves is to upload the optimal photos after the person leaves the detection area, and the uploading at intervals is to upload the optimal photos periodically when the face images corresponding to the face ID appear in the detection area.
Step 602: when the use capacity of the main memory or the external memory reaches a set value, the main controller sets a priority for each face ID according to the current access condition, the current uploading condition, the current tracking condition and the last tracking time of the face photo corresponding to the face ID.
Specifically, the face photo currently being accessed has the highest priority, the face photo not currently being uploaded obtains a higher priority, the face photo currently being tracked obtains a higher priority, and the face photo having the last tracking time and the current time interval longer obtains a lower priority.
Step 603: and preferentially deleting and covering the face photos with lower priority according to the priority of the face ID, and canceling the face ID corresponding to the covered face photos.
The first embodiment is as follows:
the camera records the video of the detection area and stores the video in the memory, and the ARM processor transmits the segmented video data to the FPGA operation chip through the BT1120 standard video data interface.
The FPGA operation chip caches the segmented video data in a memory of the FPGA operation chip, analyzes the segmented data frame by frame, picks up the brightness distribution of a face area in an image, captures a face image and records face information. The FPGA operation chip compares the face information recorded at present with the face information recorded in a plurality of recent frames, judges whether the face is consistent with the face recorded in the past, if the face is judged to be a new face, a new face ID is distributed, and if the face is judged to be an original face, the face information recorded at present is associated with the face ID corresponding to the original face.
The FPGA operation chip gives the quality parameter of the current face image according to the brightness and the definition of the current face information and the matching degree of the face features under the front face and transmits the quality parameter and the face information to the ARM processor.
The ARM processor judges whether exposure parameters and other camera parameters need to be adjusted or not according to the face brightness and quality parameters of the current face information, confirms the adjustment step length, controls the camera to complete the modification of shooting parameters, and simultaneously monitors the face information in the subsequent face image transmitted by the FPGA operation chip until the brightness is enough and the quality parameters reach the standard.
And on the premise that the quality parameter of the face image reaches a preset threshold value, the ARM processor captures the face picture and stores the face picture in the memory.
Specifically, the ARM processor extracts the face image in the cache to be used as a face photo, meanwhile, description information of the file is added, and shooting time and face ID are recorded.
And the ARM processor uploads the face photos stored in the memory to a remote server.
The scheme has the advantages that: the face ID is used for judging the acquisition condition of the face image, so that the acquisition condition of the face information can be accurately controlled and recorded conveniently, a large number of pictures do not need to be acquired and uploaded, and the hardware overhead is greatly saved.
The shooting condition of the photo is obtained according to the face information and the quality parameters of the face photo, and the parameters of the camera are adjusted in real time, so that the high-quality photo can be shot.
Example two:
on the basis of the first embodiment, when the FPGA computing chip cannot detect any face, the ARM processor backtracks the face picture meeting the standard recently, and the adjusted exposure parameters are used for setting the camera.
The scheme is convenient for setting the exposure state of the camera in the most reasonable state when no human face exists in the detection area, and is convenient for adjusting to a proper exposure position more quickly after the human face is detected again.
Example three:
on the basis of the first embodiment, the ARM controller receives the face image information and then judges a face ID, when the face ID of the face has been captured and stored into a picture, the stored quality parameters and frontal area of the face picture and the quality parameters and face area contained in the current face image information are corresponded, and if the quality parameters are higher or the frontal area is larger, the face picture is captured again and the original face picture is replaced.
And if the stored face picture is a front face, comparing the quality parameters of the face picture with the quality parameters of the current face image. If the quality parameters of the current face image are higher than those of the stored face photos, the new face photo is captured and stored, and the old face photo is replaced.
On average, the quality parameters of the face image are correspondingly higher for the photos with larger frontal face area, so the scheme is suitable for screening the images and photos which can better identify people. The scheme is convenient for rapid comparison and obtaining of the face photos with higher quality, and is convenient for storing of the face photos with better quality.
Example four:
based on the third embodiment, the ARM processor further receives an instruction from the remote server, and adjusts the fear rule according to the requirement of the remote server.
Specifically, the following picture patterns are included but not limited to:
original pictures of the current scene, human body pictures of the tracked human face or human face pictures of the tracked human face or any combination of the three pictures;
also included, but not limited to, are the following snap-shot strategies:
when the quality parameters of the face image reach a threshold value, immediately capturing and uploading the face image;
when the person of the face leaves the detection area, uploading the optimal face picture corresponding to the face;
and when the person to which the face belongs appears in the detection area, the optimal face picture is periodically transmitted.
The scheme enables the camera to be more suitable for daily monitoring tasks, and more reasonable data are reserved under the condition that no video data are filed.
Example five:
on the basis of the first embodiment, a TF card or a ROM memory is additionally arranged to serve as an external memory to be connected with an ARM processor, and the ARM processor stores the face photo in the TF card or the ROM memory.
The scheme optimizes the storage environment of the face photos, and prevents the situation that the photos are not uploaded and discarded or newly captured photos cannot be stored due to untimely uploading.
Example six:
on the basis of the first or fifth embodiment, when the storage space of the internal memory or the external memory monitored by the ARM processor is smaller than a preset threshold, the stored face photos are subjected to priority sequencing.
Specifically, whether a face photo corresponding to the face ID is being accessed is judged according to the face ID, a high priority is set for the face photo being accessed, the uploading condition of the face photo corresponding to the face ID is judged, a low priority is set for the face photo already uploaded, the face image tracking condition corresponding to the face ID is judged, a high priority is set for the face ID corresponding to the face image being tracked, and high-to-low priorities are set in sequence from near to far according to the last tracking time.
And the ARM processor deletes the face photo corresponding to the face ID with lower priority in the memory or the external memory according to the priority information which changes in real time, and cancels the face ID. And when the storage space of the internal memory or the external memory is higher than a preset threshold value, stopping deleting.
The scheme ensures that the high-value face photos are more effectively stored and does not hinder the normal operation of monitoring work under the condition of extremely severe network conditions.
The above description is only for the preferred embodiment of the present invention, but the scope of the present invention is not limited thereto, and any changes or substitutions that can be easily conceived by those skilled in the art within the technical scope of the present invention are included in the scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the protection scope of the claims.