CN111126113B - Face image processing method and device - Google Patents
Face image processing method and device Download PDFInfo
- Publication number
- CN111126113B CN111126113B CN201811294871.3A CN201811294871A CN111126113B CN 111126113 B CN111126113 B CN 111126113B CN 201811294871 A CN201811294871 A CN 201811294871A CN 111126113 B CN111126113 B CN 111126113B
- Authority
- CN
- China
- Prior art keywords
- face
- frame
- image
- newly added
- target picture
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000003672 processing method Methods 0.000 title description 17
- 238000000034 method Methods 0.000 claims abstract description 91
- 238000001514 detection method Methods 0.000 claims abstract description 63
- 230000000875 corresponding effect Effects 0.000 claims description 132
- 238000004891 communication Methods 0.000 claims description 16
- 230000002596 correlated effect Effects 0.000 claims description 10
- 230000005540 biological transmission Effects 0.000 claims description 3
- 238000010586 diagram Methods 0.000 description 9
- 238000004422 calculation algorithm Methods 0.000 description 8
- 238000004590 computer program Methods 0.000 description 3
- 238000004364 calculation method Methods 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 230000006399 behavior Effects 0.000 description 1
- 238000013527 convolutional neural network Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000000926 separation method Methods 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/16—Human faces, e.g. facial parts, sketches or expressions
- G06V40/161—Detection; Localisation; Normalisation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/40—Scenes; Scene-specific elements in video content
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Multimedia (AREA)
- Theoretical Computer Science (AREA)
- Health & Medical Sciences (AREA)
- General Health & Medical Sciences (AREA)
- Oral & Maxillofacial Surgery (AREA)
- Human Computer Interaction (AREA)
- Image Analysis (AREA)
Abstract
The embodiment of the invention discloses a method and a device for processing a face image, wherein for a target image acquired from a video ordered frame, if the target image is a detection frame, on one hand, first image information of a face image with a face number in a list in the target image is added into the list, and on the other hand, the face number is newly added for the face image which appears in the target image and does not have the face number in the list, and the image of the face image exists before the target image is traced back. The method for backtracking the face image associates the same person in each frame, prevents the condition of frame loss and detection omission, and reduces the operation amount. The method can process each picture in the video frame or the picture set without repeated starting detection, and improves the processing efficiency. In addition, the image information of each face image is stored in the list as original data information, and it is convenient to output the information of each face image in different forms according to the list.
Description
Technical Field
The embodiment of the invention relates to the technical field of image processing, in particular to a method and a device for processing a face image.
Background
The face recognition technology is developed to the present day, the accuracy rate is greatly improved to reach the commercial level, face snapshot is an important link, and the existing snapshot equipment processes video or pictures by extracting each frame of video frame in a video source, and then carries out face detection frame by frame to realize face recognition. Most of the currently used face detection parts of the method use a deep convolutional neural network method, and the problems of the method are that the operation amount is too large, the time consumption is long, the accuracy is high, the operation is complex, and the method is not suitable for running on equipment with poor performance. In order to reduce the operation complexity, the prior art also has a method for detecting every N frames, however, the method often has the phenomenon of missing detection, the algorithm cost is still large, and when people appear in non-detection frames, data of each frame cannot be acquired.
In the process of implementing the embodiment of the invention, the inventor finds that the existing method for processing the video needs repeated starting detection, and each frame is not associated with each other, so that the same person can not be confirmed between each frame, the detection is easy to miss, and the operation amount is large.
Disclosure of Invention
The invention aims to solve the technical problems that the existing method for processing the video needs repeated starting detection, the correlation processing among frames is not carried out, the fact that the frames are identical cannot be confirmed, the detection is easy to miss, and the operation amount is large.
Aiming at the technical problems, the embodiment of the invention provides a face image processing method, which comprises the following steps:
acquiring a target picture from the video ordered frames with frame numbers, and acquiring the number of face images detected from the target picture if the target picture is judged to be a detection frame according to the frame numbers of the target picture and the preset frame spacing;
if the number of the face images detected from the target picture is not zero and the detected face images comprise newly added face images with corresponding face numbers which are not in the list, creating the newly added face numbers corresponding to each newly added face image;
for each newly added face number, adding first image information of a newly added face image corresponding to the newly added face number in the target image to the list, acquiring second image information of the newly added face image corresponding to the newly added face number from an image with a frame number smaller than that of the target image, and adding the second image information to the list;
if the remainder of the frame number of the target picture and the frame interval quotient is zero, the target picture is a detection frame; the list stores information related to face images in each frame of picture of the video ordered frame; in the process of playing the video ordered frames, the smaller the frame number is, the earlier the picture playing time is.
The invention provides a device for processing a face image, which comprises the following components:
the acquisition module is used for acquiring a target picture from the video ordered frames with the frame numbers, and acquiring the number of face images detected from the target picture if the target picture is judged to be a detection frame according to the frame numbers of the target picture and the preset frame spacing;
the number creation module is used for creating a new face number corresponding to each new face number if the number of the face images detected from the target picture is not zero and the detected face images comprise the new face images with the corresponding face numbers not existing in the list;
the image tracking module is used for adding first image information of a newly added face image corresponding to the newly added face number in the target image to the list, acquiring second image information of the newly added face image corresponding to the newly added face number from an image with a frame number smaller than that of the target image, and adding the second image information to the list;
if the remainder of the frame number of the target picture and the frame interval quotient is zero, the target picture is a detection frame; the list stores information related to face images in each frame of picture of the video ordered frame; in the process of playing the video ordered frames, the smaller the frame number is, the earlier the picture playing time is.
The present embodiment provides an electronic device, including:
at least one processor, at least one memory, a communication interface, and a bus; wherein,,
the processor, the memory and the communication interface complete the communication with each other through the bus;
the communication interface is used for information transmission between the electronic device and communication devices of other electronic devices;
the memory stores program instructions executable by the processor, which invokes the program instructions to perform the method described above.
The present embodiment provides a non-transitory computer-readable storage medium storing computer instructions that cause the computer to perform the above-described method.
The embodiment of the invention provides a method and a device for processing a face image, wherein for a target image acquired from a video ordered frame, if the target image is a detection frame, image information of a face image with a face number in a list in the target image is added into the list, and on the other hand, the face number is newly added for the face image which appears in the target image and does not have the face number in the list, and the image of the face image is traced back before the target image. The method for backtracking the face image associates the same person in each frame, prevents the condition of frame loss and detection omission, and reduces the operation amount. The method can process each picture in the video frame or the picture set without repeated starting detection, and improves the processing efficiency. In addition, the image information of each face image is stored in the list as original data information, and it is convenient to output the information of each face image in different forms according to the list.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions of the prior art, the following description will briefly explain the drawings used in the embodiments or the description of the prior art, and it is obvious that the drawings in the following description are some embodiments of the present invention, and other drawings can be obtained according to these drawings without inventive effort for a person skilled in the art.
FIG. 1 is a schematic view of video frame composition in a method for face image processing for comparison according to an embodiment of the present invention;
FIG. 2 is a schematic diagram of processing images in a video frame in a method of face image processing for comparison according to another embodiment of the present invention;
FIG. 3 is a flowchart of a face image processing method according to another embodiment of the present invention;
FIG. 4 is a schematic diagram of processing an image in a video frame in a face image processing method according to another embodiment of the present invention;
FIG. 5 is a flowchart illustrating a method for face image processing according to another embodiment of the present invention;
fig. 6 is a block diagram of a face image processing apparatus according to another embodiment of the present invention;
Fig. 7 is a block diagram of an electronic device according to another embodiment of the present invention.
Detailed Description
For the purpose of making the objects, technical solutions and advantages of the embodiments of the present invention more apparent, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention, and it is apparent that the described embodiments are some embodiments of the present invention, but not all embodiments of the present invention. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
Before introducing the face image processing method provided in this embodiment, a method for processing a face image as a comparison is introduced first, fig. 1 is a schematic diagram of video frame composition in the method for processing a face image as a comparison shown in this embodiment, fig. 2 is a schematic diagram of processing an image in a video frame in the method for processing a face image as a comparison shown in this embodiment, referring to fig. 1 and fig. 2, a detection frame is usually several frames of pictures extracted from a video, in the method as a comparison, each frame of pictures in a video is identified, the operation amount is extremely large, the method cannot be used in an environment with low power consumption and calculation capability, or only the detection frame (the frame marked as D) in the video frame is detected, and the face appearing in the non-detection frame is easily ignored, so that omission is caused. In order to reduce the operation amount of the face detection process for the video, so that the face image processing method can also operate in an environment with lower power consumption and calculation capability, and avoid missing detection of the face image, fig. 3 is a schematic flow chart of the face image processing method provided in this embodiment, and referring to fig. 3, the method includes:
301: acquiring a target picture from the video ordered frames with frame numbers, and acquiring the number of face images detected from the target picture if the target picture is judged to be a detection frame according to the frame numbers of the target picture and the preset frame spacing;
302: if the number of the face images detected from the target picture is not zero and the detected face images comprise newly added face images with corresponding face numbers which are not in the list, creating the newly added face numbers corresponding to each newly added face image;
303: for each newly added face number, adding first image information of a newly added face image corresponding to the newly added face number in the target image to the list, acquiring second image information of the newly added face image corresponding to the newly added face number from an image with a frame number smaller than that of the target image, and adding the second image information to the list;
if the remainder of the frame number of the target picture and the frame interval quotient is zero, the target picture is a detection frame; the list stores information related to face images in each frame of picture of the video ordered frame; in the process of playing the video ordered frames, the smaller the frame number is, the earlier the picture playing time is.
The embodiment may be executed by a computer or a server, and the embodiment is not particularly limited, for example, a camera inputs a captured video ordered frame into the computer, the computer processes the input picture according to the face image processing method provided in the embodiment, and outputs video ordered frames of different people marked by different marking modes, so that a worker can conveniently realize face individual identification and human behavior study according to the marks in the video ordered frames.
A frame number is typically set for each picture in the order of picture play in the video sequence frame, starting with 0. The inter-frame distance is a set value, for example, the inter-frame distance N is 8. Face images of the same person correspond to the same face number. The list stores the image information of each face image in each picture, the faces in each frame of the video can be distinguished according to the list, and the action tracks of the people corresponding to each face image can be tracked by combining the frames of the video. The video ordered frames comprise a video or a picture set. Wherein, the picture set is a set of pictures obtained continuously in a time dimension.
It should be noted that, when the image before the target image is identified has the newly added face image corresponding to the newly added face number, the method may be implemented by adopting a tracking algorithm, for example, KCF, DCF, PLD, medianflow or a GOTURN algorithm is adopted to identify whether a face image in different frame images is a face image corresponding to the same person.
For a face image corresponding to a certain newly added face number, the first image information is image information (for example, position information) of the face image in a target picture, and the second image information is image information of the face image in a picture before the target picture. And after the detection frame detects the newly added face image, adding the first image information and the second image information of the newly added face image into the list. The second image information is the image information of each image of the newly added face image before the target image, which is obtained from the target image to trace back forward until the image of the newly added face image appears for the first time. After the first image information and the second image information are added to the list, all the image information from the first appearance of the newly added face image in the video to the appearance of the newly added face image in the target picture is included in the list, and the pictures in which the newly added face image appears are associated. The backtracking method enables the image information of the newly added face image in each frame of picture to be supplemented to the list under the condition that each frame of picture does not need to be detected, so that the image information of the newly added face image in each frame of picture can be read through the list. Further, the face images appearing in each frame of picture of the video ordered frame can be distinguished through the list.
The embodiment provides a face image processing method, which adds first image information of a face image with a face number in a list in a target image to the list on one hand, and adds a face number to a face image which appears in the target image and does not have the face number in the list on the other hand, and backtracks the image with the face image before the target image if the target image is a detection frame. The method for backtracking the face image associates the same person in each frame, prevents the condition of frame loss and detection omission, and reduces the operation amount. The method can process each picture in the video frame or the picture set without repeated starting detection, and improves the processing efficiency. In addition, the image information of each face image is stored in the list as original data information, and it is convenient to output the information of each face image in different forms according to the list.
Further, on the basis of the above embodiment, the method further includes:
and outputting video ordered frames which distinguish the face images corresponding to different face numbers in each frame of picture according to the face numbers and the image information corresponding to the face images in each frame of picture recorded in the list.
For example, according to the list, all face images corresponding to the same person in the video are used as boundary lines of the face images by using boxes with the same color.
The embodiment provides a face image processing method, which can distinguish people in video ordered frames in an intuitive manner according to a list, so that different people in the video ordered frames can be distinguished conveniently and quickly.
Further, on the basis of the foregoing embodiments, if the number of face images detected from the target picture is not zero, and the detected face images include newly added face images in which the corresponding face numbers do not exist in the list, creating a newly added face number corresponding to each newly added face image includes:
if the number of the face images detected from the target picture is not zero, judging whether corresponding face numbers exist in the list for each face image;
if the face image has a corresponding face number in the list, storing the position information of the face image in the target image, the face number corresponding to the face image, the frame number corresponding to the target image and the tracker corresponding to the face image in a correlated manner;
If the face image does not have the corresponding face number in the list, the face image is used as a newly added face image, and a newly added face number corresponding to the newly added face image is created;
the tracker corresponding to the face image is generated according to the characteristics of the face image and is used for identifying the face image.
It should be noted that, the position information recorded in the list may be composed of coordinates of a top left point of the face image and widths and heights of the face image, for example, table 1 is a face image information list provided in this embodiment, in table 1, numbers are frame numbers, letters are face numbers, face images of the same person correspond to the same face numbers, x and y are coordinates of a top left point of the face image, and h and w are heights and widths of the face image, respectively. For example, in table 1, image information of a face image is represented in format i-J (Xji, yji, wji, hji), where i represents a frame number of a picture, i is a number, J and J each represent a face number, J and J are letters, xji represents an abscissa of an upper left dot of the face image of the face number J (J and J are the same) in the picture of the frame number i, yji represents an ordinate of an upper left dot of the face image of the face number J in the picture of the frame number i, wji represents a width of an image area of the face image of the face number J in the picture of the frame number i, hji represents a height of the image area of the face image of the face number J in the picture of the frame number i.
The face tracker is an algorithm for identifying the face image from the picture by the feature information in the face image, for example, the face image in the picture is identified by using an area matching method or a feature matching method, which is not particularly limited in this embodiment.
Fig. 4 is a schematic diagram of processing an image in a video frame in the face image processing method provided in this embodiment, fig. 5 is a flowchart of a specific face image processing method, in fig. 4, a video frame is marked D, a tracking frame is marked T, a frame interval N is 8, and in the video frame shown in fig. 4, pictures with frame numbers of 0, 8 and 16 are detection frames. Referring to fig. 5, the method comprises the steps of:
step 1: let frame number f=0, maximum face number max=0;
step 2: taking an f video frame;
step 3: judging whether the frame is a D-class frame (detection frame), if yes, executing 4, otherwise executing 13, wherein if the formula f% N=0 is satisfied, the frame is a D-class frame, otherwise, the frame is a T-class frame, N is a frame interval, and f is a frame number of the video;
step 4: detecting the full image of the f video frame, wherein the number of detected face images is x;
step 5: judging whether the number x of the detected face images is 0, if so, executing 19, otherwise, executing 6;
Step 6: judging whether the matching item is matched with the face image with the face number in the list, if the matching item is present and the matched face number is i, executing 7, otherwise executing 8;
step 7: counting the position information of the face image in the f video frame into a list according to the existing face number i in the list, and executing 12;
step 8: when a new face image is detected, newly creating a face number max=max+1 to be counted into a list, and enabling a backtracking offset frame n=0.
For example, when the 8 th frame of picture is detected according to the method, new face images are detected, and the face images do not have corresponding face numbers in the first 7 frames of the list, so that the face numbers B and C are newly added for the new face images at the positions corresponding to the 8 th frame of picture in the list, and face region subgraphs (i.e. pictures of regions where the face images are located) of the B and C are utilized to generate face tracking templates of the B and C and store the face tracking templates in the B and C information in the list respectively. And a face tracking template is stored in the list, so that the face tracking template is convenient to track face images. For example, a face image with a face number B may be tracked using a face tracking template with a face number B.
It should be noted that, in judging whether the face image has a corresponding face number in the list, the face image is matched with the face image corresponding to each face number in the list, if so, the face number of the face image is the face number of the face image on the match. Such a face image matching method (for example, face image matching in step 6) may be realized by an area matching method or a feature matching method.
The area matching method comprises the steps of tracking the picture of the frame by using a tracker generated by the previous frame, and if the overlapping area of the tracking frame selection area and a certain face image detected by the frame exceeds a threshold value, the two face images are successfully matched and correspond to the same face number. Specifically, the area matching method includes: and (3) detecting and tracking the frame (a tracker generated by using the last detection frame) simultaneously, and if the coincidence area of the tracking frame selection area and the frame selection area of the detection frame is compared, if the coincidence area is larger than a threshold value such as 80 percent, matching the tracking frame selection area and the frame selection area, and if the length-width difference of the frame selection area is smaller than 80 percent, matching the tracking frame selection area and the frame selection area.
The feature matching method is used for directly extracting a frame selection region subgraph according to image information by adopting an image feature matching algorithm such as surf, sift or hog and matching with a detection frame selection subgraph, a feature matching threshold is set, the feature similarity is larger than the threshold, the matching is performed, the area matching method algorithm is simple and suitable for a scene with sparse faces, the opposite feature matching method is used for directly matching a subgraph small region, and the effect is little influenced by other factors.
The embodiment provides a face image processing method, when a new face image appears in a detection frame, a face number is newly added to the new face image so as to add image information of the new face image in a list.
Further, on the basis of the foregoing embodiments, the adding, for each newly added face number, first image information of a newly added face image corresponding to the newly added face number in the target picture to the list, and obtaining second image information of the newly added face image corresponding to the newly added face number from a picture with a frame number smaller than a frame number corresponding to the target picture, and adding the second image information to the list includes:
for each newly added face number, acquiring position information of a newly added face image corresponding to the newly added face number in the target picture, and storing the first image information, the newly added face number and a frame number corresponding to a tracker generated by the newly added face image in a correlated manner into the list as the first image information;
and acquiring the position information of the face image corresponding to the new face number in each picture which is smaller than the frame number corresponding to the target picture and comprises the face image corresponding to the new face number, and storing the second image information, the new face number, the tracker corresponding to the new face image and the frame number corresponding to the target picture in a correlated manner into the list as second image information.
Further, on the basis of the foregoing embodiments, the obtaining, from each picture including a face image corresponding to the new face number and having a frame number smaller than the frame number corresponding to the target picture, position information of the face image corresponding to the new face number in the picture, as second image information, and storing the second image information, the new face number, a tracker corresponding to the new face image, and the frame number corresponding to the target picture in association in the list includes:
setting the initial backtracking offset frame number to be 0, and circularly executing the first face tracking operation until no face image corresponding to the newly added face number exists in the tracking frame;
the first face tracking operation includes:
obtaining a frame number corresponding to a picture of a face image which is detected to exist at the latest time and corresponds to the newly added face number, taking the frame number as a current frame number, and calculating the sum of the current backtracking offset frame number minus 1 and the current frame number to obtain a tracking frame number;
and acquiring a picture corresponding to the tracking frame number, judging whether a face image corresponding to the newly added face number exists in the tracking frame as the tracking frame, if so, taking the position information of the face image corresponding to the newly added face number in the tracking frame as second image information, and storing the second image information, the newly added face number, a tracker corresponding to the newly added face image and the frame number corresponding to the target picture in a correlated manner into the list.
Further, the method further comprises the following steps:
before the frame numbers corresponding to the target pictures are stored, the number of the pictures which are equal to the frame spacing is used for backtracking a new face image in the stored pictures according to the list.
It should be noted that, the tracking frame is actually a frame picture with a remainder of the quotient of the frame number and the frame interval not being zero, but because the method provided in this embodiment adds the newly-appearing face image in each detection frame to the list, when tracing back the new face image, it is necessary to search for which images have newly-added face images in the tracking frame between the present detection frame and the previous detection frame.
As shown in fig. 5, the method for processing the face image further includes:
step 9: enabling the backtracking offset frame number n=n-1, taking the f+n video frame and carrying out face tracking;
step 10: tracking the face by carrying out tracking on the previous n frames, if the occurrence of the max number face is found, executing 11, otherwise, executing 12 after backtracking the frame without the occurrence of the max number face;
step 11: counting into the list according to the face number max, and repeating the execution from 9 until the condition 10 exits to 12;
step 12: let x=x-1 and continue to execute from 5 for the next face object detected.
For example, when the 8 th frame of picture is detected according to the method, as no matching item is found between the tracking and the face image in the list, the face numbers B and C are newly added in the 8 th frame of picture, the face image with the face number B is traced back, step 9-11 is executed, and when the face image with the face number B is found to exist in the 4 th, 5, 6 and 7 th frame of pictures, the second image information, the newly added face numbers and the tracking frame numbers are stored in the list in an associated manner, namely 7-B (Xb 7, yb7, wb7, hb 7), 6-B (Xb 6, yb6, hb 6), 5-B (Xb 5, yb5, wb 5), hb 5) and 4-B (Xb 4, yb4, wb4, hb 4) are added in the list. And backtracking (backward starting tracking) the face image with the face number of C, executing the steps 9-11, and adding 7-C (Xc 7, yc7, wc7 and Hc 7) to the list if the face image with the face number of C is found to exist in the 7 th frame of picture. Wherein the panoramic picture of the current frame and the trackers for B and C (including the characteristic attributes of B and C) generated from the detected region subgraph also need to be stored into the list.
The embodiment provides a face image processing method, which improves the image information of face images in a list by backtracking, and avoids missed detection.
Further, on the basis of the foregoing embodiments, the obtaining the target picture from the video ordered frame with the frame number, if the target picture is determined to be a detection frame according to the frame number of the target picture and the preset frame interval, obtaining the number of face images detected from the target picture, further includes:
if the target picture is judged to be a tracking frame according to the frame number of the target picture and the preset frame interval, the maximum face number is obtained from the list and is used as an initial tracking face number, and a second face tracking operation is circularly executed until no face image corresponding to the current tracking face number exists in the target picture;
the second face tracking operation includes:
judging whether a face image corresponding to the current tracking face number exists in the target picture, if so, storing the position information of the face image corresponding to the current tracking face number in the target picture, the current tracking face number and the frame number corresponding to the target picture in a correlated manner, otherwise, subtracting 1 from the tracking face number to obtain the current tracking face number;
And if the remainder of the frame number of the target picture and the frame interval quotient is not zero, the target picture is a tracking frame.
Further, the method further comprises the following steps: and calculating a difference value between the frame number corresponding to the target picture and the frame interval, and clearing information of pictures with frame numbers smaller than the difference value stored in the list.
As shown in fig. 5, the method for processing the face image further includes:
step 13: for a T-class face tracking frame (tracking frame), m=max is the maximum face number;
step 14: reading a number m face tracking template in the list;
step 15: carrying out face tracking on the current image of the image;
step 16: if the region matching the face number m is found, executing 17, otherwise executing 19;
step 17: counting into a list according to the existing face number m;
step 18: let m=m-1 and judge whether there is m number in the tabulation, if there is, continue to start executing from 14, otherwise carry out 19;
step 19: the frame number increases, preparing for the processing f=f+1 of the next frame;
step 20: all frame record caches in the list before the detected frame interval N are deleted and execution continues starting with 2.
For example, when the 4 th frame picture is detected in the above-described method, only the image information of the face image with the face number a is stored in the list content of the 4 th frame picture in the list. Image information of a face image with a face number of B in the 4 th frame of image is added into list contents corresponding to a detection frame with a frame number of 8 in the list.
As can also be seen from table 1, the image information of the 1 st frame picture is cleared from the image information of the 9 th frame picture stored in the list, and the image information of the 1 st and 2 nd frame pictures is cleared from the image information of the 10 th frame picture stored in the list.
As shown in fig. 4, the occlusion occurs in the 10 th, 14 th and 15 th frames, and as can be seen from the list of table 1, the image information of the face image having the face number C is not included in the 10 th, 14 th and 15 th frames due to the occlusion of the occlusion. Thus, the history tracking can be performed after the target is blocked according to the list of table 1.
The embodiment provides a face image processing method, a tracking frame adopts a simpler face image detection method, only the image information of the face image with the existing face number is added, the list generation flow is simplified, and the detection efficiency is improved. The image information of the picture before the detection frame is separated by N frames is deleted, so that the buffer space is released, and the running performance of the equipment is ensured. In addition, can realize the tracking to shelter from the thing, be difficult for losing with the root.
Further, on the basis of the above embodiments, the method further includes:
if the number of face images detected from the target picture is zero (see step 5 in fig. 5), or if there is no face image corresponding to the last newly added face number that is not traversed in the tracking frame (see step 12 in fig. 5), or if there is no face image corresponding to the current tracking face number in the target picture (see step 16 in fig. 5), the next frame picture of the target picture is obtained and is taken as a new target picture.
The embodiment provides a face image processing method, which is used for circularly processing each frame of picture in an ordered video frame until image information corresponding to a face image in each frame of picture is obtained.
The matching of the face images in the embodiment is performed by using a tracking algorithm to perform template comparison. The implementation of the method can comprise a list capable of storing a group of face frame coordinate marking sequences and a list of personnel numbers, a first-in first-out queue capable of caching N frames (N is the detection frame interval) of video, a face detector, a group of image trackers and a picture feature comparator (one or a combination of more of area or features, including surf, lift, hog or others). The method can effectively reduce the operation complexity. All frames of the same person can be confirmed in the video frame sequence even under the condition of temporary shielding, and the missing grabbing condition caused by frame separation detection can be effectively reduced. The historical frames are traced back through the tracking algorithm, the table lookup is used for continuing the personnel appearing in the history, the frame loss is prevented, the detection reliability and the efficiency are guaranteed by combining the historical frames and the historical frames, the identical personnel of each frame are associated, the frame loss is prevented, the efficiency is improved, and the identical personnel of each frame in the time dimension can be connected under the premise of shielding to facilitate the subsequent recognition processing.
TABLE 1 human face image information List
Fig. 6 is a block diagram of the face image processing apparatus provided in this embodiment, and referring to fig. 6, the face image processing apparatus includes an acquisition module 601, a number creation module 602 and an image tracking module 603, wherein,
the acquiring module 601 is configured to acquire a target picture from video ordered frames with frame numbers, and if the target picture is determined to be a detection frame according to the frame numbers of the target picture and a preset frame interval, acquire the number of face images detected from the target picture;
a number creating module 602, configured to create a new face number corresponding to each new face image if the number of face images detected from the target picture is not zero, and the detected face images include new face images in which no corresponding face number exists in the list;
an image tracking module 603, configured to add, for each new face number, first image information of a new face image corresponding to the new face number in the target picture to the list, obtain second image information of the new face image corresponding to the new face number from a picture with a frame number smaller than a frame number corresponding to the target picture, and add the second image information to the list;
If the remainder of the frame number of the target picture and the frame interval quotient is zero, the target picture is a detection frame; the list stores information related to face images in each frame of picture of the video ordered frame; in the process of playing the video ordered frames, the smaller the frame number is, the earlier the picture playing time is.
The device for processing the face image provided in this embodiment is applicable to the method for processing the face image provided in the foregoing embodiment, and will not be described herein.
The embodiment provides a device for processing a face image, which adds image information of a face image with a face number in a list in a target image in the target image to the list on one hand, and adds a face number to a face image with the face number in the list, which appears in the target image and does not exist in the list, and backtracks the image with the face image before the target image, if the target image is a detection frame, for the target image acquired from a video sequence frame. The method for backtracking the face image associates the same person in each frame, prevents the condition of frame loss and detection omission, and reduces the operation amount. The method can process each picture in the video frame or the picture set without repeated starting detection, and improves the processing efficiency. In addition, the image information of each face image is stored in the list as original data information, and it is convenient to output the information of each face image in different forms according to the list.
Fig. 7 is a block diagram showing the structure of an electronic apparatus provided in the present embodiment.
Referring to fig. 7, the electronic device includes: a processor (processor) 701, a memory (memory) 702, a communication interface (Communications Interface) 703, and a bus 704;
wherein,,
the processor 701, the memory 702 and the communication interface 703 complete communication with each other through the bus 704;
the communication interface 703 is used for information transmission between the electronic device and communication devices of other electronic devices;
the processor 701 is configured to invoke the program instructions in the memory 702 to perform the methods provided in the above method embodiments, for example, including: acquiring a target picture from the video ordered frames with frame numbers, and acquiring the number of face images detected from the target picture if the target picture is judged to be a detection frame according to the frame numbers of the target picture and the preset frame spacing; if the number of the face images detected from the target picture is not zero and the detected face images comprise newly added face images with corresponding face numbers which are not in the list, creating the newly added face numbers corresponding to each newly added face image; for each newly added face number, adding first image information of a newly added face image corresponding to the newly added face number in the target image to the list, acquiring second image information of the newly added face image corresponding to the newly added face number from an image with a frame number smaller than that of the target image, and adding the second image information to the list; if the remainder of the frame number of the target picture and the frame interval quotient is zero, the target picture is a detection frame; the list stores information related to face images in each frame of picture of the video ordered frame; in the process of playing the video ordered frames, the smaller the frame number is, the earlier the picture playing time is.
The present embodiment provides a non-transitory computer-readable storage medium storing computer instructions that cause a computer to perform the methods provided by the above-described method embodiments, for example, including: acquiring a target picture from the video ordered frames with frame numbers, and acquiring the number of face images detected from the target picture if the target picture is judged to be a detection frame according to the frame numbers of the target picture and the preset frame spacing; if the number of the face images detected from the target picture is not zero and the detected face images comprise newly added face images with corresponding face numbers which are not in the list, creating the newly added face numbers corresponding to each newly added face image; for each newly added face number, adding first image information of a newly added face image corresponding to the newly added face number in the target image to the list, acquiring second image information of the newly added face image corresponding to the newly added face number from an image with a frame number smaller than that of the target image, and adding the second image information to the list; if the remainder of the frame number of the target picture and the frame interval quotient is zero, the target picture is a detection frame; the list stores information related to face images in each frame of picture of the video ordered frame; in the process of playing the video ordered frames, the smaller the frame number is, the earlier the picture playing time is.
The present embodiment discloses a computer program product comprising a computer program stored on a non-transitory computer readable storage medium, the computer program comprising program instructions which, when executed by a computer, are capable of performing the methods provided by the above-described method embodiments, for example, comprising: acquiring a target picture from the video ordered frames with frame numbers, and acquiring the number of face images detected from the target picture if the target picture is judged to be a detection frame according to the frame numbers of the target picture and the preset frame spacing; if the number of the face images detected from the target picture is not zero and the detected face images comprise newly added face images with corresponding face numbers which are not in the list, creating the newly added face numbers corresponding to each newly added face image; for each newly added face number, adding first image information of a newly added face image corresponding to the newly added face number in the target image to the list, acquiring second image information of the newly added face image corresponding to the newly added face number from an image with a frame number smaller than that of the target image, and adding the second image information to the list; if the remainder of the frame number of the target picture and the frame interval quotient is zero, the target picture is a detection frame; the list stores information related to face images in each frame of picture of the video ordered frame; in the process of playing the video ordered frames, the smaller the frame number is, the earlier the picture playing time is.
Those of ordinary skill in the art will appreciate that: all or part of the steps for implementing the above method embodiments may be implemented by hardware associated with program instructions, where the foregoing program may be stored in a computer readable storage medium, and when executed, the program performs steps including the above method embodiments; and the aforementioned storage medium includes: various media that can store program code, such as ROM, RAM, magnetic or optical disks.
The above-described embodiments of electronic devices and the like are merely illustrative, wherein the elements described as separate elements may or may not be physically separate, and the elements shown as elements may or may not be physical elements, may be located in one place, or may be distributed over a plurality of network elements. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment. Those of ordinary skill in the art will understand and implement the present invention without undue burden.
From the above description of the embodiments, it will be apparent to those skilled in the art that the embodiments may be implemented by means of software plus necessary general hardware platforms, or of course may be implemented by means of hardware. Based on this understanding, the foregoing technical solution may be embodied essentially or in a part contributing to the prior art in the form of a software product, which may be stored in a computer readable storage medium, such as ROM/RAM, a magnetic disk, an optical disk, etc., including several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute the method described in the respective embodiments or some parts of the embodiments.
Finally, it should be noted that: the above embodiments are only for illustrating the technical solution of the embodiments of the present invention, and are not limited thereto; although embodiments of the present invention have been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical scheme described in the foregoing embodiments can be modified or some or all of the technical features thereof can be replaced by equivalents; such modifications and substitutions do not depart from the spirit of the corresponding technical solutions from the scope of the technical solutions of the embodiments of the present invention.
Claims (10)
1. A method of face image processing, comprising:
acquiring a target picture from the video ordered frames with frame numbers, and acquiring the number of face images detected from the target picture if the target picture is judged to be a detection frame according to the frame numbers of the target picture and the preset frame spacing;
if the number of the face images detected from the target picture is not zero and the detected face images comprise newly added face images with corresponding face numbers which are not in the list, creating the newly added face numbers corresponding to each newly added face image;
For each newly added face number, adding first image information of a newly added face image corresponding to the newly added face number in the target image to the list, acquiring second image information of the newly added face image corresponding to the newly added face number from an image with a frame number smaller than that of the target image, and adding the second image information to the list;
if the remainder of the frame number of the target picture and the frame interval quotient is zero, the target picture is a detection frame; the list stores information related to face images in each frame of picture of the video ordered frame; in the process of playing the video ordered frames, the smaller the frame number is, the earlier the picture playing time is.
2. The method as recited in claim 1, further comprising:
and outputting video ordered frames for distinguishing the face images corresponding to different face numbers in each frame of picture according to the face numbers and the image information corresponding to the face images in each frame of picture recorded in the list.
3. The method according to claim 1, wherein if the number of face images detected from the target picture is not zero and the detected face images include newly added face images with corresponding face numbers in the list, creating a newly added face number corresponding to each newly added face image includes:
If the number of the face images detected from the target picture is not zero, judging whether corresponding face numbers exist in the list for each face image;
if the face image has a corresponding face number in the list, storing the position information of the face image in the target image, the face number corresponding to the face image, the frame number corresponding to the target image and the tracker corresponding to the face image in a correlated manner;
if the face image does not have the corresponding face number in the list, the face image is used as a newly added face image, and a newly added face number corresponding to the newly added face image is created;
the tracker corresponding to the face image is generated according to the characteristics of the face image and is used for identifying the face image.
4. A method according to claim 3, wherein for each newly added face number, adding first image information of a newly added face image corresponding to the newly added face number in the target picture to the list, and obtaining second image information of a newly added face image corresponding to the newly added face number from a picture having a frame number smaller than a frame number corresponding to the target picture, and adding the second image information to the list, comprises:
For each newly added face number, acquiring position information of a newly added face image corresponding to the newly added face number in the target picture, and storing the first image information and the newly added face number into the list in a correlated manner by using the tracker corresponding to the newly added face image and the frame number corresponding to the target picture as the first image information;
and acquiring the position information of the face image corresponding to the new face number in each picture which is smaller than the frame number corresponding to the target picture and comprises the face image corresponding to the new face number, and storing the second image information, the new face number and the frame number corresponding to the tracker corresponding to the new face image in the list in a correlated manner as second image information.
5. The method according to claim 4, wherein the obtaining, from each picture having a frame number smaller than a frame number corresponding to the target picture and including a face image corresponding to the new face number, positional information of the face image corresponding to the new face number in the picture as second image information, and storing the second image information, the new face number, and a frame number corresponding to a tracker corresponding to the new face image in association with each other in the list includes:
Setting the initial backtracking offset frame number to be 0, and circularly executing the first face tracking operation until no face image corresponding to the newly added face number exists in the tracking frame;
the first face tracking operation includes:
obtaining a frame number corresponding to a picture of a face image which is detected to exist at the latest time and corresponds to the newly added face number, taking the frame number as a current frame number, and calculating the sum of the current backtracking offset frame number minus 1 and the current frame number to obtain a tracking frame number;
and acquiring a picture corresponding to the tracking frame number, judging whether a face image corresponding to the newly added face number exists in the tracking frame as the tracking frame, if so, taking the position information of the face image corresponding to the newly added face number in the tracking frame as second image information, and storing the second image information, the newly added face number, a tracker corresponding to the newly added face image and the frame number corresponding to the target picture in a correlated manner into the list.
6. The method according to claim 5, wherein the obtaining the target picture from the video ordered frame with the frame number, if the target picture is determined to be a detection frame according to the frame number of the target picture and the preset frame interval, obtaining the number of face images detected from the target picture, further comprises:
If the target picture is judged to be a tracking frame according to the frame number of the target picture and the preset frame interval, the maximum face number is obtained from the list and is used as an initial tracking face number, and a second face tracking operation is circularly executed until no face image corresponding to the current tracking face number exists in the target picture;
the second face tracking operation includes:
judging whether a face image corresponding to the current tracking face number exists in the target picture, if so, storing the position information of the face image corresponding to the current tracking face number in the target picture, the current tracking face number and the frame number corresponding to the target picture in a correlated manner, otherwise, subtracting 1 from the tracking face number to obtain the current tracking face number;
and if the remainder of the frame number of the target picture and the frame interval quotient is not zero, the target picture is a tracking frame.
7. The method as recited in claim 6, further comprising:
and if the number of the face images detected from the target picture is zero, or when the last face image corresponding to the newly-added face number which is not traversed does not exist in the tracking frame, or when the face image corresponding to the current tracking face number does not exist in the target picture, acquiring the next frame picture of the target picture as a new target picture.
8. A device for processing a face image, comprising:
the acquisition module is used for acquiring a target picture from the video ordered frames with the frame numbers, and acquiring the number of face images detected from the target picture if the target picture is judged to be a detection frame according to the frame numbers of the target picture and the preset frame spacing;
the number creation module is used for creating a new face number corresponding to each new face number if the number of the face images detected from the target picture is not zero and the detected face images comprise the new face images with the corresponding face numbers not existing in the list;
the image tracking module is used for adding first image information of a newly added face image corresponding to the newly added face number in the target image to the list, acquiring second image information of the newly added face image corresponding to the newly added face number from an image with a frame number smaller than that of the target image, and adding the second image information to the list;
if the remainder of the frame number of the target picture and the frame interval quotient is zero, the target picture is a detection frame; the list stores information related to face images in each frame of picture of the video ordered frame; in the process of playing the video ordered frames, the smaller the frame number is, the earlier the picture playing time is.
9. An electronic device, comprising:
at least one processor, at least one memory, a communication interface, and a bus; wherein,,
the processor, the memory and the communication interface complete the communication with each other through the bus;
the communication interface is used for information transmission between the electronic device and communication devices of other electronic devices;
the memory stores program instructions executable by the processor, the processor invoking the program instructions to perform the method of any of claims 1-7.
10. A non-transitory computer readable storage medium storing computer instructions that cause the computer to perform the method of any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811294871.3A CN111126113B (en) | 2018-11-01 | 2018-11-01 | Face image processing method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811294871.3A CN111126113B (en) | 2018-11-01 | 2018-11-01 | Face image processing method and device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111126113A CN111126113A (en) | 2020-05-08 |
| CN111126113B true CN111126113B (en) | 2023-10-10 |
Family
ID=70494783
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201811294871.3A Active CN111126113B (en) | 2018-11-01 | 2018-11-01 | Face image processing method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111126113B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113705422B (en) * | 2021-08-25 | 2024-04-09 | 山东浪潮超高清视频产业有限公司 | Method for obtaining character video clips through human faces |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2005339357A (en) * | 2004-05-28 | 2005-12-08 | Nippon Telegr & Teleph Corp <Ntt> | Device, method, and program for image processing |
| CN101620629A (en) * | 2009-06-09 | 2010-01-06 | 中兴通讯股份有限公司 | Method and device for extracting video index and video downloading system |
| CN102214479A (en) * | 2011-04-28 | 2011-10-12 | 深圳市同洲电子股份有限公司 | Hard disk data processing method and video stream pushing server |
| CN105354543A (en) * | 2015-10-29 | 2016-02-24 | 小米科技有限责任公司 | Video processing method and apparatus |
| CN105590097A (en) * | 2015-12-17 | 2016-05-18 | 重庆邮电大学 | Security system and method for recognizing face in real time with cooperation of double cameras on dark condition |
| CN108200432A (en) * | 2018-02-03 | 2018-06-22 | 王灏 | A kind of target following technology based on video compress domain |
| CN108230574A (en) * | 2017-12-20 | 2018-06-29 | 阿里巴巴集团控股有限公司 | Object storage terminal pickup method and device and electronic equipment |
| WO2018133666A1 (en) * | 2017-01-17 | 2018-07-26 | 腾讯科技（深圳）有限公司 | Method and apparatus for tracking video target |
-
2018
- 2018-11-01 CN CN201811294871.3A patent/CN111126113B/en active Active
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2005339357A (en) * | 2004-05-28 | 2005-12-08 | Nippon Telegr & Teleph Corp <Ntt> | Device, method, and program for image processing |
| CN101620629A (en) * | 2009-06-09 | 2010-01-06 | 中兴通讯股份有限公司 | Method and device for extracting video index and video downloading system |
| CN102214479A (en) * | 2011-04-28 | 2011-10-12 | 深圳市同洲电子股份有限公司 | Hard disk data processing method and video stream pushing server |
| CN105354543A (en) * | 2015-10-29 | 2016-02-24 | 小米科技有限责任公司 | Video processing method and apparatus |
| CN105590097A (en) * | 2015-12-17 | 2016-05-18 | 重庆邮电大学 | Security system and method for recognizing face in real time with cooperation of double cameras on dark condition |
| WO2018133666A1 (en) * | 2017-01-17 | 2018-07-26 | 腾讯科技（深圳）有限公司 | Method and apparatus for tracking video target |
| CN108230574A (en) * | 2017-12-20 | 2018-06-29 | 阿里巴巴集团控股有限公司 | Object storage terminal pickup method and device and electronic equipment |
| CN108200432A (en) * | 2018-02-03 | 2018-06-22 | 王灏 | A kind of target following technology based on video compress domain |
Non-Patent Citations (2)
| Title |
|---|
| 杨帅等.基于异步时域事件的高速运动目标的跟踪算法.计算机时代.2016,全文. * |
| 高灿.跨摄像机的多目标跟踪系统.《中国优秀硕博士学位论文全文数据库（硕士） 信息科技辑》.2018,(第第10期期),全文. * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111126113A (en) | 2020-05-08 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11803749B2 (en) | Method and device for identifying key time point of video, computer apparatus and storage medium | |
| CN110147717B (en) | Human body action recognition method and device | |
| CN110135246B (en) | Human body action recognition method and device | |
| Yan et al. | Learning the change for automatic image cropping | |
| US8971585B2 (en) | Image processing apparatus for retrieving object from moving image and method thereof | |
| CN110751674A (en) | Multi-target tracking method and corresponding video analysis system | |
| Long et al. | A c3d-based convolutional neural network for frame dropping detection in a single video shot | |
| US20150319373A1 (en) | Method and device to compose an image by eliminating one or more moving objects | |
| CN110610127B (en) | Face recognition method and device, storage medium and electronic equipment | |
| US9892320B2 (en) | Method of extracting attack scene from sports footage | |
| CN112509003B (en) | Method and system for solving target tracking frame drift | |
| US11544926B2 (en) | Image processing apparatus, method of processing image, and storage medium | |
| CN112257595A (en) | Video matching method, device, equipment and storage medium | |
| JP4496992B2 (en) | Animal up-frame detection method, program, and storage medium storing program, and animal up-shot detection method, animal up-frame or shot detection method, program, and storage medium | |
| CN103955494A (en) | Searching method and device of target object and terminal | |
| CN112381071A (en) | Behavior analysis method of target in video stream, terminal device and medium | |
| CN112118483A (en) | Video processing method, device, equipment and storage medium | |
| CN111126113B (en) | Face image processing method and device | |
| JP2014110020A (en) | Image processor, image processing method and image processing program | |
| CN109508408B (en) | Video retrieval method based on frame density and computer readable storage medium | |
| JP2021111228A (en) | Learning device, learning method, and program | |
| CN112651322B (en) | Cheek shielding detection method and device and electronic equipment | |
| CN112989869B (en) | Optimization method, device, equipment and storage medium of face quality detection model | |
| JP2018194956A (en) | Image recognition dive, method and program | |
| CN103268606B (en) | A kind of depth information compensation method of motion blur image and device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |