CN108875480A - A kind of method for tracing of face characteristic information, apparatus and system - Google Patents

A kind of method for tracing of face characteristic information, apparatus and system Download PDF

Info

Publication number
CN108875480A
CN108875480A CN201710698369.8A CN201710698369A CN108875480A CN 108875480 A CN108875480 A CN 108875480A CN 201710698369 A CN201710698369 A CN 201710698369A CN 108875480 A CN108875480 A CN 108875480A
Authority
CN
China
Prior art keywords
frame
face
image
picture frame
picture
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN201710698369.8A
Other languages
Chinese (zh)
Inventor
熊鹏飞
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Beijing Megvii Technology Co Ltd
Beijing Maigewei Technology Co Ltd
Original Assignee
Beijing Megvii Technology Co Ltd
Beijing Maigewei Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Beijing Megvii Technology Co Ltd, Beijing Maigewei Technology Co Ltd filed Critical Beijing Megvii Technology Co Ltd
Priority to CN201710698369.8A priority Critical patent/CN108875480A/en
Publication of CN108875480A publication Critical patent/CN108875480A/en
Pending legal-status Critical Current

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/16—Human faces, e.g. facial parts, sketches or expressions
    • G06V40/161—Detection; Localisation; Normalisation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/40—Scenes; Scene-specific elements in video content
    • G06V20/41—Higher-level, semantic clustering, classification or understanding of video scenes, e.g. detection, labelling or Markovian modelling of sport events or news items
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/40—Scenes; Scene-specific elements in video content
    • G06V20/46—Extracting features or characteristics from the video content, e.g. video fingerprints, representative shots or key frames
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/16—Human faces, e.g. facial parts, sketches or expressions
    • G06V40/168—Feature extraction; Face representation

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Physics & Mathematics (AREA)
  • Multimedia (AREA)
  • Oral & Maxillofacial Surgery (AREA)
  • Health & Medical Sciences (AREA)
  • Data Mining & Analysis (AREA)
  • Human Computer Interaction (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • General Health & Medical Sciences (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • General Engineering & Computer Science (AREA)
  • Evolutionary Computation (AREA)
  • Evolutionary Biology (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Artificial Intelligence (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Computational Linguistics (AREA)
  • Software Systems (AREA)
  • Image Analysis (AREA)
  • Image Processing (AREA)

Abstract

The present invention provides a kind of method for tracing of face characteristic information, apparatus and system, are related to the technical field of image recognition, and this method includes：The first picture frame and processed second picture frame to be processed are obtained, the first picture frame and the second picture frame are at least a pair of of successive image frame in video flowing, include estimating face frame in the first picture frame, include the face frame having detected that in the second picture frame；It is handled using the region that the face frame that has detected that of estimating region and second picture frame that face frame identified of the deep neural network model to the first picture frame is identified, the face characteristic information of the first picture frame is tracked based on processing result, face characteristic information includes face frame and/or human face characteristic point, Face detection method for tracing in the prior art is solved when coping with complicated face movement, the poor technical problem of the accuracy of recognition of face.

Description

A kind of method for tracing of face characteristic information, apparatus and system
Technical field
The present invention relates to the technical fields of image recognition, method for tracing, dress more particularly, to a kind of face characteristic information It sets and system.
Background technique
Face shape point tracking technique refers to tracks one or more faces in continuous video sequence, and exports in real time The shape point of face in every frame.The technology all has very important effect in many occasions, for example, the activity of locking particular person Track, driver fatigue detect, the processing of face block diagram picture in mobile phone U.S. face, the addition etc. of stage property in net cast.Face with The accuracy of track, robustness and efficiency are technology main problem of concern.
In the prior art, which is broadly divided into two classes, and the first kind is by there is the gradient descent method of supervision (Supervised Descent Method, abbreviation SDM) model or convolutional neural networks (Convolutional Neural Network, abbreviation CNN) iterative calculation export face shape point.Such method is the face based on previous frame in video sequence Shape point identifies the face character state of present frame.Due to such method depend on previous frame shape point, cause this method compared with It is often difficult to cope with when big face movement or attitudes vibration.Another kind of method is in attitudes vibration or the biggish feelings of expression shape change Under condition, complicated tracking situation is coped with by tracking organ profile point, but still can not to solve face movement too fast for this method The problem of.
In view of the above-mentioned problems, not putting forward effective solutions also.
Summary of the invention
In view of this, the purpose of the present invention is to provide a kind of method for tracing of face characteristic information, apparatus and system, with Face tracking methods in the prior art are solved when coping with complicated face movement, the poor technology of the accuracy of recognition of face Problem.
In a first aspect, the embodiment of the invention provides a kind of method for tracing of face characteristic information, including：It obtains to be processed The first picture frame and processed second picture frame, the first image frame and second picture frame be in video flowing extremely Lack a pair of successive image frame, include estimating face frame in the first image frame, includes having detected in second picture frame The face frame arrived；Using deep neural network model to estimated described in the first image frame region that face frame is identified and The region that the face frame having detected that of second picture frame is identified is handled, based on described in processing result tracking The face characteristic information of first picture frame, the face characteristic information include face frame and/or human face characteristic point.
Further, using deep neural network model to estimating what face frame was identified described in the first image frame The region that the face frame having detected that of region and second picture frame is identified is handled, and is chased after based on processing result The face characteristic information of track the first image frame includes：Using the deep neural network model to the first image frame The region that the face frame having detected that for estimating region and second picture frame that face frame is identified is identified It is handled, to obtain the first processing result, wherein first processing result includes：The first image frame is relative to institute The face frame offset of the second picture frame is stated, the first image frame is deviated relative to the human face characteristic point of second picture frame Amount；Face characteristic information based on first processing result tracking the first image frame.
Further, the deep neural network includes：First nerves network and nervus opticus network, wherein described One neural network includes the identical basic network of two structures, and the network end-point of two basic networks is respectively with described second The input terminal of neural network is connected, and the input terminal of two basic networks is for inputting at least a pair of of consecutive image Frame；The first nerves network and the nervus opticus network are by one or more convolutional layers, one or more pond layers The network being formed by connecting with one or more non-linear layers according to predetermined connection type, and the first nerves network and described The connection type of two neural networks is same or different.
Further, based on first processing result tracking the first image frame face frame include：In conjunction with described The coordinate of the face frame of face frame offset and second picture frame determines the face frame of the first image frame.
Further, based on first processing result tracking the first image frame human face characteristic point include：In conjunction with The coordinate of the human face characteristic point of the human face characteristic point offset and second picture frame determines intermediate human face characteristic point, In, the intermediate human face characteristic point is located in the coordinate system of the face frame of the first image frame；By the intermediate face characteristic Point is normalized, and obtains the human face characteristic point of the first image frame, wherein the face characteristic of the first image frame Point is located in the coordinate system of the first image frame.
Further, the method also includes：Characteristic point is instructed in acquisition, wherein described to instruct characteristic point to have detected The second picture frame human face characteristic point；It is described to be estimated using deep neural network model to described in the first image frame The region that the face frame having detected that in region and second picture frame that face frame is identified is identified is handled Including：By the region and second figure for instructing to estimate that face frame is identified described in characteristic point, the first image frame The region that the face frame having detected that as described in frame is identified, which is input in the deep neural network model, to be handled, and is obtained To second processing result, wherein the second processing result includes：The first image frame is relative to second picture frame The human face characteristic point of face frame offset and the first image frame；And described is determined based on the face frame offset The face frame of one picture frame.
Further, at use deep neural network model is to the first image frame and second picture frame While reason, the method also includes：Obtain what the deep neural network model was calculated based on the first image frame Confidence level；Determine in the face frame of the first image frame whether include face based on the confidence level；It is determining not including In the case where face, stopping tracks the face in the video flowing.
Further, when the first image frame is first frame image in the video flowing, second image Frame is at least one duplicating image frame of the first image frame, and comprising first passing through Face datection in advance in second picture frame The face characteristic information that algorithm detects；When the first image frame is not first frame image, second figure At least one successive image frame before being the first image frame as frame.
Further, the position phase of the face frame for estimating face frame and second picture frame in the first image frame Together.
Further, the method also includes：Acquisition carries the first face video sequence of the first change information, In, first change information includes at least one of：Motion change, expression shape change and attitudes vibration；To described the first The face frame of each frame and human face characteristic point are labeled in face video sequence；Based on first face video after mark Sequence construct multiple images pair, wherein described multiple images are to for carrying out learning training to the deep neural network model.
Further, described multiple images centering includes the first image and the second image, the first image and described the Two images meet following at least one relationship：The adjacent picture frame of any two in the first face video sequence；It is described The picture frame of one video frame in any two interval in first face video sequence；Any two in second face video sequence Adjacent picture frame, the second face video sequence are the reversed video sequence of the first face video sequence；Described The picture frame of one video frame in any two interval in two face video sequences.
Further, the method also includes：Obtain the first face still image, wherein the first face static map It seem the figure that in the case where different illumination conditions and/or collected personnel collect in the case where different faces expression Picture；To the face performance objective processing in the first face still image, the second face still image is obtained, wherein described Target processing includes at least one of：Translation scales, rotation, affine transformation, and variation illumination changes expression, blocks processing； To in the first face still image and the second face still image face frame and human face characteristic point mark respectively Note；Based on after mark the first face still image and the second face still image construct described multiple images pair, Wherein, described multiple images are to for carrying out learning training to the deep neural network model.
Second aspect, the embodiment of the present invention also provide a kind of follow-up mechanism of face characteristic information, including：First obtains list Member, for obtaining the first picture frame and processed second picture frame to be processed, the first image frame and second figure It include estimating face frame, second figure in the first image frame as frame is at least a pair of of successive image frame in video flowing As including the face frame detected in frame；First processing units, for using deep neural network model to described first The face frame having detected that for estimating region and second picture frame that face frame is identified of picture frame is marked The region of knowledge is handled, based on the face characteristic information of processing result tracking the first image frame, the face characteristic letter Breath includes face frame and/or human face characteristic point.
Further, the first processing units include：First processing module, for using the deep neural network mould Type is had detected that described in the region and second picture frame that face frame is identified to estimating described in the first image frame The region that is identified of face frame handled, to obtain the first processing result, wherein first processing result includes：Institute Face frame offset of first picture frame relative to second picture frame is stated, the first image frame is relative to second figure As the human face characteristic point offset of frame；Tracing module, for based on first processing result tracking the first image frame Face characteristic information.
Further, the deep neural network includes：First nerves network and nervus opticus network, wherein described One neural network includes the identical basic network of two structures, and the network end-point of two basic networks is respectively with described second The input terminal of neural network is connected, and the input terminal of two basic networks is for inputting at least a pair of of consecutive image Frame；The first nerves network and the nervus opticus network are by one or more convolutional layers, one or more pond layers The network being formed by connecting with one or more non-linear layers according to predetermined connection type, and the first nerves network and described The connection type of two neural networks is same or different.
Further, tracing module is used for：In conjunction with the face frame of the face frame offset and second picture frame Coordinate determines the face frame of the first image frame.
Further, the tracing module is also used to：In conjunction with the human face characteristic point offset and second picture frame The coordinate of human face characteristic point determine intermediate human face characteristic point, wherein the intermediate human face characteristic point is located at the first image In the coordinate system of the face frame of frame；The intermediate human face characteristic point is normalized, the first image frame is obtained Human face characteristic point, wherein the human face characteristic point of the first image frame is located at seat belonging to the face frame of the first image frame In mark system.
Further, described device further includes：Second acquisition unit instructs characteristic point for obtaining, wherein the guidance The human face characteristic point for including in the face characteristic information that characteristic point has detected for second picture frame；First processing is single Member includes：Second processing module, for by it is described instruct to estimate face frame described in characteristic point, the first image frame identified Region and the region that is identified of the face frame having detected that of second picture frame be input to the depth nerve net It is handled in network model, obtains second processing result, wherein the second processing result includes：The first image frame phase The human face characteristic point of face frame offset and the first image frame for second picture frame；And determining module, it uses In the face frame for determining the first image frame based on the face frame offset.
Further, described device further includes：Third acquiring unit is used in use deep neural network model to described While first picture frame and second picture frame are handled, obtains the deep neural network model and be based on described first The confidence level that picture frame is calculated；Determination unit, for determining the face frame of the first image frame based on the confidence level In whether include face；Stop unit, for stopping to the people in the video flowing in the case where determining not including face Face is tracked.
Further, when the first image frame is first frame image in the video flowing, second image Frame is at least one duplicating image frame of the first image frame, and comprising first passing through Face datection in advance in second picture frame The face characteristic information that algorithm detects；When the first image frame is not first frame image, second figure At least one successive image frame before being the first image frame as frame.
Further, the position phase of the face frame for estimating face frame and second picture frame in the first image frame Together.
Further, described device further includes：Acquisition unit, for acquiring the first face for carrying the first change information Video sequence, wherein first change information includes at least one of：Motion change, expression shape change and attitudes vibration；The One mark unit, for each frame in the first face video sequence face frame and human face characteristic point be labeled；The One construction unit, for constructing multiple images pair based on the first face video sequence after mark, wherein the multiple Image is to for carrying out learning training to the deep neural network model.
Further, described multiple images centering includes the first image and the second image, the first image and described the Two images meet following at least one relationship：The adjacent picture frame of any two in the first face video sequence；It is described The picture frame of one video frame in any two interval in first face video sequence；Any two in second face video sequence Adjacent picture frame, the second face video sequence are the reversed video sequence of the first face video sequence；Described The picture frame of one video frame in any two interval in two face video sequences.
Further, described device further includes：4th acquiring unit, for obtaining the first face still image, wherein institute State that the first face still image is in the case where different illumination conditions and/or collected personnel are the different faces expression the case where Under the image that collects；The second processing unit, for handling the face performance objective in the first face still image, Obtain the second face still image, wherein the target processing includes at least one of：Translation scales, rotation, affine change It changes, variation illumination, changes expression, block processing；Second mark unit, for the first face still image and described the Face frame and human face characteristic point in two face still images are labeled respectively；Second construction unit, after based on mark The first face still image and mark after the second face still image construct described multiple images pair, wherein Described multiple images are to for carrying out learning training to the deep neural network model.
The third aspect, the embodiment of the present invention also provide a kind of tracing system of face characteristic information, the system comprises：It takes the photograph As device, processor and storage device；The photographic device, for acquire face in real time or offline video stream；The storage Computer program is stored on device, the computer program executes such as claim 1 to 12 when being run by the processor Described in any item methods.
Fourth aspect, the embodiment of the present invention also provide a kind of meter of non-volatile program code that can be performed with processor Calculation machine readable medium, said program code make the processor execute any of the above-described method.
In embodiments of the present invention, current time the first picture frame and processed second image to be processed is obtained first Frame；Then, the region identified using estimated face frame of the deep neural network model to the first picture frame and the second image The region that the face frame of frame having detected that is identified is analyzed and processed, it will be able to is adjusted, be obtained to the first picture frame Target face frame and target human face characteristic point after first image framing control.Method in compared with the existing technology can only solve Face is mobile the case where varying less, in embodiments of the present invention, by deep neural network model can face it is mobile compared with In the case where for complexity, the human face characteristic point of each frame image is still accurately detected, and then solves face in the prior art Method for tracing cope with complicated face it is mobile when, the poor technical problem of the accuracy of recognition of face, thus realize it is accurate right The technical effect that facial image in video flowing is identified.
Other features and advantages of the present invention will illustrate in the following description, also, partly become from specification It obtains it is clear that understand through the implementation of the invention.The objectives and other advantages of the invention are in specification, claims And specifically noted structure is achieved and obtained in attached drawing.
To enable the above objects, features and advantages of the present invention to be clearer and more comprehensible, preferred embodiment is cited below particularly, and cooperate Appended attached drawing, is described in detail below.
Detailed description of the invention
It, below will be to specific in order to illustrate more clearly of the specific embodiment of the invention or technical solution in the prior art Embodiment or attached drawing needed to be used in the description of the prior art be briefly described, it should be apparent that, it is described below Attached drawing is some embodiments of the present invention, for those of ordinary skill in the art, before not making the creative labor It puts, is also possible to obtain other drawings based on these drawings.
Fig. 1 is the schematic diagram for the electronic equipment that the present invention implements embodiment；
Fig. 2 is a kind of flow chart of the method for tracing of face characteristic information according to an embodiment of the present invention；
Fig. 3 is a kind of schematic diagram of the follow-up mechanism of human face characteristic point according to an embodiment of the present invention；
Fig. 4 is a kind of schematic diagram of the identifying system of human face characteristic point according to an embodiment of the present invention.
Specific embodiment
In order to make the object, technical scheme and advantages of the embodiment of the invention clearer, below in conjunction with attached drawing to the present invention Technical solution be clearly and completely described, it is clear that described embodiments are some of the embodiments of the present invention, rather than Whole embodiments.Based on the embodiments of the present invention, those of ordinary skill in the art are not making creative work premise Under every other embodiment obtained, shall fall within the protection scope of the present invention.
Embodiment one：
Firstly, describing the method for tracing and device of the face characteristic information for realizing the embodiment of the present invention referring to Fig.1 Exemplary electronic device 100.
As shown in Figure 1, electronic equipment 100 include one or more processors 102, it is one or more storage device 104, defeated Enter device 106, output device 108 and image collecting device 110, these components pass through bus system 112 and/or other forms Bindiny mechanism's (not shown) interconnection.It should be noted that the component and structure of electronic equipment 100 shown in FIG. 1 are only exemplary, And not restrictive, as needed, the electronic equipment also can have other assemblies and structure.
The processor 102 can be central processing unit (CPU) or have data-handling capacity and/or instruction execution The processing unit of the other forms of ability, and the other components that can control in the electronic equipment 100 are desired to execute Function.
The storage device 104 may include one or more computer program products, and the computer program product can To include various forms of computer readable storage mediums, such as volatile memory and/or nonvolatile memory.It is described easy The property lost memory for example may include random access memory (RAM) and/or cache memory (cache) etc..It is described non- Volatile memory for example may include read-only memory (ROM), hard disk, flash memory etc..In the computer readable storage medium On can store one or more computer program instructions, processor 102 can run described program instruction, to realize hereafter institute The client functionality (realized by processor) in the embodiment of the present invention stated and/or other desired functions.In the meter Can also store various application programs and various data in calculation machine readable storage medium storing program for executing, for example, the application program use and/or The various data etc. generated.
The input unit 106 can be the device that user is used to input instruction, and may include keyboard, mouse, wheat One or more of gram wind and touch screen etc..
The output device 108 can export various information (for example, image or sound) to external (for example, user), and It and may include one or more of display, loudspeaker etc..
Described image sensor 110 can be shot the desired image of user (such as photo, video etc.), and will be captured Image be stored in the storage device 104 for other components use.
Illustratively, for realizing the method for tracing of face characteristic information according to an embodiment of the present invention and the example of device Electronic equipment may be implemented as on the mobile terminals such as smart phone, tablet computer.
Embodiment two：
In the following, reference Fig. 2 to be described to the method for tracing of face characteristic information according to an embodiment of the present invention.
According to embodiments of the present invention, a kind of embodiment of the method for tracing of face characteristic information is provided, needs to illustrate It is that step shown in the flowchart of the accompanying drawings can execute in a computer system such as a set of computer executable instructions, Also, although logical order is shown in flow charts, and it in some cases, can be to be different from sequence execution herein Shown or described step.
Fig. 2 is a kind of flow chart of the method for tracing of face characteristic information according to an embodiment of the present invention, as shown in Fig. 2, This method comprises the following steps：
Step S202 obtains the first picture frame and processed second picture frame to be processed, the first picture frame and second Picture frame is at least a pair of of successive image frame in video flowing, includes estimating face frame in the first picture frame, in the second picture frame Including the face frame identified.
In embodiments of the present invention, can by photographic device acquire face in real time or offline video stream, for example, passing through Installation front camera on mobile terminals or rear camera acquisition face are in real time or offline video stream, and successively right Picture frame in video flowing is handled.Wherein, it after getting video flowing, needs using Face datection algorithm to the video First picture frame in stream carries out Face datection, to obtain corresponding image.
It should be noted that the second picture frame is first when the first picture frame is first frame image in video flowing At least one duplicating image frame of picture frame, and comprising first passing through the people that Face datection algorithm detects in advance in the second picture frame Face characteristic information.For example, video flowing includes N number of video frame images, when the first picture frame estimates face to be marked in video flowing When first video frame images of frame, the second picture frame is the duplicating image of the first picture frame, and the duplicating image is logical in advance Image after unusually face detection algorithm is detected.
When the first picture frame is not first frame image, at least one before the second picture frame is the first picture frame connects Continuous picture frame.For example, video flowing includes N number of video frame images, when the first picture frame is m-th video frame images in video flowing When, M is less than or equal to N, then can be the M-1 video frame images in video flowing in the second picture frame；Second image It can also be M-1 video frame images and the M-2 video frame images in video flowing in frame.That is, processed Two picture frames can be one, can also be multiple.
It should be noted that in embodiments of the present invention, face frame is estimated in calibration in advance in m-th video frame images, it should Estimating face frame is to be made according to the face collimation mark of M-1 video frame images or the M-2 video frame images.It is preferred that The position on ground, the face frame for estimating face frame and the second picture frame in the first picture frame is identical.
Step S204 estimates the region and that face frame is identified to the first picture frame using deep neural network model The region that the face frame of two picture frames having detected that is identified is handled, and the people of the first picture frame is tracked based on processing result Face characteristic information, face characteristic information include face frame and/or human face characteristic point.
In embodiments of the present invention, the deep neural network model (Deep Neural Network, abbreviation DNN) of use It is a kind of end-to-end network, the machine-learning process (namely deep learning process) of deep neural network model is a kind of end pair The learning process at end.DNN is the important branch of artificial intelligence field, and the deep learning process of DNN is to determine network weight weight values Process, in general, learning process be referred to as training network (training).Once training is completed, program be can be used by training Determining weight is calculated.
In embodiments of the present invention, the human face characteristic point and face of each video frame in video flowing can be identified by DNN Frame, recognition speed is faster.DNN copes with complicated face movement, attitudes vibration and expression shape change, greatly improves feature The efficiency of point location.
In embodiments of the present invention, current time the first picture frame and processed second image to be processed is obtained first Frame；Then, the region identified using estimated face frame of the deep neural network model to the first picture frame and the second image The region that the face frame of frame having detected that is identified is analyzed and processed, it will be able to is adjusted, be obtained to the first picture frame Target face frame and target human face characteristic point after first image framing control.Method in compared with the existing technology can only solve Face is mobile the case where varying less, in embodiments of the present invention, by deep neural network model can face it is mobile compared with In the case where for complexity, the human face characteristic point of each frame image is still accurately detected, and then solves face in the prior art Method for tracing cope with complicated face it is mobile when, the poor technical problem of the accuracy of recognition of face, thus realize it is accurate right The technical effect that facial image in video flowing is identified.
Face block diagram picture corresponding to frame image every in video flowing is being known by above-mentioned deep neural network model It before not, needs to carry out learning training to deep neural network model, in training, firstly, obtaining to training data；Then, Learning training is carried out to deep neural network model according to training data.
It obtains there are many modes to training data, in embodiments of the present invention, is mainly being obtained using following two mode It takes to training data：Continuous data set is compiled based on video sequence, image log evidence is compiled based on disturbance of data Collection.Mode of the above two acquisition to training data will be introduced respectively below.
Mode one compiles continuous data set based on video sequence
Firstly, acquisition carries the first face video sequence of the first change information, wherein the first change information include with It is at least one lower：Motion change, expression shape change and attitudes vibration；
In embodiments of the present invention, can acquire a batch by photographic device has motion change, expression shape change and posture The face video sequence (that is, above-mentioned first face video sequence) of variation.
Then, the face frame and human face characteristic point of each frame in the first face video sequence are labeled；Specifically, exist Face location and human face characteristic point are marked out in each frame image in first face video sequence.
It in embodiments of the present invention, can be special to each frame image labeling face location and face using Face datection algorithm Sign point.For example, collecting a large amount of facial images in advance, the position of face frame is manually marked out on every image, utilizes engineering Learning method (deep learning, alternatively, the adaboost method based on haar feature) is trained to obtain Face datection model.So Afterwards, face location is marked out in each frame image by the face detection model in the first face video sequence and face is special Sign point.
Finally, constructing multiple images pair based on the first face video sequence after mark, wherein multiple images are to being used for Learning training is carried out to deep neural network model.
Based on after mark the first face video sequence form multiple images clock synchronization, can after mark first 2 frame of positive arbitrary continuation is extracted in face video sequence respectively, positive arbitrary continuation jumps 1 frame, and reversed continuous 2 frame is reversed continuous Jump the image composition image pair of 1 frame.
That is, the first image for including in the multiple images pair of building and the second image meet following at least one Relationship：
Positive 2 frame of arbitrary continuation：The adjacent picture frame of any two in first face video sequence；
Positive arbitrary continuation jumps 1 frame：The picture frame of one video frame in any two interval in first face video sequence；
Reversed continuous 2 frame：The adjacent picture frame of any two in second face video sequence, the second face video sequence For the reversed video sequence of the first face video sequence；
1 frame of reversed continuous jump：The picture frame of one video frame in any two interval in second face video sequence.
Since collected number of videos is limited, it in embodiments of the present invention, can also be using based on disturbance of data Image log is compiled according to the mode of collection to construct image pair.
Mode two compiles image log according to collection based on disturbance of data
Firstly, obtaining the first face still image, wherein the first face still image is the different illumination conditions the case where The image that lower and/or collected personnel collect in the case where different faces expression；
Specifically, acquisition a batch has the face still image of different illumination conditions and/or different expressions (that is, the first Face still image).Wherein, the quantity of collected first face still image is multiple.
Then, the face performance objective in the first face still image is handled, obtains the second face still image, In, target processing includes at least one of：Translation scales, rotation, affine transformation, and variation illumination changes expression, blocks place Reason.
Next, to face frame and human face characteristic point difference in the first face still image and the second face still image It is labeled；
After collecting multiple first face still images, so that it may to every first face still image and the first Face still image is labeled, with mark every first face still image and the first face still image human face characteristic point and Face frame.After being labeled to every first face still image and the first face still image, so that it may to every One people's still image and the first face still image translate at random, scaling and rotation transformation, radiation transformation, variation illumination, Variation expression is handled with the targets such as processing are blocked.
Finally, based on the first face still image and the second face still image building multiple images pair after mark, In, multiple images are to for carrying out learning training to deep neural network model.After performance objective processing, so that it may will be former Image after image and performance objective processing is as one group of image pair.
It should be noted that through the above way one and mode two description it is found that building each image pair include Two images, i.e. the first image and the second image, then, using first image and the second image as deep neural network model Input, to be trained to deep neural network model.However, it is desirable to explanation, each image pair is in addition to the first figure It can also include third image or N image etc., to train depth by N number of image except picture and the second image Neural network model.It can specifically be chosen according to actual needs using several images training deep neural network model.
It should be noted that the input of deep neural network model is with two images in the following embodiments of the present invention For be illustrated.
Above-mentioned multiple images are being constructed to later, so that it may instruct to deep neural network model based on multiple image Practice.In embodiments of the present invention, the network structure of deep neural network model can be any number of convolutional layers, pond layer and non- Linear layer is connected by way of cascading or jumping.
In an optional embodiment, deep neural network includes：First nerves network and nervus opticus network, In, first nerves network includes the identical basic network of two structures, and the network end-point of two basic networks is refreshing with second respectively Input terminal through network is connected, and the input terminal of two basic networks is for inputting at least a pair of of successive image frame；First nerves Network and nervus opticus network are by one or more convolutional layers, one or more pond layers and one or more non-linear layers According to the network that predetermined connection type is formed by connecting, and first nerves network it is identical with the connection type of nervus opticus network or It is different.
Specifically, in the case where each image pair includes the first image and the second image, the network is by two groups of structures Identical basic network (that is, above-mentioned first nerves network) composition, two groups of basic networks are respectively used to input each image pair In two images, to carry out feature extractions to two images, extraction obtains feature vector, wherein this feature includes color The characteristic informations such as feature, shape feature and textural characteristics.In the end of two basic networks, two basic networks are combined into one A network, the network after combining continue subsequent convolutional layer of connecting, and pond layer and non-linear layer etc. are (that is, above-mentioned nervus opticus Network).
It should be noted that in embodiments of the present invention, it can be using the depth nerve that two kinds of pattern pairs have constructed Network model is trained.
Mode one, the first image of each image pair to have marked face frame and the second image are as depth nerve net The input of network model, using face frame offset (pre-set) and human face characteristic point offset (pre-set) as The output of deep neural network model, is trained deep neural network model.
Mode two, the first image of each image pair to have marked face frame and the second image and the first image Input of the human face characteristic point as deep neural network model, with face frame offset (pre-set) and the second image Output of the human face characteristic point as deep neural network model, deep neural network model is trained.It needs to illustrate It is, at this point, the first image is the previous frame image of the second image.Specifically, the can be inputted in the end of two basic networks The human face characteristic point of one image, as instructing characteristic point.
To deep neural network model training complete after, so that it may to deep neural network model after training into Row is disposed, and the human face characteristic point and face of every frame image in video flowing are determined based on the deep neural network model after deployment Frame.
In an optional embodiment, marked using estimate face frame of the deep neural network model to the first picture frame The region that the face frame of the region of knowledge and the second picture frame having detected that is identified is handled, based on processing result tracking the The face characteristic information of one picture frame includes the following steps：
Step S11 estimates the region and that face frame is identified to the first picture frame using deep neural network model The region that the face frame of two picture frames having detected that is identified is handled, to obtain the first processing result, wherein at first Managing result includes：Face frame offset of first picture frame relative to the second picture frame, the first picture frame is relative to the second image The human face characteristic point offset of frame；
Step S12 tracks the face characteristic information of the first picture frame based on the first processing result.
The mode of above-mentioned steps S11 and step S12 description is to pass through depth corresponding to mode one in the case where mode one Spend target face frame and target human face characteristic point that neural network model tracks the first picture frame.
For example, by the nth frame image (that is, above-mentioned first picture frame) in video flowing and the N-1 frame image in video flowing (that is, above-mentioned second picture frame) is input in deep neural network model, so that the deep neural network is to the pre- of nth frame image Estimate the region that the face frame having detected that in region and N-1 frame image that face frame is identified is identified to be analyzed and processed, Processing obtains the face frame and human face characteristic point of nth frame image.
It should be noted that the nth frame image being input in deep neural network model is with N-1 frame image with identical The face frame of position, that is to say, that the face frame of estimating of nth frame image is determined according to the face frame of N-1 frame image, Wherein, the face frame of N-1 frame image is what the method provided through the embodiment of the present invention identified.
When face occurs mobile, the face for estimating the choosing of face frame institute's frame in the first picture frame is possible to be groups of people Face, therefore, it is necessary to the face frames of estimating to the first picture frame to be updated, and determine the human face characteristic point of the first picture frame. In embodiments of the present invention, by constantly adjusting face frame, the big of face frame can not quickly be adjusted by avoiding traditional tracking Small and position defect, so that tracking is more robust.
It is assumed that face frame offset-lists are shown as DF (x, y), face characteristic shape point offset-lists are shown as DS (x, y), N- The coordinate representation of the human face characteristic point of 1 frame image is S0, and the coordinate representation of the face frame of N-1 frame image is F0, N-1 frame figure As being expressed as I0, nth frame image is expressed as I1.At this point, the input of deep neural network model is (I0, I1), depth mind Output through network model is (DF, DS).Wherein, the output of the model is expressed as：DF=(xc1-xc0, yc1-yc0) and DS =S1-S0.
Wherein, x0=min (S0 (x)), y0=min (S0 (y))；X1=max (S0 (x)), y1=max (S0 (y))；W= X1-x0, h=y1-y0；Xc0=(x0+x1) * 0.5, yc0=(y0+y1) * 0.5；F0=(x0-w*0.2, y0-h*0.2, x1+w* 0.2, y1+h*0.2).
Then, S0 is normalized under F0 coordinate system, obtains S0=(S0-(xc0, yc0))/(w, h).
By the deep learning of deep neural network model, deep neural network model combine from the image of input (I0, I1 in), study has obtained DF (x, y) and DS (x, y) of the nth frame image relative to N-1 frame image.
After getting face frame offset DF (x, y) and face characteristic shape point offset DS (x, y), so that it may base The face frame and human face characteristic point of nth frame image are determined in DF (x, y) and DS (x, y).
It, can be in conjunction with the face frame of N-1 frame image during determining target face frame based on the first processing result The information of F0 determines the face frame of nth frame image.
When based on the first processing result tracking target face frame, an optional embodiment is：It is inclined in conjunction with face frame The coordinate of the face frame of shifting amount and the second picture frame determines the face frame of the first picture frame.
In embodiments of the present invention, can described in the formula F 1=F0+DF by the way of calculate the people of nth frame image Face frame, wherein F1 is expressed as the coordinate of the face frame of nth frame image.
During based on the first processing result tracking target human face characteristic point, another optional embodiment is： Firstly, intermediate human face characteristic point is determined in conjunction with the coordinate of human face characteristic point offset and the human face characteristic point of the second picture frame, In, intermediate human face characteristic point is located in the coordinate system of the face frame of the first picture frame；Finally, intermediate human face characteristic point is returned One change processing, obtains the human face characteristic point of the first picture frame, wherein the human face characteristic point of the first picture frame is located at the first picture frame Coordinate system in.
Specifically, the above process can be described by formula S 1=(S0+DS) * (w, h)+(xc1, yc1), wherein S1 The as human face characteristic point of nth frame image, xc1 and yc1 can be determined by DF=(xc1-xc0, yc1-yc0).
As can be seen from the above description, in embodiments of the present invention, the face frame of nth frame image is obtained first, then, it is determined that The human face characteristic point of nth frame image, and by deformed face shape point back projection into nth frame image.Pass through foregoing description It is found that in embodiments of the present invention, during the recognition of face information to nth frame image identifies, human face characteristic point It is mutually constrained with face frame, so that very accurate positioning can be realized in the very low model of complexity, it is fixed to greatly improve shape point The efficiency of position.Wherein, the face frame and human face characteristic point that nth frame image obtains input the mould into deep neural network model again In type, iteration obtains the face frame and human face characteristic point of N+1 frame image.
In an optional embodiment, marked using estimate face frame of the deep neural network model to the first picture frame The region that the face frame of the region of knowledge and the second picture frame having detected that is identified carries out processing and includes the following steps：
Characteristic point is instructed in step S21, acquisition, wherein instructs the face that characteristic point is the second picture frame identified special Sign point；
Step S22 will instruct characteristic point, the first picture frame to estimate region that face frame is identified and the second picture frame The region that the face frame having detected that is identified, which is input in deep neural network model, to be handled, and second processing knot is obtained Fruit, wherein second processing result includes：Face frame offset and first picture frame of first picture frame relative to the second picture frame Human face characteristic point；
Step S23 determines the face frame of the first picture frame based on face frame offset.
The mode that above-mentioned steps S21 to step S23 is described is to pass through depth corresponding to mode two in the case where mode two Spend target face frame and target human face characteristic point that neural network model tracks the first picture frame.
For example, by the nth frame image (that is, above-mentioned first picture frame) in video flowing, the N-1 frame image in video flowing (that is, above-mentioned second picture frame), and the human face characteristic point (that is, above-mentioned instruct characteristic point) of N-1 frame image identified It is input in deep neural network model and is analyzed and processed, the face frame offset and nth frame figure of processing output nth frame image The human face characteristic point (that is, target human face characteristic point in step S22) of picture.
After determining the face frame offset of nth frame image, so that it may using described in formula F 1=F0+DF The face frame of mode tracing computation nth frame image, wherein F1 is expressed as the face frame of nth frame image.
It should be noted that in embodiments of the present invention, the output of the deep neural network model is in addition to above-mentioned described The first processing result and second processing result except, further include confidence level, the confidence level is for determining the face frame that detects In whether include face.
In embodiments of the present invention, at use deep neural network model is to the first picture frame and the second picture frame While reason, the confidence level that deep neural network model is calculated based on the first picture frame can also be obtained；Then, based on setting Whether it includes face that reliability determines in the face frame of the first picture frame；Finally, stopping in the case where determining not including face Face in video flowing is identified.
Specifically, the confidence level and preset threshold that deep neural network model can be calculated based on the first picture frame It is compared, wherein the confidence level indicates probability of each face frame comprising face in the first picture frame.If comparison result is The confidence level is less than preset threshold (for example, 50%), shows not including face in corresponding face frame, at this point, stopping to view Face in frequency stream is tracked identification.
If including multiple face frames in the first picture frame, multiple confidence levels will be obtained, at this point it is possible to set each Compared with reliability continues with preset threshold.For example, comprising face frame 1 and face frame 2, if deep neural network model is calculated The confidence level of face frame 1 is 0.3, and the confidence level of face frame 2 is 0.6, wherein preset threshold 0.5.At this point, abandoning to face The tracking of frame 1, and continue to be tracked the face in face frame 2.If if the calculated people of deep neural network model The confidence level of face frame 1 is 0.3, and the confidence level of face frame 2 is 0.2, at this point, abandoning being tracked knowledge to the face in video flowing Not.
It should be noted that when being trained to deep neural network model, it is also necessary to another group of training data is established, The input data of the training data is multiple face block diagram pictures, and the output data of the training data is setting for each face block diagram picture Reliability.
In embodiments of the present invention, it while realizing the decision logic of confidence level, reduces the load of Face datection, and shared Parameter reduces calculation amount, so that real-time face positioning is easier to realize with tracking.
To sum up, the method for tracing of face characteristic information provided by the invention has following features：
Deep neural network model employed in the embodiment of the present invention is that network, the network can be straight end to end for one kind The human face characteristic point of every frame image in outputting video streams, and multiple model parameter sharings are connect, speed is faster；
Deep neural network model based on tracking copes with complicated face movement, attitudes vibration and expression shape change；
Based on the guidance of face tracking model, accurate Face detection is can be realized in simplified shape localization model, so that Real-time face feature point tracking can be suitable for camera, memory and the not high low-end platform of processor requirement.
Embodiment three：
The embodiment of the invention also provides a kind of follow-up mechanism of face characteristic information, the tracking of the face characteristic information is filled The method for tracing for being mainly used for executing face characteristic information provided by above content of the embodiment of the present invention is set, below to the present invention The follow-up mechanism for the face characteristic information that embodiment provides does specific introduction.
Fig. 3 is a kind of schematic diagram of the follow-up mechanism of face characteristic information according to an embodiment of the present invention, as shown in figure 3, The follow-up mechanism of the face characteristic information mainly includes：First acquisition unit 10 and first processing units 20, wherein：
First acquisition unit 10, for obtaining the first picture frame and processed second picture frame to be processed, the first figure It include estimating face frame in the first picture frame as frame and the second picture frame are at least a pair of of successive image frame in video flowing, the It include the face frame having detected that in two picture frames；
First processing units 20, for being identified using deep neural network model to the face frame of estimating of the first picture frame Region and the region that is identified of the face frame having detected that of the second picture frame handled, based on processing result tracking first The face characteristic information of picture frame, face characteristic information include face frame and/or human face characteristic point.
In embodiments of the present invention, current time the first picture frame and processed second image to be processed is obtained first Frame；Then, the region identified using estimated face frame of the deep neural network model to the first picture frame and the second image The region that the face frame of frame having detected that is identified is analyzed and processed, it will be able to is adjusted, be obtained to the first picture frame Target face frame and target human face characteristic point after first image framing control.Method in compared with the existing technology can only solve Face is mobile the case where varying less, in embodiments of the present invention, by deep neural network model can face it is mobile compared with In the case where for complexity, the human face characteristic point of each frame image is still accurately detected, and then solves face in the prior art Location tracking method is when coping with complicated face movement, the poor technical problem of the accuracy of recognition of face, to realize standard The technical effect that really facial image in video flowing is identified.
Optionally, first processing units include：First processing module, for using deep neural network model to the first figure At the region identified as the face frame having detected that for estimating region and the second picture frame that face frame is identified of frame Reason, to obtain the first processing result, wherein the first processing result includes：Face of first picture frame relative to the second picture frame Frame offset, human face characteristic point offset of first picture frame relative to the second picture frame；Tracing module, for being based at first Manage the face characteristic information that result tracks the first picture frame.
Optionally, deep neural network includes：First nerves network and nervus opticus network, wherein first nerves network Including the identical basic network of two structures, the network end-point of two basic networks respectively with the input terminal phase of nervus opticus network Connection, the input terminal of two basic networks is for inputting at least a pair of of successive image frame；First nerves network and nervus opticus net Network is by one or more convolutional layers, and one or more pond layers and one or more non-linear layers are according to predetermined connection type The network being formed by connecting, and the connection type of first nerves network and nervus opticus network is same or different.
Optionally, tracing module is used for：Is determined in conjunction with the coordinate of face frame offset and the face frame of the second picture frame The face frame of one picture frame.
Optionally, tracing module is also used to：In conjunction with the human face characteristic point of human face characteristic point offset and the second picture frame Coordinate determines intermediate human face characteristic point, wherein intermediate human face characteristic point is located in the coordinate system of the face frame of the first picture frame；It will Intermediate human face characteristic point is normalized, and obtains the human face characteristic point of the first picture frame, wherein the face of the first picture frame Characteristic point is located in the coordinate system of the first picture frame.
Optionally, which further includes：Second acquisition unit instructs characteristic point for obtaining, wherein instruct the characteristic point to be The human face characteristic point of the second picture frame detected；First processing units include：Second processing module, for feature will to be instructed The area that is identified of the face frame having detected that for estimating region and the second picture frame that face frame is identified of point, the first picture frame Domain is input in deep neural network model and is handled, and obtains second processing result, wherein second processing result includes：The One picture frame is relative to the face frame offset of the second picture frame and the human face characteristic point of the first picture frame；And determining module, For determining the face frame of the first picture frame based on face frame offset.
Optionally, which further includes：Third acquiring unit is used in use deep neural network model to the first image While frame and the second picture frame are handled, the confidence that deep neural network model is calculated based on the first picture frame is obtained Degree；Whether determination unit includes face in the face frame for determining the first picture frame based on confidence level；Stop unit is used for In the case where determining not including face, stopping tracks the face in video flowing.
Optionally, when the first picture frame is first frame image in video flowing, the second picture frame is the first picture frame At least one duplicating image frame, and comprising first passing through the face characteristic that Face datection algorithm detects in advance in the second picture frame Information；When the first picture frame is not first frame image, at least one before the second picture frame is the first picture frame is continuous Picture frame, and the face characteristic information in the second picture frame comprising having identified.
Optionally, the position of the face frame for estimating face frame and the second picture frame in the first picture frame is identical.
Optionally, which further includes：Acquisition unit, for acquiring the first face video for carrying the first change information Sequence, wherein the first change information includes at least one of：Motion change, expression shape change and attitudes vibration；First mark is single Member, for each frame in the first face video sequence face frame and human face characteristic point be labeled；First construction unit is used In based on the first face video sequence building multiple images pair after mark, wherein multiple images are to for depth nerve Network model carries out learning training.
Optionally, multiple images centering includes the first image and the second image, and the first image and the second image meet following At least one relationship：The adjacent picture frame of any two in first face video sequence；It is any in first face video sequence The picture frame of two intervals, one video frame；The adjacent picture frame of any two in second face video sequence, the second face Video sequence is the reversed video sequence of the first face video sequence；One, any two interval view in second face video sequence The picture frame of frequency frame.
Optionally, which further includes：4th acquiring unit, for obtaining the first face still image, wherein the first Face still image is in the case where different illumination conditions and/or collected personnel acquire in the case where different faces expression The image arrived；The second processing unit obtains the second face for handling the face performance objective in the first face still image Still image, wherein target processing includes at least one of：Translation scales, rotation, affine transformation, variation illumination, variation Expression blocks processing；Second mark unit, for the face frame in the first face still image and the second face still image It is labeled respectively with human face characteristic point；Second construction unit, for based on the first face still image and mark after mark The second face still image afterwards constructs multiple images pair, wherein multiple images are to for carrying out deep neural network model Learning training.
Example IV：
The present invention also provides a kind of identifying system of face characteristic information, Fig. 4 is one kind according to an embodiment of the present invention The schematic diagram of the identifying system of face characteristic information, including：Data generating unit 41, neural network configuration unit 42, nerve net Network training unit 43, video acquisition unit 44, Face datection unit 45, shape point tracking cell 46 and face judging unit 47.
Specifically, data generating unit 41, for collecting true and virtual image pair, to constitute training set of images.That is, The multiple images pair that two pass-through mode one of above-described embodiment and mode two are determined.
Neural network configuration unit 42, for constructing a deep neural network model.
Neural metwork training unit 43, image set training 42 structure of neural network configuration unit for being put in order based on collection The deep neural network model made, and after training is completed, deep neural network model is disposed.
Video acquisition unit 44 (for example, preposition or rear camera etc. of mobile phone), it is in real time or offline for acquiring face Video flowing.
Face datection unit 45, for face in first picture frame in video flowing to be processed position and size Carry out Face datection.
Shape point tracking cell 46, for determining figure to be identified in video flowing based on the deep neural network model disposed As the face frame and human face characteristic point of frame.
Face judging unit 47 is in the face frame for judging current image frame based on the deep neural network disposed No includes face, wherein if it is judged that not including face, then stops the tracking to face in video flowing.Specific implementation process As above, which is not described herein again.
In another embodiment of the present invention, a kind of tracing system of face characteristic information, the system packet are also provided It includes：Photographic device, processor and storage device；
The photographic device, for acquire face in real time or offline video stream；
Computer program is stored on the storage device, the computer program is executed when being run by the processor Method as described in above-described embodiment.
It is apparent to those skilled in the art that for convenience and simplicity of description, the system of foregoing description Specific work process, can refer to corresponding processes in the foregoing method embodiment, details are not described herein.
In another embodiment of the present invention, a kind of non-volatile program code that can be performed with processor is also provided Computer-readable medium, said program code makes the processor execute the method as described in preceding method embodiment.
In addition, in the description of the embodiment of the present invention unless specifically defined or limited otherwise, term " installation ", " phase Even ", " connection " shall be understood in a broad sense, for example, it may be being fixedly connected, may be a detachable connection, or be integrally connected；It can To be mechanical connection, it is also possible to be electrically connected；It can be directly connected, can also can be indirectly connected through an intermediary Connection inside two elements.For the ordinary skill in the art, above-mentioned term can be understood at this with concrete condition Concrete meaning in invention.
In the description of the present invention, it should be noted that term " center ", "upper", "lower", "left", "right", "vertical", The orientation or positional relationship of the instructions such as "horizontal", "inner", "outside" be based on the orientation or positional relationship shown in the drawings, merely to Convenient for description the present invention and simplify description, rather than the device or element of indication or suggestion meaning must have a particular orientation, It is constructed and operated in a specific orientation, therefore is not considered as limiting the invention.In addition, term " first ", " second ", " third " is used for descriptive purposes only and cannot be understood as indicating or suggesting relative importance.
It is apparent to those skilled in the art that for convenience and simplicity of description, the system of foregoing description, The specific work process of device and unit, can refer to corresponding processes in the foregoing method embodiment, and details are not described herein.
In several embodiments provided herein, it should be understood that disclosed systems, devices and methods, it can be with It realizes by another way.The apparatus embodiments described above are merely exemplary, for example, the division of the unit, Only a kind of logical function partition, there may be another division manner in actual implementation, in another example, multiple units or components can To combine or be desirably integrated into another system, or some features can be ignored or not executed.Another point, it is shown or beg for The mutual coupling, direct-coupling or communication connection of opinion can be through some communication interfaces, device or unit it is indirect Coupling or communication connection can be electrical property, mechanical or other forms.
The unit as illustrated by the separation member may or may not be physically separated, aobvious as unit The component shown may or may not be physical unit, it can and it is in one place, or may be distributed over multiple In network unit.It can select some or all of unit therein according to the actual needs to realize the mesh of this embodiment scheme 's.
In addition, each functional unit in each embodiment of the present invention can integrate in a first processing units, It can be each unit to physically exist alone, can also be integrated in one unit with two or more units.
It, can be with if the function is realized in the form of SFU software functional unit and when sold or used as an independent product It is stored in the executable non-volatile computer-readable storage medium of a processor.Based on this understanding, of the invention Technical solution substantially the part of the part that contributes to existing technology or the technical solution can be with software in other words The form of product embodies, which is stored in a storage medium, including some instructions use so that One computer equipment (can be personal computer, server or the network equipment etc.) executes each embodiment institute of the present invention State all or part of the steps of method.And storage medium above-mentioned includes：USB flash disk, mobile hard disk, read-only memory (ROM, Read- Only Memory), random access memory (RAM, Random AccessMemory), magnetic or disk etc. is various to deposit Store up the medium of program code.
Finally it should be noted that：Embodiment described above, only a specific embodiment of the invention, to illustrate the present invention Technical solution, rather than its limitations, scope of protection of the present invention is not limited thereto, although with reference to the foregoing embodiments to this hair It is bright to be described in detail, those skilled in the art should understand that：Anyone skilled in the art In the technical scope disclosed by the present invention, it can still modify to technical solution documented by previous embodiment or can be light It is readily conceivable that variation or equivalent replacement of some of the technical features；And these modifications, variation or replacement, do not make The essence of corresponding technical solution is detached from the spirit and scope of technical solution of the embodiment of the present invention, should all cover in protection of the invention Within the scope of.Therefore, the protection scope of the present invention shall be subject to the protection scope of the claims.

Claims (26)

1. a kind of method for tracing of face characteristic information, which is characterized in that including：
Obtain the first picture frame and processed second picture frame to be processed, the first image frame and second picture frame It include estimating face frame, second picture frame in the first image frame at least a pair of of successive image frame in video flowing In include the face frame that has detected that；
Using deep neural network model to estimating the region and described that face frame is identified described in the first image frame The region that the face frame having detected that of two picture frames is identified is handled, and tracks first figure based on processing result As the face characteristic information of frame, the face characteristic information includes face frame and/or human face characteristic point.
2. the method according to claim 1, wherein using deep neural network model to the first image frame The area that is identified of the face frame having detected that for estimating region and second picture frame that face frame is identified Domain is handled, and the face characteristic information based on processing result tracking the first image frame includes：
Using the deep neural network model to estimating the region and institute that face frame is identified described in the first image frame The region that the face frame having detected that of the second picture frame is identified is stated to be handled, to obtain the first processing result, In, first processing result includes：Face frame offset of the first image frame relative to second picture frame, it is described Human face characteristic point offset of first picture frame relative to second picture frame；
Face characteristic information based on first processing result tracking the first image frame.
3. according to the method described in claim 2, it is characterized in that, the deep neural network includes：
First nerves network and nervus opticus network, wherein the first nerves network includes the identical facilities network of two structures Network, the network end-point of two basic networks are connected with the input terminal of the nervus opticus network respectively, two bases The input terminal of plinth network is for inputting at least a pair of of successive image frame；
The first nerves network and the nervus opticus network are by one or more convolutional layers, one or more pond layers The network being formed by connecting with one or more non-linear layers according to predetermined connection type, and the first nerves network and described The connection type of two neural networks is same or different.
4. according to the method described in claim 2, it is characterized in that, tracking the first image based on first processing result The face frame of frame includes：
The people of the first image frame is determined in conjunction with the coordinate of the face frame offset and the face frame of second picture frame Face frame.
5. according to the method described in claim 4, it is characterized in that, tracking the first image based on first processing result The human face characteristic point of frame includes：
Determine that intermediate face is special in conjunction with the coordinate of the human face characteristic point offset and the human face characteristic point of second picture frame Levy point, wherein the intermediate human face characteristic point is located in the coordinate system of the face frame of the first image frame；
The intermediate human face characteristic point is normalized, the human face characteristic point of the first image frame is obtained, wherein institute The human face characteristic point for stating the first picture frame is located in the coordinate system of the first image frame.
6. the method according to claim 1, wherein
The method also includes：Characteristic point is instructed in acquisition, wherein described to instruct characteristic point be second image detected The human face characteristic point of frame；
It is described using deep neural network model to estimating the region and institute that face frame is identified described in the first image frame It states the region that the face frame having detected that of the second picture frame is identified and carries out processing and include：By it is described instruct characteristic point, The people having detected that for estimating region and second picture frame that face frame is identified of the first image frame The region that face frame is identified is input in the deep neural network model and is handled, and obtains second processing result, wherein institute Stating second processing result includes：The first image frame is relative to the face frame offset of second picture frame and described first The human face characteristic point of picture frame；And the face frame of the first image frame is determined based on the face frame offset.
7. the method according to claim 1, wherein in use deep neural network model to the first image While frame and second picture frame are handled, the method also includes：
Obtain the confidence level that the deep neural network model is calculated based on the first image frame；
Determine in the face frame of the first image frame whether include face based on the confidence level；
In the case where determining not including face, stopping tracks the face in the video flowing.
8. method according to any one of claim 1 to 7, which is characterized in that
When the first image frame is first frame image in the video flowing, second picture frame is first figure As at least one duplicating image frame of frame, and comprising first passing through what Face datection algorithm detected in advance in second picture frame Face characteristic information；
When the first image frame is not first frame image, before second picture frame is the first image frame At least one successive image frame.
9. method according to any one of claim 1 to 7, which is characterized in that estimate people in the first image frame Face frame is identical with the position of the face frame of second picture frame.
10. the method according to claim 1, wherein the method also includes：
Acquisition carries the first face video sequence of the first change information, wherein first change information include with down toward It is one of few：Motion change, expression shape change and attitudes vibration；
The face frame and human face characteristic point of each frame in the first face video sequence are labeled；
Multiple images pair are constructed based on the first face video sequence after mark, wherein described multiple images are to being used for Learning training is carried out to the deep neural network model.
11. according to the method described in claim 10, it is characterized in that, described multiple images centering includes the first image and second Image, the first image and second image meet following at least one relationship：
The adjacent picture frame of any two in the first face video sequence；
The picture frame of one video frame in any two interval in the first face video sequence；
The adjacent picture frame of any two in second face video sequence, the second face video sequence are described the first The reversed video sequence of face video sequence；
The picture frame of one video frame in any two interval in the second face video sequence.
12. method described in 0 or 11 according to claim 1, which is characterized in that the method also includes：
Obtain the first face still image, wherein the first face still image be in the case where different illumination conditions and/ Or the image that collected personnel collect in the case where different faces expression；
To the face performance objective processing in the first face still image, the second face still image is obtained, wherein described Target processing includes at least one of：Translation scales, rotation, affine transformation, and variation illumination changes expression, blocks processing；
To in the first face still image and the second face still image face frame and human face characteristic point respectively into Rower note；
Based on after mark the first face still image and the second face still image construct described multiple images pair, Wherein, described multiple images are to for carrying out learning training to the deep neural network model.
13. a kind of follow-up mechanism of face characteristic information, which is characterized in that including：
First acquisition unit, for obtaining the first picture frame and processed second picture frame to be processed, the first image Frame and second picture frame are at least a pair of of successive image frame in video flowing, include estimating face in the first image frame Frame includes the face frame detected in second picture frame；
First processing units, for being marked using deep neural network model to estimating face frame described in the first image frame The region that the face frame having detected that of the region of knowledge and second picture frame is identified is handled, based on processing knot Fruit tracks the face characteristic information of the first image frame, and the face characteristic information includes face frame and/or human face characteristic point.
14. device according to claim 13, which is characterized in that the first processing units include：
First processing module, for using the deep neural network model to estimating face frame described in the first image frame The region that the face frame having detected that in the region and second picture frame that are identified is identified is handled, to obtain First processing result, wherein first processing result includes：People of the first image frame relative to second picture frame Face frame offset, human face characteristic point offset of the first image frame relative to second picture frame；
Tracing module, for the face characteristic information based on first processing result tracking the first image frame.
15. device according to claim 14, which is characterized in that the deep neural network includes：
First nerves network and nervus opticus network, wherein the first nerves network includes the identical facilities network of two structures Network, the network end-point of two basic networks are connected with the input terminal of the nervus opticus network respectively, two bases The input terminal of plinth network is for inputting at least a pair of of successive image frame；
The first nerves network and the nervus opticus network are by one or more convolutional layers, one or more pond layers The network being formed by connecting with one or more non-linear layers according to predetermined connection type, and the first nerves network and described The connection type of two neural networks is same or different.
16. device according to claim 14, which is characterized in that tracing module is used for：
The people of the first image frame is determined in conjunction with the coordinate of the face frame offset and the face frame of second picture frame Face frame.
17. device according to claim 16, which is characterized in that the tracing module is also used to：
Determine that intermediate face is special in conjunction with the coordinate of the human face characteristic point offset and the human face characteristic point of second picture frame Levy point, wherein the intermediate human face characteristic point is located in the coordinate system of the face frame of the first image frame；
The intermediate human face characteristic point is normalized, the human face characteristic point of the first image frame is obtained, wherein institute The human face characteristic point for stating the first picture frame is located in the coordinate system of the first image frame.
18. device according to claim 13, which is characterized in that
Described device further includes：Second acquisition unit instructs characteristic point for obtaining, wherein described to instruct characteristic point to have examined The human face characteristic point for second picture frame measured；
The first processing units include：Second processing module, for by the institute for instructing characteristic point, the first image frame State that estimate the region that the face frame having detected that in region that face frame is identified and second picture frame is identified defeated Enter into the deep neural network model and handled, obtain second processing result, wherein the second processing result packet It includes：The first image frame is relative to the face frame offset of second picture frame and the face characteristic of the first image frame Point；And determining module, for determining the face frame of the first image frame based on the face frame offset.
19. device according to claim 13, which is characterized in that described device further includes：
Third acquiring unit, for using deep neural network model to the first image frame and second picture frame into While row processing, the confidence level that the deep neural network model is calculated based on the first image frame is obtained；
Whether determination unit includes face in the face frame for determining the first image frame based on the confidence level；
Stop unit, in the case where determining not including face, stopping to track the face in the video flowing.
20. device described in any one of 3 to 19 according to claim 1, which is characterized in that
When the first image frame is first frame image in the video flowing, second picture frame is first figure As at least one duplicating image frame of frame, and comprising first passing through what Face datection algorithm detected in advance in second picture frame Face characteristic information；
When the first image frame is not first frame image, before second picture frame is the first image frame At least one successive image frame, and comprising the face characteristic information that has identified in second picture frame.
21. device described in any one of 3 to 19 according to claim 1, which is characterized in that estimating in the first image frame Face frame is identical with the position of the face frame of second picture frame.
22. device according to claim 13, which is characterized in that described device further includes：
Acquisition unit, for acquiring the first face video sequence for carrying the first change information, wherein the first variation letter Breath includes at least one of：Motion change, expression shape change and attitudes vibration；
Mark unit, for each frame in the first face video sequence face frame and human face characteristic point be labeled；
First construction unit, for constructing multiple images pair based on the first face video sequence after mark, wherein institute Multiple images are stated to for carrying out learning training to the deep neural network model.
23. device according to claim 22, which is characterized in that described multiple images centering includes the first image and second Image, the first image and second image meet following at least one relationship：
The adjacent picture frame of any two in the first face video sequence；
The picture frame of one video frame in any two interval in the first face video sequence；
The adjacent picture frame of any two in second face video sequence, the second face video sequence are described the first The reversed video sequence of face video sequence；
The picture frame of one video frame in any two interval in the second face video sequence.
24. the device according to claim 22 or 23, which is characterized in that described device further includes：
4th acquiring unit, for obtaining the first face still image, wherein the first face still image is not share the same light In the case where according to condition and/or image that collected personnel collect in the case where different faces expression；
The second processing unit obtains the second face for handling the face performance objective in the first face still image Still image, wherein the target processing includes at least one of：Translation scales, rotation, affine transformation, variation illumination, Change expression, blocks processing；
Unit is marked, for the face frame and face in the first face still image and the second face still image Characteristic point is labeled respectively；
Second construction unit, for quiet based on second face after the first face still image and mark after mark State picture construction described multiple images pair, wherein described multiple images are to for the deep neural network model Practise training.
25. a kind of tracing system of face characteristic information, which is characterized in that the system comprises：It photographic device, processor and deposits Storage device；
The photographic device, for acquire face in real time or offline video stream；
Computer program is stored on the storage device, the computer program is executed when being run by the processor as weighed Benefit requires 1 to 12 described in any item methods.
26. a kind of computer-readable medium for the non-volatile program code that can be performed with processor, which is characterized in that described Program code makes the processor execute any method in the claim 1-12.
CN201710698369.8A 2017-08-15 2017-08-15 A kind of method for tracing of face characteristic information, apparatus and system Pending CN108875480A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201710698369.8A CN108875480A (en) 2017-08-15 2017-08-15 A kind of method for tracing of face characteristic information, apparatus and system

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201710698369.8A CN108875480A (en) 2017-08-15 2017-08-15 A kind of method for tracing of face characteristic information, apparatus and system

Publications (1)

Publication Number Publication Date
CN108875480A true CN108875480A (en) 2018-11-23

Family

ID=64325489

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201710698369.8A Pending CN108875480A (en) 2017-08-15 2017-08-15 A kind of method for tracing of face characteristic information, apparatus and system

Country Status (1)

Country Link
CN (1) CN108875480A (en)

Cited By (18)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109635693A (en) * 2018-12-03 2019-04-16 武汉烽火众智数字技术有限责任公司 A kind of face image detection method and device
CN109815840A (en) * 2018-12-29 2019-05-28 上海依图网络科技有限公司 A kind of method and device of determining identification information
CN109829380A (en) * 2018-12-28 2019-05-31 北京旷视科技有限公司 A kind of detection method, device, system and the storage medium of dog face characteristic point
CN110176024A (en) * 2019-05-21 2019-08-27 腾讯科技（深圳）有限公司 Method, apparatus, equipment and the storage medium that target is detected in video
CN110348394A (en) * 2019-07-15 2019-10-18 广东名阳信息科技有限公司 A method of detection video static object
CN110765952A (en) * 2019-10-24 2020-02-07 上海眼控科技股份有限公司 Vehicle illegal video processing method and device and computer equipment
CN110969110A (en) * 2019-11-28 2020-04-07 杭州趣维科技有限公司 Face tracking method and system based on deep learning
CN110991296A (en) * 2019-11-26 2020-04-10 腾讯科技（深圳）有限公司 Video annotation method and device, electronic equipment and computer-readable storage medium
CN111242081A (en) * 2020-01-19 2020-06-05 深圳云天励飞技术有限公司 Video detection method, target detection network training method, device and terminal equipment
WO2020151156A1 (en) * 2019-01-25 2020-07-30 平安科技（深圳）有限公司 Video stream playing method and system, computer apparatus and readable storage medium
CN111860154A (en) * 2020-06-12 2020-10-30 歌尔股份有限公司 Forehead detection method and device based on vision and electronic equipment
CN112016371A (en) * 2019-05-31 2020-12-01 广州市百果园信息技术有限公司 Face key point detection method, device, equipment and storage medium
CN112434678A (en) * 2021-01-27 2021-03-02 成都无糖信息技术有限公司 Face measurement feature space searching system and method based on artificial neural network
CN112767436A (en) * 2019-10-21 2021-05-07 深圳云天励飞技术有限公司 Face detection tracking method and device
CN112857746A (en) * 2020-12-29 2021-05-28 上海眼控科技股份有限公司 Tracking method and device of lamplight detector, electronic equipment and storage medium
CN113449194A (en) * 2021-07-14 2021-09-28 青岛科技大学 Expression recognition music recommendation system based on convolutional neural network
CN114596687A (en) * 2020-12-01 2022-06-07 咸瑞科技股份有限公司 In-vehicle driving monitoring system
CN114670189A (en) * 2020-12-24 2022-06-28 精工爱普生株式会社 Storage medium, and method and system for generating control program of robot

Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US8385610B2 (en) * 2006-08-11 2013-02-26 DigitalOptics Corporation Europe Limited Face tracking for controlling imaging parameters
CN103824054A (en) * 2014-02-17 2014-05-28 北京旷视科技有限公司 Cascaded depth neural network-based face attribute recognition method
CN104573622A (en) * 2013-10-09 2015-04-29 爱信精机株式会社 Face detection apparatus, face detection method, and program
CN105512627A (en) * 2015-12-03 2016-04-20 腾讯科技（深圳）有限公司 Key point positioning method and terminal
CN105844206A (en) * 2015-01-15 2016-08-10 北京市商汤科技开发有限公司 Identity authentication method and identity authentication device
CN106326853A (en) * 2016-08-19 2017-01-11 厦门美图之家科技有限公司 Human face tracking method and device
CN106845385A (en) * 2017-01-17 2017-06-13 腾讯科技（上海）有限公司 The method and apparatus of video frequency object tracking

Patent Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US8385610B2 (en) * 2006-08-11 2013-02-26 DigitalOptics Corporation Europe Limited Face tracking for controlling imaging parameters
CN104573622A (en) * 2013-10-09 2015-04-29 爱信精机株式会社 Face detection apparatus, face detection method, and program
CN103824054A (en) * 2014-02-17 2014-05-28 北京旷视科技有限公司 Cascaded depth neural network-based face attribute recognition method
CN105844206A (en) * 2015-01-15 2016-08-10 北京市商汤科技开发有限公司 Identity authentication method and identity authentication device
CN105512627A (en) * 2015-12-03 2016-04-20 腾讯科技（深圳）有限公司 Key point positioning method and terminal
CN106326853A (en) * 2016-08-19 2017-01-11 厦门美图之家科技有限公司 Human face tracking method and device
CN106845385A (en) * 2017-01-17 2017-06-13 腾讯科技（上海）有限公司 The method and apparatus of video frequency object tracking

Cited By (25)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109635693A (en) * 2018-12-03 2019-04-16 武汉烽火众智数字技术有限责任公司 A kind of face image detection method and device
CN109635693B (en) * 2018-12-03 2023-03-31 武汉烽火众智数字技术有限责任公司 Front face image detection method and device
CN109829380A (en) * 2018-12-28 2019-05-31 北京旷视科技有限公司 A kind of detection method, device, system and the storage medium of dog face characteristic point
CN109815840A (en) * 2018-12-29 2019-05-28 上海依图网络科技有限公司 A kind of method and device of determining identification information
WO2020151156A1 (en) * 2019-01-25 2020-07-30 平安科技（深圳）有限公司 Video stream playing method and system, computer apparatus and readable storage medium
US11900676B2 (en) 2019-05-21 2024-02-13 Tencent Technology (Shenzhen) Company Limited Method and apparatus for detecting target in video, computing device, and storage medium
WO2020233397A1 (en) * 2019-05-21 2020-11-26 腾讯科技（深圳）有限公司 Method and apparatus for detecting target in video, and computing device and storage medium
CN110176024A (en) * 2019-05-21 2019-08-27 腾讯科技（深圳）有限公司 Method, apparatus, equipment and the storage medium that target is detected in video
CN112016371B (en) * 2019-05-31 2022-01-14 广州市百果园信息技术有限公司 Face key point detection method, device, equipment and storage medium
CN112016371A (en) * 2019-05-31 2020-12-01 广州市百果园信息技术有限公司 Face key point detection method, device, equipment and storage medium
CN110348394A (en) * 2019-07-15 2019-10-18 广东名阳信息科技有限公司 A method of detection video static object
CN112767436A (en) * 2019-10-21 2021-05-07 深圳云天励飞技术有限公司 Face detection tracking method and device
CN110765952A (en) * 2019-10-24 2020-02-07 上海眼控科技股份有限公司 Vehicle illegal video processing method and device and computer equipment
CN110991296A (en) * 2019-11-26 2020-04-10 腾讯科技（深圳）有限公司 Video annotation method and device, electronic equipment and computer-readable storage medium
CN110991296B (en) * 2019-11-26 2023-04-07 腾讯科技（深圳）有限公司 Video annotation method and device, electronic equipment and computer-readable storage medium
CN110969110A (en) * 2019-11-28 2020-04-07 杭州趣维科技有限公司 Face tracking method and system based on deep learning
CN110969110B (en) * 2019-11-28 2023-05-02 杭州小影创新科技股份有限公司 Face tracking method and system based on deep learning
CN111242081A (en) * 2020-01-19 2020-06-05 深圳云天励飞技术有限公司 Video detection method, target detection network training method, device and terminal equipment
CN111860154A (en) * 2020-06-12 2020-10-30 歌尔股份有限公司 Forehead detection method and device based on vision and electronic equipment
CN114596687A (en) * 2020-12-01 2022-06-07 咸瑞科技股份有限公司 In-vehicle driving monitoring system
CN114670189A (en) * 2020-12-24 2022-06-28 精工爱普生株式会社 Storage medium, and method and system for generating control program of robot
CN114670189B (en) * 2020-12-24 2024-01-12 精工爱普生株式会社 Storage medium, and method and system for generating control program of robot
CN112857746A (en) * 2020-12-29 2021-05-28 上海眼控科技股份有限公司 Tracking method and device of lamplight detector, electronic equipment and storage medium
CN112434678A (en) * 2021-01-27 2021-03-02 成都无糖信息技术有限公司 Face measurement feature space searching system and method based on artificial neural network
CN113449194A (en) * 2021-07-14 2021-09-28 青岛科技大学 Expression recognition music recommendation system based on convolutional neural network

Similar Documents

Publication Publication Date Title
CN108875480A (en) A kind of method for tracing of face characteristic information, apparatus and system
Feng et al. Spatio-temporal fall event detection in complex scenes using attention guided LSTM
US11074461B2 (en) People flow estimation device, display control device, people flow estimation method, and recording medium
Camuñas-Mesa et al. Event-driven stereo visual tracking algorithm to solve object occlusion
Idrees et al. Tracking in dense crowds using prominence and neighborhood motion concurrence
CA2986406A1 (en) Techniques for assessing group level cognitive states
CN107545582A (en) Video multi-target tracking and device based on fuzzy logic
CN107122736A (en) A kind of human body based on deep learning is towards Forecasting Methodology and device
CN104599287B (en) Method for tracing object and device, object identifying method and device
Hwang et al. Lightweight 3d human pose estimation network training using teacher-student learning
JPWO2018051944A1 (en) People flow estimation device, people flow estimation method and program
US11417095B2 (en) Image recognition method and apparatus, electronic device, and readable storage medium using an update on body extraction parameter and alignment parameter
CN109871775A (en) A kind of the ice rink monitoring method and device of Behavior-based control detection
CN110298238A (en) Pedestrian's visual tracking method, model training method, device, equipment and storage medium
CN109598242A (en) A kind of novel biopsy method
CN110246160A (en) Detection method, device, equipment and the medium of video object
CN113642431A (en) Training method and device of target detection model, electronic equipment and storage medium
CN109685037A (en) A kind of real-time action recognition methods, device and electronic equipment
CN109389016B (en) Method and system for counting human heads
CN108812407A (en) Animal health status monitoring method, equipment and storage medium
CN108491766A (en) A kind of people counting method end to end based on depth decision forest
CN109872342A (en) A kind of method for tracking target under special scenes
CN110096938A (en) A kind for the treatment of method and apparatus of action behavior in video
CN110909625A (en) Computer vision basic network training, identifying and constructing method and device
KR101675692B1 (en) Method and apparatus for crowd behavior recognition based on structure learning

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
RJ01 Rejection of invention patent application after publication
RJ01 Rejection of invention patent application after publication

Application publication date: 20181123