CN110334627A - The device and system that the behavior of personnel is detected - Google Patents

The device and system that the behavior of personnel is detected Download PDF

Info

Publication number
CN110334627A
CN110334627A CN201910561189.4A CN201910561189A CN110334627A CN 110334627 A CN110334627 A CN 110334627A CN 201910561189 A CN201910561189 A CN 201910561189A CN 110334627 A CN110334627 A CN 110334627A
Authority
CN
China
Prior art keywords
state
convolutional layer
yawn
layer
combination
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN201910561189.4A
Other languages
Chinese (zh)
Inventor
马乾力
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Shenzhen Micro & Nano Integrated Circuit And System Application Institute
Original Assignee
Shenzhen Micro & Nano Integrated Circuit And System Application Institute
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Shenzhen Micro & Nano Integrated Circuit And System Application Institute filed Critical Shenzhen Micro & Nano Integrated Circuit And System Application Institute
Priority to CN201910561189.4A priority Critical patent/CN110334627A/en
Publication of CN110334627A publication Critical patent/CN110334627A/en
Pending legal-status Critical Current

Links

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00Pattern recognition
    • G06F18/20Analysing
    • G06F18/21Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G06F18/214Generating training patterns; Bootstrap methods, e.g. bagging or boosting
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06NCOMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00Computing arrangements based on biological models
    • G06N3/02Neural networks
    • G06N3/04Architecture, e.g. interconnection topology
    • G06N3/045Combinations of networks
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/16Human faces, e.g. facial parts, sketches or expressions
    • G06V40/172Classification, e.g. identification
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/20Movements or behaviour, e.g. gesture recognition

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • General Health & Medical Sciences (AREA)
  • Data Mining & Analysis (AREA)
  • Health & Medical Sciences (AREA)
  • Artificial Intelligence (AREA)
  • Evolutionary Computation (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • General Engineering & Computer Science (AREA)
  • Human Computer Interaction (AREA)
  • Multimedia (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Biomedical Technology (AREA)
  • Biophysics (AREA)
  • Computational Linguistics (AREA)
  • Evolutionary Biology (AREA)
  • Molecular Biology (AREA)
  • Computing Systems (AREA)
  • Mathematical Physics (AREA)
  • Software Systems (AREA)
  • Psychiatry (AREA)
  • Social Psychology (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Oral & Maxillofacial Surgery (AREA)
  • Image Analysis (AREA)

Abstract

The present invention provides the detection device that the behavior of a kind of couple of personnel is detected, it is configured to handle the video and/or image that include personnel, with in each frame image simultaneously testing staff eye state, mouth state, cigarette state and telephone state, wherein, the detection device carries out above-mentioned detection using the algorithm based on convolutional neural networks, the convolutional neural networks include at least one convolutional layer, at least one residual layers, one global pool layer and a full articulamentum, wherein, each convolutional layer is all disposed with one BN layers and one LeakyReLU layers.

Description

The device and system that the behavior of personnel is detected
Technical field
The present invention relates to field of visual inspection, the device and system detected more particularly, to the behavior to personnel.
Background technique
Current domestic and international anti-terrorism and public security situation are relative complex, therefore in the flow of personnel close quarters of main cities as The places such as iron, airport, bus, bus station are generally provided with Security Personnel.
And the Security Personnel for executing task is generally supplied to user by manpower outsourcing company or AnBao Co., Ltd, for effect Rate, which maximizes, to be considered, outsourcing company or AnBao Co., Ltd may be by rushing towards it after completing a certain work with a group of people again He executes task dispatching mode in place, uses Security Personnel to greatest extent, improves the income of company.But accompanying problem is that Because the case where Security Personnel's working time and intensity increase, and absorbed degree declines, weariness working increases, human factor causes A possibility that security loophole, greatly increases.
To avoid the occurrence of the above problem, Security Personnel user needs set of system to complete related work: passing through face Attendance is completed in identification, and ensures that operator on duty is consistent with shift report;Such as tired shape is carried out to operator on duty by deep learning The behavioral value of state, focus etc.;With for finding the problem of timely feedbacks to backstage and person liable, to carry out in real time Intervene.
Behavioral value is generally divided into fatigue detecting and violation motion detection.(on duty) fatigue refers to Security Personnel due to sleep Insufficient or decline long lasting for respond caused by duty, this decline shows that Security Personnel is sleepy, dozes, operates Make mistakes or completely lose security ability etc..The fatigue detecting of early stage carries out mainly from medical angle by medical device Physiological characteristic measurement, research fatigue, which is knocked, sleeps Producing reason and other risk factors, finds the side that can reduce this danger Method.Another method is to develop intelligent alarm system, prevents Security Personnel on duty in the case where knocking the state of sleeping.For example, using some Signal processing method obtains the fatigue datas such as Security Personnel's frequency of wink and duration, to judge whether Security Personnel beats It knocks and sleeps or fall asleep.Movement refers in Security Personnel during on duty in violation of rules and regulations, some influences made movement or behavior on duty, than It such as makes a phone call, plays mobile phone, smokes, whispering to each other.Violation motion detection solves generally by way of computer vision, needle Different algorithms is respectively adopted to different movements.Since various algorithms differ greatly, if being all deployed to mobile terminal too Huge redundancy generally can seldom be applied in mobile terminal.
Summary of the invention
The present invention at high cost, traditional images identification technology scheme redundancy complexity spy for existing sensor technology scheme Point proposes a kind of Computer Vision Detection scheme end to end neural network based, and it is on and off duty that integration solves Security Personnel Attendance, Postmatch, fatigue is on duty, smokes, many demands such as motion detection of making a phone call.The present invention is mentioned using convolutional neural networks Feature is taken, the state (eyes opening and closing, mouth are opened and closed, make a phone call, smoke) of each instantaneous time point target human body is detected, one The fatigue state and violation state on duty for interior comprehensive descision target body of fixing time, and it is on duty or make in Security Personnel's fatigue Alarm signal is issued when movement in violation of rules and regulations.
The present invention provides the device that the behavior of a kind of couple of personnel is detected, it is configured to the video including personnel And/or image is handled, in each frame image simultaneously testing staff eye state, mouth state, cigarette state and electricity Speech phase, wherein the detection device carries out above-mentioned detection, the convolutional neural networks using the algorithm based on convolutional neural networks Including at least one convolutional layer, at least one residual layers, a global pool layer and a full articulamentum, wherein each volume Lamination is all disposed with one BN layers and one LeakyReLU layers.
The present invention provides Security Personnel's identity neural network based and behavior personnel behavioral value to detect end to end System solves the problems such as traditional detection mode is at high cost or traditional images processing computing redundancy is complicated, passes through a system It is unified to export all judging results, it settles at one go.Compared to traditional image detection means, the deep learning that the present invention uses Mode is not necessarily to do the pretreatment in terms of image enhancement, and it is each to be adapted to that uneven illumination is even, target signature is diversified, background is complicated etc. Kind extreme environment, and support the incremental training for being directed to scene, in actual use, pass through regular manual intervention school appropriate Quasi- training sample, the accuracy rate being lifted under dedicated scene.The present invention includes simultaneously Face datection and behavioral value, Neng Gougao The analysis and early warning of progress related personnel's behavior of effect solve the two in traditional scheme and different system or product are needed The problems such as at, complex interfaces, response speed is slow.
By the way that the description of exemplary embodiment, other features of the invention be will be apparent referring to the drawings.
Detailed description of the invention
The attached drawing being included in the description and forms part of the description instantiates exemplary embodiment of the present invention, spy In terms of seeking peace, and the principle used to explain the present invention together with explanatory note.
Fig. 1 is the module diagram according to the detection system of one aspect of the invention.
Fig. 2 is the schematic diagram according to the Face datection process of one aspect of the invention.
Fig. 3 is the schematic diagram according to the detection process of one aspect of the invention.
Fig. 4 is the structural schematic diagram of the convolutional neural networks of one aspect according to the present invention.
Fig. 5 is the overhaul flow chart of the fatigue of one aspect or violation state according to the present invention.
Specific embodiment
The various exemplary embodiments of detailed description of the present invention, feature and aspect below with reference to accompanying drawings.It should be pointed out that removing Non- in addition to illustrate, the relative configuration of the component, digital representation and the numerical value that describe in these embodiments does not limit the present invention Range.It it should be pointed out that the following examples are not intended to limit the scope of the present invention recorded in claim, and is not these Whole combinations of feature described in embodiment are necessary to the present invention.
To solve the above problems, as shown in Figure 1, the present invention provides the system detected to human behavior, comprising: image Acquisition unit, behavioral value unit and data storage cell (or " memory module ").Preferably, detection system further includes face Detection unit and alarm unit.In addition to this, system of the invention further includes for the communication unit and interface with PERCOM peripheral communication Unit.
As an example, image acquisition units are preferably by device for obtg. high fidelity image, such as camera, camera Deng, acquisition fixed range (the high-incidence region of fire, such as forest, residential block, basement, overhead route, in city such as market et al. The big public domain etc. of flow) video and/or image, and by collected video and/or image for example, by RTSP agreement It is separately sent to detection unit and data storage cell.Wherein, camera includes but is not limited to simulate camera, IP camera Deng.Image acquisition units are configured to continuously record the behavior of related personnel, to form the view including multiple image Frequently.Image, which can be sent in the behavioral value unit or Face datection unit that will be described below, carries out feature extraction and fortune It calculates.Alternatively, meet predetermined condition frame image (for example, the 5th frame, the 10th frame, the 15th frame, the 20th frame, and so on etc.) can quilt It is transmitted in the behavioral value unit or Face datection unit that will be described below and carries out feature extraction and operation.
As an example, behavioral value unit handles the collected video of image acquisition units and/or image, according to Prestore model and network and carry out feature extraction, judge in monitoring area operator on duty with the presence or absence of fatigue, make a phone call, play mobile phone, The violations states such as smoking.Such as no exceptions, then continue to detect;It such as notes abnormalities, saves exception information and image/video Data, data storage cell is arrived in storage, and passes backstage early warning back by wired or wireless way;Behavioral value unit described in it Including algorithm and computing hardware two parts, hardware includes but is not limited to general GPU, embedded gpu, CPU, the dedicated core of artificial intelligence Piece etc..Algorithm part is described in detail below.
<processing of image>
As shown in figure 3, detection system (behavioral value unit) of the invention can extract video or figure by convolutional neural networks As in feature and classify, detect the position at each position of human body and state and cigarette and phone in image respectively Presence or absence and their respective positions.Then, the open and-shut mode of eye and mouth directly is obtained by convolutional neural networks, inhale Cigarette state (or smoking state), telephoning state (or talking state) etc..The continuous duration or number of each state can quilts It calculates.It can determine that personnel are in a state of fatigue if detecting successional eye closing or yawn, system can export pair The caution signal answered;If the case where detecting the presence of smoking and call, can determine that personnel are in violation state, at this time may be used Export corresponding warning message or directly as warning output.
<<convolutional neural networks architectural overview>>
Firstly, convolutional neural networks of the invention are made of the convolutional layer of 1*1 and 3*3 a series of as general introduction, each convolutional layer It afterwards all can be with one BN layers and one LeakyReLU layers.Meanwhile in order to solve due to convolutional network depth increase caused by property The problem of capable of declining, also introduces residual layers, finally in the end of convolutional neural networks addition global pool layer and entirely Articulamentum reuses softmax and classifies.Wherein, the step-length (strides) of convolution is defaulted as (1,1), " padding " ( Whether boundary pixel point is lost when convolution) being defaulted as same(, i.e. in convolution, the side length of padding default is 1, using 0 filling (before convolution algorithm, a circle 0 is mended around image, then does convolution again).Preferably, in network of the invention, Padding is preferably always by the way of same.
<<first embodiment of convolutional neural networks structure>>
Hereinafter, carrying out the specific structure for the convolutional neural networks that the present invention will be described in detail referring to Fig. 4.
Firstly, being input into the 1st convolutional layer by the video or a series of images of image acquisition units acquisition (" Convolutional "), to carry out preliminary feature extraction to image.Here, using the image having a size of 256*256 as showing Example is illustrated.It will be understood by those skilled in the art that the image of other sizes can also be inputted.Certainly, subsequent convolution knot Structure also can occur to change accordingly because of the size of input picture difference, such as increase or decrease convolutional layer, increase or decrease volume Product core size and quantity, increase or less residual layers of quantity change the position etc. of residual in a network.As Example, the convolution kernel size of the 1st convolutional layer are configured to 3*3, and convolution nuclear volume is arranged to 32.After layer processing, output Image having a size of 256*256.
Then, the image of the 1st convolutional layer output enters the 2nd convolutional layer, to carry out down-sampling, downscaled images ruler to image It is very little.The convolution kernel size of 2nd convolutional layer is configured to 3*3/2, and convolution nuclear volume is arranged to 64.2nd convolutional layer is by image Size reduction is 128*128 and exports.
Then, the image of the 2nd convolutional layer output enters the 3rd combination layer, to extract feature and increase network depth.3rd group It closes layer and includes the 31st convolutional layer, the 32nd convolutional layer and Residual layers.Wherein, the convolution kernel size of the 31st convolutional layer is configured to 1*1, convolution nuclear volume are arranged to 32;The convolution kernel size of 32nd convolutional layer is configured to 3*3, and convolution nuclear volume is arranged to 64.Through the layer processing after, still Output Size be 128*128 image.
Then, the image of the 3rd combination layer output enters the 4th convolutional layer, to carry out down-sampling, downscaled images ruler to image It is very little.The convolution kernel size of 4th convolutional layer is configured to 3*3/2, and convolution nuclear volume is arranged to 128.After layer processing, output Image having a size of 64*64.
Then, the image of the 4th convolutional layer output sequentially enters 2 (2x) the 5th combination layers, to extract feature and increase network Depth.Each 5th combination layer includes the 51st convolutional layer, the 52nd convolutional layer and Residual layers.Wherein, the volume of the 51st convolutional layer Product core size is configured to 1*1, and convolution nuclear volume is arranged to 64;The convolution kernel size of 52nd convolutional layer is configured to 3*3, volume Product nuclear volume is arranged to 128.After the processing of 2 the 5th combination layers, still Output Size be 64*64 image.
Next, the image of the 5th combination layer output enters the 6th convolutional layer, to carry out down-sampling, downscaled images ruler to image It is very little.The convolution kernel size of 6th convolutional layer is configured to 3*3/2, and convolution nuclear volume is arranged to 256.After layer processing, output Image having a size of 32*32.
Continue, the image of the 6th convolutional layer output sequentially enters 4 the 7th combination layers, to extract feature and increase network depth Degree.Each 7th combination layer includes the 71st convolutional layer, the 72nd convolutional layer and Residual layers.Wherein, the convolution kernel of the 71st convolutional layer Size is configured to 1*1, and convolution nuclear volume is arranged to 128;The convolution kernel size of 72nd convolutional layer is configured to 3*3, convolution Nuclear volume is arranged to 256.Through the layer processing after, still Output Size be 32*32 image.
Then, the data of the 7th combination layer output enter the 8th convolutional layer, to carry out down-sampling, downscaled images ruler to image It is very little.The convolution kernel size of 8th convolutional layer is configured to 3*3/2, and convolution nuclear volume is arranged to 512.After layer processing, output Image having a size of 16*6.
Subsequently, the image of the 8th convolutional layer output sequentially enters 4 the 9th combination layers, to extract feature and increase network depth Degree.Each 9th combination layer includes the 91st convolutional layer, the 92nd convolutional layer and Residual layers.Wherein, the convolution kernel of the 91st convolutional layer Size is configured to 1*1, and convolution nuclear volume is arranged to 256;The convolution kernel size of 92nd convolutional layer is configured to 3*3, convolution Nuclear volume is arranged to 512.Through the layer processing after, still Output Size be 16*16 image.
Continue, the image of the 9th combination layer output enters the 10th convolutional layer, to carry out down-sampling, downscaled images ruler to image It is very little.The convolution kernel size of 10th convolutional layer is configured to 3*3/2, and convolution nuclear volume is arranged to 1024.It is defeated after layer processing Out having a size of the image of 8*8.
Then, the image of the 10th convolutional layer output sequentially enters 2 the 11st combination layers, to extract feature and increase network depth Degree.Each 11st combination layer includes the 111st convolutional layer, the 112nd convolutional layer and Residual layers.Wherein, the volume of the 111st convolutional layer Product core size is configured to 1*1, and convolution nuclear volume is arranged to 512;The convolution kernel size of 112nd convolutional layer is configured to 3*3, Convolution nuclear volume is arranged to 1024.After layer processing, Output Size is the image of 8*8.
Then, the image of the 11st combination layer output sequentially enters global pool layer and full articulamentum, to classify.Complete Office's pond layer carries out global pool to obtained characteristic pattern 8*8, obtains a characteristic point.In full articulamentum, input dimension is used Two layers of neural network for being 2 for 256, output dimension handles the characteristic point, and wherein first layer neural network passes through TanH activation primitive, second layer neural network connect softmax function.
<<second embodiment of convolutional neural networks structure>>
If in order to reduce the parameter of network and calculation amount, on the one hand can the parameter appropriate that reduce network, on the other hand can be with A part of network layer is dismissed, and indistinctively influences neural network accuracy.For example, can be slightly on the basis of the first specific embodiment Deformation obtains the second specific embodiment.The parameter setting of the convolutional layer being identical with the first embodiment and combination layer will not described here And arrangement mode.The difference is that 2 points: first, second embodiment does not have the 7th for second embodiment and the first specific embodiment Combination layer, that is, the image of the 6th convolutional layer output is directly entered the 8th convolutional layer.Second, second embodiment the 11st combination layer it Afterwards, the 12nd convolutional layer and the 13rd combination layer are increased.
As an example, the convolution kernel size of the 12nd convolutional layer is configured to 3*3/2, convolution nuclear volume is arranged to 1024. After layer processing, Output Size is the image of 8*8.
As an example, the 13rd combination layer includes the 131st convolutional layer, the 132nd convolutional layer and Residual layers.Wherein, the 131st The convolution kernel size of convolutional layer is configured to 1*1, and convolution nuclear volume is arranged to 512;The convolution kernel size quilt of 132nd convolutional layer It is configured to 3*3, convolution nuclear volume is arranged to 1024.Through the layer processing after, still Output Size be 8*8 image.Later, the figure As entering global pool layer.
<<training methods and parameter of convolutional neural networks>>
Convolution kernel and full articulamentum in convolutional layer are carried out using the random numbers of Gaussian distribution that mean value is 0, standard deviation is 0.1 is obeyed Initialization, bias term are initialized using the uniform random number that section is [0,1] is obeyed.
In batch processing layer, momentum is set as 0.95, and constant is set as 0.01.
Using AdaDelta gradient descent algorithm training weight, batch processing is dimensioned to 64.
According to a certain percentage be arranged data training set, verifying collection and test set, after the training in 20 generations, every generation all into The test of row verifying collection, that as a result best generation training pattern can be saved and used for the test of test set, and result is The result entirely learnt.
Setting total data changes cycle of training as 100 generations, and in training, the positive negative sample ratio in training set is 10:1, often In generation training, the negative sample and whole positive samples for successively upsetting 20% are trained, until whole negative samples have trained completion one A cycle of training.
Above-mentioned experimental method and parameter are obtained on the basis of scientific research by many experiments.These methods and ginseng Number is very applicable for personnel environment of the present invention, especially in detection eye state, mouth state, smoking state It is especially pronounced when with talking state.
<judgement of fatigue or violation state>
Video or image pass through convolutional neural networks feature extraction, and divide an image into 11*11 sub-box in advance, with each Centered on grid, 5 Random candidate frames are randomly generated respectively, each candidate frame is divided in the full articulamentum of the last layer Class obtains classification results and the position of each candidate frame with this;In network training, following several states are drafted: in image Whether the position and open and-shut mode of personnel's eye or mouth, personnel lift the position that mobile phone is fitted in the state of face, mobile phone It sets, the position of cigarette;State judgement or alert if:
Fatigue state: the state that eye is in closure is eye strain characterization, if the continuous duration of eye closure is more than 3s (that is, eye closing scheduled duration, such as 3s, 5s, 10s etc.) is then assert and is in eye closing fatigue state;Mouth is in the state opened greatly i.e. For mouth fatigue characterization, if the big Zhang Lianxu duration of mouth is more than 1s(that is, yawn scheduled duration, such as 2s) and when yawn is set Between during detect in (for example, at least 60s, 100,120s etc.) 3 times or more, then assert and be in yawn fatigue state.It closes one's eyes Fatigue state and yawn fatigue state are referred to as fatigue state.
Smoking state: as long as detecting the presence of cigarette and cigarette is defined as smoking state close to mouth.If such State reaches 3 times or 4 times or 5 times (smoking pre-determined number) in (such as 5s, 10s, 20s etc.) during setting time of smoking, then It can be determined that personnel smoke in violation of rules and regulations.
Talking state: personnel, which lift mobile phone and are fitted in face, is defined as talking state, if the continuous example of the state As 5s or more (that is, call scheduled duration, such as 6s, 8s, 15s etc.) then can be determined that personnel are conversing in violation of rules and regulations.
As the detection example of eye closing fatigue state, during video flow detection, when detecting that eye is in for the first time When closed state, record the current time (such as 10:10:10) and/or record present frame number (that is, time or number, under Together).Later during continuous detection, if being consecutively detected this kind of state, continuous integration variable, if subsequent detection In continuous several frames or back to back next frame can't detect this kind of state, illustrate that eyes are opened, just interrupt statistics, during this section Variate-value (unit: frame) or time started to the time difference (unit: second (s)) terminated between record be exactly closed-eye state Continuous duration.The present invention sets the maximum continuous time (that is, eye closing scheduled duration) of eye closing as 3s.As known to those skilled in the art, The other times such as 4s, 5s are also set to maximum continuous time of closing one's eyes.
As an example, if 1-10 frame is not detected eye and is in closed state, initial time of closing one's eyes and the company of eye closing Continuous duration is disposed as 0.If detecting that eye is in closed state in the 11st frame, current time, for example, 10 are recorded: 10:10, and eye closing initial time is set by the time.If until the 20th frame detects that eye is constantly in closed state, Continuous updating current time is until the time of the 20th frame, for example, 10:10:11, then close one's eyes a length of 1s of consecutive hours, not up to closes one's eyes Scheduled duration not can determine that personnel are in eye closing fatigue state at this time.If detecting that eye is in the 21st frame opens state, Expression personnel are not in the state continuously closed one's eyes, and exclude the possibility of weariness working.Eye closing initial time and eye closing are continuous at this time Duration is updated to 0.Alternatively, if the 11st frame is to during the 20th frame, and in the 21st frame to continuous during the 60th frame Detect that eye is constantly in closed state in image, then the current time being recorded constantly refreshes (to be opened from the time of the 12nd frame Begin record, flushes to the time of the 60th frame always) to the time of the 60th frame, for example, 10:10:15, then continuous duration quilt of closing one's eyes It is updated to 5s.(the present embodiment be more than) eye closing scheduled duration (such as 3s), then assert at this point, reach since eye closing consecutive hours is long Personnel are in sleep or doze state, and triggering alarm unit makes a sound or the alarm of light, and controls associated picture or video It is transmitted to external equipment (such as console).After alarm, eye closing initial time and continuous duration of closing one's eyes are reset as 0, enter Next round detection.
As the detection example of yawn fatigue state, during video flow detection, when detecting that mouth is in for the first time When opening state greatly, records the current time (such as 10:10:10) and/or record the number of present frame.It continuously detects later In the process, if being consecutively detected this kind of state, then continuous integration variable, if continuous several frames or back to back in subsequent detection Next frame can't detect this kind of state, just interrupt statistics, the variate-value (unit: frame) or time started during this section to end Time difference (unit: second (s)) between record is exactly the continuous duration of yawn state.The present invention sets yawn maximum consecutive hours Between (that is, yawn scheduled duration) be 1s.As known to those skilled in the art, other times are also set to yawn maximum consecutive hours Between.
As an example, if 1-10 frame is not detected mouth and is in a state greatly, by yawn initial time and yawn Continuous duration is disposed as 0.If detecting that mouth is in a state greatly in the 11st frame, current time, for example, 10 are recorded: 10:10, and yawn initial time is set by the time.If still detecting that mouth is constantly in a shape greatly until the 15th frame State then records current time, for example, 10:10:10 ' 30, then a length of 0.5s of yawn consecutive hours.At this point, not up to yawn is pre- Timing is long (the present embodiment 1s), therefore not can determine that personnel are in yawn fatigue state.If until the since the 11st frame 40 frames detect that mouth is constantly in a state, the then current time being recorded greatly and constantly refreshes (since the time of the 12nd frame Record, is recorded always the time of the 60th frame) to the time of the 40th frame, for example, 10:10:12, then the continuous duration of yawn is by more It is newly 2s.(the present embodiment be more than) yawn scheduled duration (such as 1s), then yawn time at this point, reach since yawn consecutive hours is long Number is updated to 1 from 0, indicates that personnel have played a yawn at this time, while yawn initial time and the continuous duration of yawn are by more New is 0.Hereafter detection process continues, if record is worked as just detecting that mouth is again at a state greatly until the 100th frame Preceding time, for example, 10:10:16 then record the current time, and set yawn initial time for the time.If from the 100th Frame starts to detect that mouth is constantly in state greatly until the 140th frame, then the current time being recorded constantly refreshes (from the The time start recording of 101 frames, is recorded always the time of the 140th frame) to the time of the 140th frame, for example, 10:10:18, then The continuous duration of yawn is updated to 2s.(the present embodiment be more than) yawn scheduled duration at this point, reach since yawn consecutive hours is long (such as 1s), then yawn number is updated to 2 from 1, indicates that personnel have beaten 2 yawns, while yawn initial time and Kazakhstan at this time It owes continuous duration and is updated to 0.And so on.If in the yawn setting time that the yawn initial time of first time yawn starts In period (for example, 30s, 40s, 50s), detect that yawn number is that 4 times (or 5 times or 6 times) are greater than yawn pre-determined number 3 times, Then show that personnel are in yawn fatigue state.Triggering alarm unit makes a sound or the alarm of light at this time, and controls and scheme correlation Picture or video are transmitted to external equipment (such as console).After alarm, yawn initial time, the continuous duration of yawn and yawn number It is reset as 0, is detected into next round.
As smoking violation state detection example, during video flow detection, when detect cigarette exist and first Secondary when detecting cigarette close to mouth, then number of smoking is arranged to 1.Later during continuous detection, if detecting This kind of state, then continuous integration variable.The present invention sets smoking maximum times (that is, smoking pre-determined number) as 3 times.This field skill Art personnel know, 4 times, 5 other inferior numbers be also set to smoking pre-determined number.
As an example, setting 0 for smoking number if cigarette is not detected in 1-10 frame.If being detected in the 11st frame To cigarette and its close to mouth until the 20th frame cigarette is far from mouth, then number of smoking is incremented by 1.If being detected again in the 50th frame For cigarette close to mouth until the 60th frame cigarette is far from mouth, then number of smoking is incremented by 1 again becomes 2.And so on.If smoking Smoking number increases to 3 times or 4 times or 5 inferior in (for example, 10s, 20s, 60s, 90s, 120s etc.) during setting time, then table The person of leting others have a look at is in smoking violation state, triggers alarm unit at this time and makes a sound or the alarm of light, and control associated picture or Video is transmitted to external equipment (such as console).After alarm, smoking number is reset as 0, detects into next round.
As the detection example of call violation state, during video flow detection, when detecting that phone is in for the first time When near mouth, records the current time (such as 10:10:10) and/or record the number of present frame.It continuously detects later In the process, if being consecutively detected this kind of state, continuous integration variable, if in subsequent detection continuous several frames or it is back to back under As soon as frame can't detect this kind of state, statistics is interrupted, knot is recorded in the variate-value (unit: frame) or time started during this section Time difference (unit: second (s)) between beam recording is exactly the continuous duration of telephoning state.The present invention sets maximum of making a phone call Continuous time (that is, call scheduled duration) is 5s.As known to those skilled in the art, the other times such as 10s are also set to lead to Talk about maximum continuous time.
As an example, if 1-10 frame is not detected phone and is located near mouth, by call start time and call Continuous duration is disposed as 0.If detecting that phone is near mouth in the 11st frame, current time, for example, 10 are recorded: 10:10, and call start time is set by the time.If until the 20th frame detects that phone is constantly near mouth, Continuous updating current time is until the time of the 20th frame, for example, 10:10:11, then converse a length of 1s of consecutive hours, not up to converses Scheduled duration not can determine that personnel are in call violation state at this time.If detecting that phone leaves mouth in the 21st frame, determine Personnel are not in talking state, exclude the possibility of violation.Call start time and continuous duration of conversing are updated at this time 0.Alternatively, if the 11st frame is to during the 20th frame, and electricity is detected into the consecutive image during the 60th frame in the 21st frame Words are constantly near mouth, then the current time being recorded, which constantly refreshes, (from the time start recording of the 12nd frame, to be recorded always To the time of the 60th frame) to the time of the 50th frame, for example, 10:10:15, then continuous duration of conversing is updated to 5s.At this point, by Reach call scheduled duration (such as 5s) in call consecutive hours length, then assert that personnel are in call violation state, triggering is alarmed single Member makes a sound or the alarm of light, and controls associated picture or video being transmitted to external equipment (such as console).After alarm, Call start time and continuous duration of conversing are reset as 0, detect into next round.
The above embodiment of the present invention is exemplary only.The selection of video frame can be timing, and it is non-fixed to be also possible to When, herein with no restrictions.For example, can also be with every in preceding 100 frame every 10 milliseconds or 0.5 second 1 frame videos of interception 10 milliseconds intercept video for unit, and rear 100 frame is with every 5 milliseconds of interceptions video.For example, it may be possible to choose the 1st frame figure in 10:10:10 Picture, chooses the 10th frame image in 10:10:11, chooses the 100th frame image in 10:10:15.In addition, above-mentioned example is to record the time To judge duration, number etc..Those skilled in the art can also judge duration, secondary by way of recording the number of present frame Number etc., this is not as limitation of the present invention.
As an example, data storage cell is mainly used for storing video and image document that image acquisition units obtain, and The functions such as video playback and data backup are supported when needed.Data storage cell can plug into support ONVIF, PSIA, The third party's video camera and main brand video camera of RTSP agreement;Support IPv4, IPv6, HTTP, UPnP, NTP, SADP, SNMP, The network protocols such as PPPoE, DNS, FTP, ONVIF, PSIA;Support maximum 64 road network videos access;Support monitoring camera more Kind mainstream resolution ratio access;Support image locally playback and inquiry;It is single to support that the testing result that will test unit is sent to alarm Member;The testing result that will test unit is supported to be sent to the platform that other need to dock.
As an example, as shown in Fig. 2, Face datection unit is in the image and video counts for receiving image acquisition units transmission According to rear, according to prestoring model and network carries out feature extraction, and it is compared with the personnel characteristics recorded in database, confirms quilt The identity of attendance personnel;It is further compared with watch bill after personnel identity confirmation, confirms the operator on duty in current post It is consistent with watch bill.It can not verify such as attendance personnel identity or not be inconsistent with index table, save exception information and image/video money Material, and store and arrive data storage cell, and backstage early warning is passed back by transmission unit;Attendance unit described in it include algorithm and Computing hardware two parts, hardware include but is not limited to general GPU, embedded gpu, CPU, artificial intelligence special chip etc..
Face datection unit executes following steps: using the Face datection algorithm MTCNN of current superior performance;Of the invention MTCNN is made of first order network and second level network.Wherein, first order network can regard a random forest as, each The model of tree is the same.The picture size of all inputs is reduced, and utilizes face similar in P_Net detection image All areas.Through can probably remove 70% image unrelated with face as a result,.In the case where reducing by 70% image and subtracting While doubtful human face region quantity can be detected by having lacked first order network, human face region will not be omitted, it thus can be significantly Reduce algorithm complexity." 70% " this numerical value will be configured according to concrete scene, including will appear maximum number quantity, The pixel number etc. that face occupies.Wherein, second level network carries out again the face that first order network detects with R_Net true Recognize, to obtain the image for only including face.This equally greatly reduces false alarm rate.Unlike existing MTCNN algorithm, this hair Bright improved MTCNN does not need third level network, that is, does not need to detect face.
As an example, needing to carry out image different degrees of scaling, to every when first order network detects image One zoomed image is all detected with P_Net.Existing MTCNN is needed using 10 scale values or so, this allow for The human body contained in altimetric image is more, and has come out human body in the present invention, so only need to be using 4,3 or even less The scale value of quantity.Five points for there was only existing MTCNN by the doubtful human face region quantity that first order network extracts in this way One of or so, in addition the MTCNN in the present invention does not use third level network, improved MTCNN algorithm of the invention is being kept While performance, the speed of service is improved close to 8 times, is suitable for most embedded device.For example, if utilizing full will A64 Chip, then the runing time of single-frame images is promoted by 200ms or so to 20 to 30ms.
Further, first order network use little framework 4 layers of convolutional neural networks, input image size be 12 × 12 × 3.Second level network uses 4 layers of convolutional neural networks of little framework, and input image size is 24 × 24 × 3.As an example, convolution The training method and parameter of neural network, specific as follows:
Convolution kernel and full articulamentum in convolutional layer are carried out using the random numbers of Gaussian distribution that mean value is 0, standard deviation is 0.1 is obeyed Initialization.
Using stochastic gradient descent algorithm training weight, batch processing is dimensioned to 128, when training only to 70% loss compared with Small data carry out backpropagation.
Training sample has used 15000 front face figures, and 50000 are free of facial image and 50000 part faces Image, positive sample in training set, negative sample, the ratio of three kinds of samples of part sample are 3:1:1, and front face image is from multiple The face image set of open source and image set with Face detection coordinate, partial face image largely pass through front face image Interception obtain, all data are cut to 12 × 12 × 3 and 24 × 24 × 3 two kinds of sizes respectively.
According to a certain percentage be arranged data training set, verifying collection and test set, after the training in 10 generations, every generation all into The test of row verifying collection.Those skilled in the art can be arranged according to known instruction the number of iterations and training set, verifying collection and The ratio of test set.
The setting total data repetitive exercise period was 1000 generations.More (1200 generations, 2000 generations) or less (500 generations, 800 Generation) the repetitive exercise period it is also possible.
It will be understood by those skilled in the art that the data occurred in above-mentioned training method and parameter are not limiting.This Field technical staff can use different picture size, training sample and iteration cycle according to the difference of application scenarios, with Guarantee that arithmetic speed and precision are optimized simultaneously.
While property retention, the speed of service is improved close to 8 times simplified Face datection algorithm, is suitable for big portion The embedded device divided.
As an example, alarm unit includes field mode and background mode.After receiving the warning message of detection unit, Early warning is issued at the scene that image acquisition units are arranged by different sensors such as sound, light, electricity, surrounding is reminded to have alert. Alternatively, alarm unit can also send live signal to the contingency management department early warning platform of system docking, the information content includes: Camera number, camera position, alert time of origin, alert type, live image/video etc..Help contingency management department Quickly judgement and decision shorten the response time.
It is illustrated in figure 5 the schematic flow diagram of the method for the invention detected to human behavior.In step S501, Eye state, mouth state, cigarette state and the talking state of acquisition personnel.Then, in step S502, while eye is detected Whether state meets closed-eye state, whether mouth state meets yawn state, whether cigarette state meets smoking state and phone Whether state meets talking state, and further judges whether there is personnel and be in eye closing fatigue state, yawn fatigue state, inhales Cigarette violation state and call violation state.Finally, in step S503, if any of the above-described state meets fatigue or violation state, Then issue alarm.Preferably, acquisition show personnel be in fatigue or violation state video or picture can be sent to it is outer Portion's equipment, such as Central Control Room or security room.
The present invention provides human behaviors neural network based to detect detection mode end to end, solves traditional detection The problems such as mode is at high cost or traditional images processing computing redundancy is complicated, all judgement knots are uniformly exported by a kind of network Fruit settles at one go.Compared to traditional image detection means, the present invention using deep learning by the way of without doing image enhancement side The pretreatment in face is adapted to the various extreme environments such as uneven illumination is even, target signature is diversified, background is complicated, and supports needle To the incremental training of scene, in actual use, training sample is calibrated by periodically manual intervention appropriate, is lifted at specially With the accuracy rate under scene.
Can be with one BN layers and a LeakyReLU after each convolutional layer, and introduce residual layers of solution network Because of degradation problem caused by depth;Training method and parameter be also by a large number of experimental results show that preferable skill And parameter.Application: application of the convolution algorithm in human behavior detection, end-to-end direct solution test problems simplify and pass Complexity, the detection means of redundancy of system.
Detection device provided by the present invention and system are described in detail above.Specific case used herein Principle and implementation of the present invention are described, the above embodiments are only used to help understand side of the invention Method and its core concept.It should be pointed out that for those skilled in the art, not departing from the principle of the invention Under the premise of, it can be with several improvements and modifications are made to the present invention, these improvement and modification also fall into the claims in the present invention In protection scope.

Claims (10)

1. the device that the behavior of a kind of couple of personnel is detected, which is characterized in that be configured to include personnel video and/or Image is handled, in each frame image simultaneously testing staff eye state, mouth state, cigarette state and phone shape State, wherein the detection device carries out above-mentioned detection using the algorithm based on convolutional neural networks, which includes At least one convolutional layer, at least one residual layers, a global pool layer and a full articulamentum, wherein each convolutional layer All it is disposed with one BN layers and one LeakyReLU layers.
2. the apparatus according to claim 1, which is characterized in that detection device is configured to:
Whether detection eye state meets closed-eye state, if detecting, eye state does not meet closed-eye state, originates closing one's eyes Time and continuous duration of closing one's eyes are disposed as 0;If detecting that eye state meets closed-eye state for the first time, current time is set It is set to eye closing initial time;If previous frame image and detecting eye state symbol with the continuous next frame image of previous frame image Closed-eye state is closed, then is set as closing one's eyes by duration continuous between the current time and eye closing initial time of the next frame image Continuous duration;
Whether detection mouth state meets yawn state, if detecting, mouth state does not meet yawn state, and yawn is originated Time, the continuous duration of yawn and yawn number are disposed as 0;It, will if detecting that mouth state meets yawn state for the first time Current time is set as yawn initial time;If previous frame image and being detected with the continuous next frame image of the previous frame image Meet yawn state to mouth state, then when will be continuous between the current time and yawn initial time of the next frame image Length is set as the continuous duration of yawn;
Whether detection cigarette state meets smoking state, if detecting, cigarette state does not meet smoking state, and will smoke number It is initialized as 0;If detecting, cigarette state meets smoking state, smoking number is added 1, and smoking setting time hereafter During will smoking number be classified as 0;
Whether detection telephone state meets talking state, if detecting, telephone state does not meet talking state, by call starting Time and continuous duration of conversing are disposed as 0;If detecting that telephone state meets talking state for the first time, current time is recorded For call start time;If previous frame image and with the continuous next frame image of the previous frame image detect telephone state accord with Talking state is closed, then is set as conversing by duration continuous between the current time and call start time of the next frame image Continuous duration.
3. the apparatus according to claim 1, which is characterized in that convolutional neural networks be construed as including sequential connection as Lower layer:
1st convolutional layer, image are directly input into the 1st convolutional layer,
2nd convolutional layer,
1 the 3rd combination layer comprising the 31st convolutional layer, the 32nd convolutional layer and Residual layers,
4th convolutional layer,
2 the 5th combination layers, each 5th combination layer include the 51st convolutional layer, the 52nd convolutional layer and Residual layers,
6th convolutional layer,
4 the 7th combination layers, each 7th combination layer include the 71st convolutional layer, the 72nd convolutional layer and Residual layers,
8th convolutional layer,
4 the 9th combination layers, each 9th combination layer include the 91st convolutional layer, the 92nd convolutional layer and Residual layers,
10th convolutional layer,
2 the 11st combination layers, each 11st combination layer include the 111st convolutional layer, the 112nd convolutional layer and Residual layers,
Global pool layer, and
Full articulamentum.
4. the apparatus according to claim 1, which is characterized in that convolutional neural networks be construed as including sequential connection as Lower layer:
1st convolutional layer, image are directly input into the 1st convolutional layer,
2nd convolutional layer,
1 the 3rd combination layer comprising the 31st convolutional layer, the 32nd convolutional layer and Residual layers,
4th convolutional layer,
2 the 5th combination layers, each 5th combination layer include the 51st convolutional layer, the 52nd convolutional layer and Residual layers,
6th convolutional layer,
8th convolutional layer,
4 the 9th combination layers, each 9th combination layer include the 91st convolutional layer, the 92nd convolutional layer and Residual layers,
10th convolutional layer,
2 the 11st combination layers, each 11st combination layer include the 111st convolutional layer, the 112nd convolutional layer and Residual layers,
12nd convolutional layer,
1 the 13rd combination layer, each 13rd combination layer include the 131st convolutional layer, 132 convolutional layers and Residual layers,
Global pool layer, and
Full articulamentum.
5. device according to claim 3 or 4, which is characterized in that
The convolution kernel size of 1st convolutional layer is 3*3, and convolution nuclear volume is 32, and output picture size is 256*256;
The convolution kernel size of 2nd convolutional layer is 3*3/2, and convolution nuclear volume is 64, and output picture size is 128*128;
The convolution kernel size of 31st convolutional layer is 1*1, and convolution nuclear volume is that the convolution kernel size of the 32, the 32nd convolutional layer is 3*3, volume Product nuclear volume is that the 64, the 3rd combination layer output picture size is 128*128;
The convolution kernel size of 4th convolutional layer is 3*3/2, and convolution nuclear volume is 128, and output picture size is 64*64;
The convolution kernel size of 51st convolutional layer is 1*1, and convolution nuclear volume is that the convolution kernel size of the 64, the 52nd convolutional layer is 3*3, volume Product nuclear volume is that the 128, the 5th combination layer output picture size is 64*64;
The convolution kernel size of 6th convolutional layer is 3*3/2, and convolution nuclear volume is 256, and output picture size is 32*32;
The convolution kernel size of 71st convolutional layer is 1*1, and convolution nuclear volume is that the convolution kernel size of the 128, the 72nd convolutional layer is 3*3, Convolution nuclear volume is that the 256, the 7th combination layer output picture size is 32*32;
The convolution kernel size of 8th convolutional layer is 3*3/2, and convolution nuclear volume is 512, and output picture size is 16*16;
The convolution kernel size of 91st convolutional layer is 1*1, and convolution nuclear volume is that the convolution kernel size of the 256, the 92nd convolutional layer is 3*3, Convolution nuclear volume is that the 512, the 9th combination layer output picture size is 16*16;
The convolution kernel size of 10th convolutional layer is 3*3/2, and convolution nuclear volume is 1024, and output picture size is 8*8;
The convolution kernel size of 111st convolutional layer is 1*1, and convolution nuclear volume is that the convolution kernel size of the 512, the 112nd convolutional layer is 3* 3, convolution nuclear volume is that the 1024, the 11st combination layer output picture size is 8*8;
The convolution kernel size of 12nd convolutional layer is 3*3/2, and convolution nuclear volume is 1024, and output picture size is 8*8;
The convolution kernel size of 131st convolutional layer is 1*1, and convolution nuclear volume is that the convolution kernel size of the 512, the 132nd convolutional layer is 3* 3, convolution nuclear volume is that the 1024, the 131st combination layer output picture size is 8*8.
6. the system that a kind of pair of human behavior is detected characterized by comprising
Image acquisition units are used to obtain video/image of the personnel under behavior, with simultaneously obtain personnel eye state, The multiple image of each of mouth state, cigarette state and the telephone state of personnel;
Behavioral value unit is detection device as described in any one of the preceding claims,
Judging unit is configured to judge whether personnel are in eye closing fatigue state, yawn fatigue state, smoking violation state With call violation state,
Data storage cell is used to save the video and/or image and behavioral value unit of image acquisition units acquisitions Testing result.
7. system according to claim 6, which is characterized in that judging unit is configured to be performed simultaneously following fatigue of closing one's eyes State judgement, the judgement of yawn fatigue state, the judgement of smoking violation state and call violation state judgment step:
In the judgement of eye closing fatigue state, whether the continuous duration that judges to close one's eyes reaches eye closing scheduled duration, if so, personnel are in Eye closing fatigue state;
In the judgement of yawn fatigue state, judge whether the continuous duration of yawn reaches yawn scheduled duration, if the continuous duration of yawn Reach yawn scheduled duration, then yawn number adds 1, if yawn number reaches yawn predetermined time during yawn setting time Number, then personnel are in yawn fatigue state;
In smoking violation state judgement, judge whether smoking number reaches smoking predetermined time during smoking setting time Number, if so, personnel are in smoking violation state;
In call violation state judgement, whether the continuous duration that judges to converse reaches call scheduled duration, if so, personnel are in Call violation state.
8. system according to claim 7, which is characterized in that eye closing fatigue state refers to the state of eye closure, yawn Fatigue state refers to that mouth is opened greatly, and smoking violation state refers to cigarette close to mouth, and violation state of conversing refers to that mobile phone is fitted in Near face;
Eye closing scheduled duration is arranged at least 3 seconds,
Yawn scheduled duration is arranged at least 1 second,
Yawn pre-determined number is arranged at least 3 times,
It is at least 30 seconds during yawn setting time,
Smoking pre-determined number is arranged at least 3 times,
It is at least 10 seconds during setting time of smoking,
Call scheduled duration is arranged at least 5 seconds.
9. system according to claim 6, which is characterized in that further include Face datection unit, be configured to judgement personnel Identity.
10. system according to claim 6, which is characterized in that further include alarm unit, in the inspection of behavior detection unit Surveying result is the scene sending early warning arranged by sound, light or electric transducer in behavior detection unit when needing early warning, or Warning information is sent out.
CN201910561189.4A 2019-06-26 2019-06-26 The device and system that the behavior of personnel is detected Pending CN110334627A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201910561189.4A CN110334627A (en) 2019-06-26 2019-06-26 The device and system that the behavior of personnel is detected

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201910561189.4A CN110334627A (en) 2019-06-26 2019-06-26 The device and system that the behavior of personnel is detected

Publications (1)

Publication Number Publication Date
CN110334627A true CN110334627A (en) 2019-10-15

Family

ID=68142393

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201910561189.4A Pending CN110334627A (en) 2019-06-26 2019-06-26 The device and system that the behavior of personnel is detected

Country Status (1)

Country Link
CN (1) CN110334627A (en)

Cited By (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111027390A (en) * 2019-11-11 2020-04-17 北京三快在线科技有限公司 Object class detection method and device, electronic equipment and storage medium
CN111611966A (en) * 2020-05-29 2020-09-01 北京每日优鲜电子商务有限公司 Target person detection method, device, equipment and storage medium
CN111970489A (en) * 2020-08-05 2020-11-20 北京必可测科技股份有限公司 Intelligent monitoring management method and system based on man-machine two-way
CN112836556A (en) * 2019-11-25 2021-05-25 西安诺瓦星云科技股份有限公司 Behavior recognition device and method and behavior recognition system based on display screen
CN113194293A (en) * 2021-04-30 2021-07-30 重庆天智慧启科技有限公司 Personnel monitoring system
CN113392800A (en) * 2021-06-30 2021-09-14 浙江商汤科技开发有限公司 Behavior detection method and device, computer equipment and storage medium

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107657236A (en) * 2017-09-29 2018-02-02 厦门知晓物联技术服务有限公司 Vehicle security drive method for early warning and vehicle-mounted early warning system
KR20180066610A (en) * 2016-12-09 2018-06-19 동국대학교 산학협력단 Device and method for classifying open and close eyes based on convolutional neural network
CN108491858A (en) * 2018-02-11 2018-09-04 南京邮电大学 Method for detecting fatigue driving based on convolutional neural networks and system
CN108596087A (en) * 2018-04-23 2018-09-28 合肥湛达智能科技有限公司 A kind of driving fatigue degree detecting regression model based on dual network result
CN108764034A (en) * 2018-04-18 2018-11-06 浙江零跑科技有限公司 A kind of driving behavior method for early warning of diverting attention based on driver's cabin near infrared camera
CN108960065A (en) * 2018-06-01 2018-12-07 浙江零跑科技有限公司 A kind of driving behavior detection method of view-based access control model

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR20180066610A (en) * 2016-12-09 2018-06-19 동국대학교 산학협력단 Device and method for classifying open and close eyes based on convolutional neural network
CN107657236A (en) * 2017-09-29 2018-02-02 厦门知晓物联技术服务有限公司 Vehicle security drive method for early warning and vehicle-mounted early warning system
CN108491858A (en) * 2018-02-11 2018-09-04 南京邮电大学 Method for detecting fatigue driving based on convolutional neural networks and system
CN108764034A (en) * 2018-04-18 2018-11-06 浙江零跑科技有限公司 A kind of driving behavior method for early warning of diverting attention based on driver's cabin near infrared camera
CN108596087A (en) * 2018-04-23 2018-09-28 合肥湛达智能科技有限公司 A kind of driving fatigue degree detecting regression model based on dual network result
CN108960065A (en) * 2018-06-01 2018-12-07 浙江零跑科技有限公司 A kind of driving behavior detection method of view-based access control model

Non-Patent Citations (4)

* Cited by examiner, † Cited by third party
Title
JOSEPH REDMON 等: "YOLOv3:An Incremental Improvement", 《ARXIV》, pages 1 - 6 *
YINGYU JI 等: "Fatigue State Detection Based on Multi-Index Fusion and State Recognition Network", 《IEEE ACCESS》, pages 64136 - 64147 *
刘小双;方志军;刘翔;高永彬;张祥祥;: "基于眼口状态的疲劳检测系统", 《传感器与微系统》, vol. 37, no. 10, pages 108 - 110 *
刘炜煌;钱锦浩;姚增伟;焦新涛;潘家辉;: "基于多面部特征融合的驾驶员疲劳检测算法", 《计算机系统应用》, vol. 27, no. 10, pages 177 - 182 *

Cited By (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111027390A (en) * 2019-11-11 2020-04-17 北京三快在线科技有限公司 Object class detection method and device, electronic equipment and storage medium
CN111027390B (en) * 2019-11-11 2023-10-10 北京三快在线科技有限公司 Object class detection method and device, electronic equipment and storage medium
CN112836556A (en) * 2019-11-25 2021-05-25 西安诺瓦星云科技股份有限公司 Behavior recognition device and method and behavior recognition system based on display screen
CN111611966A (en) * 2020-05-29 2020-09-01 北京每日优鲜电子商务有限公司 Target person detection method, device, equipment and storage medium
CN111970489A (en) * 2020-08-05 2020-11-20 北京必可测科技股份有限公司 Intelligent monitoring management method and system based on man-machine two-way
CN113194293A (en) * 2021-04-30 2021-07-30 重庆天智慧启科技有限公司 Personnel monitoring system
CN113194293B (en) * 2021-04-30 2022-09-30 重庆天智慧启科技有限公司 Personnel monitoring system
CN113392800A (en) * 2021-06-30 2021-09-14 浙江商汤科技开发有限公司 Behavior detection method and device, computer equipment and storage medium

Similar Documents

Publication Publication Date Title
CN110334627A (en) The device and system that the behavior of personnel is detected
CN109769099B (en) Method and device for detecting abnormality of call person
CN106951867B (en) Face identification method, device, system and equipment based on convolutional neural networks
CN109886209A (en) Anomaly detection method and device, mobile unit
CN110674790B (en) Abnormal scene processing method and system in video monitoring
CN109948447B (en) Character network relation discovery and evolution presentation method based on video image recognition
CN109697416A (en) A kind of video data handling procedure and relevant apparatus
CN101753992A (en) Multi-mode intelligent monitoring system and method
CN110309760A (en) The method that the driving behavior of driver is detected
CN107220633A (en) A kind of intelligent mobile enforcement system and method
CN108534832A (en) A kind of civil engineering work monitoring system
CN110135476A (en) A kind of detection method of personal safety equipment, device, equipment and system
CN110321780A (en) Exception based on spatiotemporal motion characteristic falls down behavioral value method
CN108109331A (en) Monitoring method and monitoring system
CN113411542A (en) Intelligent working condition monitoring equipment
CN112541393A (en) Transformer substation personnel detection method and device based on deep learning
CN110442742A (en) Retrieve method and device, processor, electronic equipment and the storage medium of image
CN109063977A (en) A kind of no-induction transaction risk monitoring method and device
CN106973039A (en) A kind of network security situation awareness model training method and device based on information fusion technology
CN110263680A (en) Image processing method, device and system and storage medium
CN110427824A (en) A kind of the security protection automatic test approach and system of artificial intelligence virtual scene
CN111325133A (en) Image processing system based on artificial intelligence recognition
CN107495971A (en) Morbidity&#39;s alarm medical system and its detection method based on skeleton identification
CN108776452A (en) A kind of special equipment scene maintenance monitoring method and system
CN110807117A (en) User relationship prediction method and device and computer readable storage medium

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination