CN112001320B - Gate detection method based on video - Google Patents

Gate detection method based on video Download PDF

Info

Publication number
CN112001320B
CN112001320B CN202010864191.1A CN202010864191A CN112001320B CN 112001320 B CN112001320 B CN 112001320B CN 202010864191 A CN202010864191 A CN 202010864191A CN 112001320 B CN112001320 B CN 112001320B
Authority
CN
China
Prior art keywords
gate
top line
detected
detection
video
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202010864191.1A
Other languages
Chinese (zh)
Other versions
CN112001320A (en
Inventor
薛超
高旭麟
刘琰
陈澎祥
孙雅彬
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Tiandy Technologies Co Ltd
Original Assignee
Tiandy Technologies Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Tiandy Technologies Co Ltd filed Critical Tiandy Technologies Co Ltd
Priority to CN202010864191.1A priority Critical patent/CN112001320B/en
Publication of CN112001320A publication Critical patent/CN112001320A/en
Application granted granted Critical
Publication of CN112001320B publication Critical patent/CN112001320B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00Scenes; Scene-specific elements
    • G06V20/40Scenes; Scene-specific elements in video content
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06NCOMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00Computing arrangements based on biological models
    • G06N3/02Neural networks
    • G06N3/04Architecture, e.g. interconnection topology
    • G06N3/045Combinations of networks
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06NCOMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00Computing arrangements based on biological models
    • G06N3/02Neural networks
    • G06N3/08Learning methods
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00Scenes; Scene-specific elements
    • G06V20/50Context or environment of the image
    • G06V20/52Surveillance or monitoring of activities, e.g. for recognising suspicious objects

Abstract

The invention provides a gate detection method based on video, which is characterized by comprising the following steps: s1, establishing and training a deep learning model for gate detection; s2, setting a gate detection rule; s3, preprocessing an image to be detected; s4, inputting the preprocessed image into a deep learning model for gate detection to detect. The gate detection method based on the video is used for detecting the gate state by analyzing the video image, achieves real-time performance and automation of gate state detection, saves a great amount of labor cost and time cost, effectively improves the identification accuracy, reduces the false alarm rate, and has applicability to various scenes.

Description

Gate detection method based on video
Technical Field
The invention belongs to the technical field of video monitoring, and particularly relates to a gate detection method based on video.
Background
With the wide application of the video monitoring system in the water conservancy industry, the water conservancy video monitoring system plays an increasingly important role in daily management of the water conservancy departments, and staff can monitor each water conservancy facility on a monitoring center or a working computer in real time through the monitoring system to observe the current situation of water resources in real time, so that the working efficiency is greatly improved. The gate in hydraulic engineering is various, because the gate is huge in stress in the switching process, is easy to damage, and the situations of incomplete switching, blocking and the like often occur, and the existing solution is to adopt a gate switching detection switch, but sometimes the gate needs to operate in a semi-closed state, and the switching detection switch is not easy to detect at the moment.
Disclosure of Invention
In view of the above, the present invention is directed to a method for detecting a gate based on video, so as to solve the problem that the gate is not easy to detect during the opening and closing process. The gate detection method based on the video can analyze video streams transmitted by the existing video camera, has good detection effect on various gates for videos with different definition, and can automatically monitor the gate continuously by analyzing the gate state according to the position relation between the top line of the current gate position and the top line of the opening state and the top line of the closing state due to different gate positions detected by the gate in the opening and closing states.
In order to achieve the above purpose, the technical scheme of the invention is realized as follows:
A video-based gate detection method, comprising the steps of:
S1, establishing and training a deep learning model for gate detection;
S2, setting a gate detection rule;
s3, preprocessing an image to be detected;
S4, inputting the preprocessed image into a deep learning model for gate detection to detect.
Further, the deep learning model detection gate in step S4 includes: gate status determination, gate movement direction determination, and gate opening percentage calculation.
Further, the training of the deep learning model in step S1 adopts a random gradient descent method to perform iterative training on the model, each iteration makes the loss function smaller, and the loss function used is as follows:
where i is the ith grid, j is the jth bbox, Meaning that if the ith grid, j is the jth bbox is a gate; the first two terms are coordinate predictions,/>For predicted gate center point coordinates, x i,yi is the labeled gate center point, ω i、hi is the gate box width, height,/>To predict the width and height of the output gate box, C i represents the confidence of the gate box,Representing confidence of gate box prediction; the third term is to predict confidence of the box; the fourth term is to predict a box that does not contain gates; lambda coordnoobj is the weight coefficient, B is the number of anchor boxes, and s 2 is the total cell number on the feature map, i.e. the grid number.
Further, the detection setting in step S2 mainly includes setting of a top line position when the shutter is opened and setting of a top line position when the shutter is closed. When the gate is opened, the top line is the position of the upper edge of the gate, and the upper edge position is marked by drawing a line segment; when the gate is closed, the top line is the position of the upper edge of the gate, the upper edge position is marked by drawing a line segment, and the current state of the gate can be judged by judging the position relationship between the detected upper edge position of the current position of the gate and the two top lines.
Further, in the step S3, a gaussian filtering method is adopted to preprocess the image, so as to reduce image noise.
Further, the deep learning model detection gate in step S4 detects the image obtained in step S2, and obtains a score corresponding to the detected gate position, where the score calculation formula is as follows:
where θ is a parameter vector, h θ (x) represents the probability that its corresponding class label belongs to the positive case, i.e., the score in the text, for a given input x.
Further, the gate status judging process is as follows:
If the gate target is not detected by the continuous multi-frame in the step S4, the gate is considered to be closed; if the gate target is detected in step S4, the positional relationship between the top line of the gate position detected in step S4 and the top line when the set gate is completely closed is analyzed, and if the top line of the current gate position is above the set closing top line, the gate is considered to be opened, otherwise the gate is considered to be closed.
Further, the gate movement direction judging process is as follows:
In the judging of the gate movement direction, the top line of the gate position detected in the step S4 of two continuous frames is stored, and if the top line of the gate position of the current frame is above the top line of the gate of the previous frame, the gate is considered to move upwards; if the top line of the gate position of the current frame coincides with the top line of the gate position of the previous frame, the gate is considered to be stationary; if the top line of the gate position of the current frame is below the top line position of the gate of the previous frame, the gate is considered to be moving downwards.
Further, the gate opening percentage calculating method comprises the following steps: and calculating the distance d between the top line when the gate is fully opened and the top line when the gate is fully closed, and then calculating the distance delta d between the top line at the gate position detected in the step S4 and the top line when the gate is fully closed, wherein delta d/d is the percentage of gate opening.
Further, in the step S4, the process of detecting the image obtained in the step S3 is as follows: detecting the whole image through the trained YOLO model, recording the position and score of the detected target, if the detected target is larger than 0.8, considering the detected target as an effective target, and judging the moving direction of the effective target; if the detection target is smaller than 0.8, the gate is in a closed state.
Compared with the prior art, the gate detection method based on the video has the following advantages:
(1) According to the gate detection method based on the video, the gate state is detected by analyzing the video image, so that the real-time performance and the automation of the gate state detection are realized;
(2) The gate detection method based on the video, disclosed by the invention, has the advantages that a great amount of labor cost and time cost are saved, the identification accuracy is effectively improved, and the false alarm rate is reduced;
(3) The gate detection method based on the video has the applicability of various scenes.
Drawings
The accompanying drawings, which are included to provide a further understanding of the invention and are incorporated in and constitute a part of this specification, illustrate embodiments of the invention and together with the description serve to explain the invention. In the drawings:
FIG. 1 is a training flow chart of a gate detection flow chart deep learning model of a gate detection method based on video according to an embodiment of the present invention;
fig. 2 is a gate detection flow chart of a gate detection method based on video according to an embodiment of the present invention.
Detailed Description
It should be noted that, without conflict, the embodiments of the present invention and features of the embodiments may be combined with each other.
In the description of the present invention, it should be understood that the terms "center", "longitudinal", "lateral", "upper", "lower", "front", "rear", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", "outer", etc. indicate orientations or positional relationships based on the orientations or positional relationships shown in the drawings, are merely for convenience in describing the present invention and simplifying the description, and do not indicate or imply that the devices or elements referred to must have a specific orientation, be configured and operated in a specific orientation, and thus should not be construed as limiting the present invention. Furthermore, the terms "first," "second," and the like, are used for descriptive purposes only and are not to be construed as indicating or implying a relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defining "a first", "a second", etc. may explicitly or implicitly include one or more such feature. In the description of the present invention, unless otherwise indicated, the meaning of "a plurality" is two or more.
In the description of the present invention, it should be noted that, unless explicitly specified and limited otherwise, the terms "mounted," "connected," and "connected" are to be construed broadly, and may be either fixedly connected, detachably connected, or integrally connected, for example; can be mechanically or electrically connected; can be directly connected or indirectly connected through an intermediate medium, and can be communication between two elements. The specific meaning of the above terms in the present invention can be understood by those of ordinary skill in the art in a specific case.
The invention will be described in detail below with reference to the drawings in connection with embodiments.
Referring to fig. 1 and 2, the present invention provides a technical solution: a video-based gate detection method, comprising the steps of:
And S1, training a deep learning model. The method specifically comprises sample collection, data enhancement, sample labeling and model training under DarkNet framework. The collection of the sample needs to cover various possible patterns of the target in the application scene, and the sample needs to comprise a scene without a gate but with a gate easy to be detected by mistake so as to reduce the false detection rate; after the sample is collected, carrying out image enhancement on the sample, adjusting the brightness, angle, contrast and other information of the image, increasing the diversity of the sample and improving the robustness of the model; after the data enhancement is completed, marking the gate in the sample, wherein the marking of the sample requires the accuracy of the marked target position; after labeling is complete, the YOLO model is trained under DarkNet framework.
And S2, setting detection rules. The regular arrangement mainly comprises arrangement of the top line position when the gate is opened and arrangement of the top line position when the gate is closed. When the gate is opened, the top line is the position of the upper edge of the gate, and the upper edge position is marked by drawing a line segment; when the gate is closed, the top line is the position of the upper edge of the gate, the upper edge position is marked by drawing a line segment, and the current state of the gate can be judged by judging the position relationship between the detected upper edge position of the current position of the gate and the two top lines.
And S3, preprocessing the image. The image preprocessing is to smooth and remove the noise of the image to be processed before detection so as to achieve a better detection effect. In the method, the Gaussian filtering method is used for preprocessing the image, so that the image noise is reduced.
And S4, detecting a gate by the deep learning model. And (3) detecting the whole image by using the YOLO model trained in the step (S1), recording the position and the score of the detected target, and if the target score is greater than 0.8, considering the target as an effective target.
And judging the state of the gate. Since the gate cannot be seen after the gate is completely closed in some scenes, if the gate target is not detected in the consecutive multi-frame in step S4, the gate is considered to be closed; if the gate target is detected in step S4, the positional relationship between the top line of the gate position detected in step S4 and the top line when the set gate is completely closed is analyzed, and if the top line of the current gate position is above the set closing top line, the gate is considered to be opened, otherwise the gate is considered to be closed.
And judging the movement direction of the gate. Storing the top line of the gate position detected in the step S4 of two continuous frames, and if the top line of the gate position of the current frame is above the top line position of the gate of the previous frame, considering that the gate moves upwards; if the top line of the gate position of the current frame coincides with the top line of the gate position of the previous frame, the gate is considered to be stationary; if the top line of the gate position of the current frame is below the top line position of the gate of the previous frame, the gate is considered to be moving downwards.
And calculating the gate opening percentage. And (3) calculating the distance d between the top line when the gate is fully opened and the top line when the gate is fully closed, and then calculating the distance delta d between the top line at the gate position detected in the step (4) and the top line when the gate is fully closed, wherein delta d/d is the percentage of gate opening.
The working procedure of this embodiment is as follows:
The method is used for training the detection model in advance, samples of various gates in various scenes are required to be collected first, the samples are marked, namely, real position coordinates are marked according to the positions of the gates in images, the model is repeatedly and iteratively trained by adopting a random gradient descent method, the loss function is smaller in each iteration, and the used loss function has the following formula:
where i is the ith grid, j is the jth bbox, Meaning that if the ith grid, j is the jth bbox is a gate; the first two terms are coordinate predictions,/>For predicted gate center point coordinates, x i,yi is the labeled gate center point, ω i、hi is the gate box width, height,/>To predict the width and height of the output gate box, C i represents the confidence of the gate box,Representing confidence of gate box prediction; the third term is to predict confidence of the box; the fourth term is to predict a box that does not contain gates; lambda coordnoobj is the weight coefficient, B is the number of anchor boxes, and s 2 is the total cell number on the feature map, i.e. the grid number.
The iteration is continuous, so that box errors are smaller and smaller, and prediction is more accurate and more accurate.
And finally, determining the specific position of the gate in the image by using a YOLO model with the best detection effect.
Starting detection;
step S2, setting the orientation of the gate when the gate is completely opened and the top line of the gate when the gate is completely closed;
S3, preprocessing the video image, and removing image noise points; the Gaussian filter is used for preprocessing the image, so that noise can be effectively restrained, and the image can be smoothed;
and S4, detecting the image obtained in the step S3 by using a deep learning YOLO model to obtain a score corresponding to the detected gate position, wherein a score calculation formula is as follows:
Where θ is a parameter vector, h θ (x) represents its corresponding class label for a given input x, the probability of belonging to a positive case, i.e., the score in the text; the score is 0 at the lowest and 1 at the highest, and the result with the score less than 0.8 is filtered out, so that a correct detection result is left;
And (3) comparing the top line position of the effective gate target of each frame of video image obtained in the step (S4) with the top line position of the gate when the gate is completely opened and the top line position of the gate when the gate is completely closed, so as to judge the current state, the movement direction and the opening percentage of the gate.
The foregoing description of the preferred embodiments of the invention is not intended to be limiting, but rather is intended to cover all modifications, equivalents, alternatives, and improvements that fall within the spirit and scope of the invention.

Claims (5)

1. A video-based gate detection method, comprising the steps of:
S1, building and training a YOLO model for gate detection;
S2, setting a gate detection rule;
The detection setting in the step S2 mainly comprises the setting of the top line position when the gate is opened and the setting of the top line position when the gate is closed;
s3, preprocessing an image to be detected;
S4, inputting the preprocessed image into a YOLO model for gate detection for detection;
The YOLO model inspection gate in step S4 includes: judging the state of the gate, judging the moving direction of the gate and calculating the opening percentage of the gate;
The gate state judging process is as follows:
If the gate target is not detected by the continuous multi-frame in the step S4, the gate is considered to be closed; if the gate target is detected in the step S4, analyzing the position relation between the top line of the gate position detected in the step S4 and the top line when the set gate is completely closed, if the top line of the current gate position is above the set closing top line, the gate is considered to be opened, otherwise, the gate is considered to be closed;
the gate movement direction judging process is as follows:
Judging the gate movement direction, storing the top line of the gate position detected in the step S4 of two continuous frames, and if the top line of the gate position of the current frame is above the top line position of the gate of the previous frame, considering that the gate moves upwards; if the top line of the gate position of the current frame coincides with the top line of the gate position of the previous frame, the gate is considered to be stationary; if the top line of the gate position of the current frame is below the top line of the gate position of the previous frame, the gate is considered to move downwards;
The method for calculating the gate opening percentage comprises the following steps: and calculating the distance d between the top line when the gate is fully opened and the top line when the gate is fully closed, and then calculating the distance delta d between the top line at the gate position detected in the step S4 and the top line when the gate is fully closed, wherein delta d/d is the percentage of gate opening.
2. The video-based gate detection method of claim 1, wherein:
In the step S1, the model is trained repeatedly and iteratively by adopting a random gradient descent method, and a loss function is used as follows:
where i is the ith grid, j is the jth bbox, The j bbox th, representing the i-th grid, is a gate; the first two terms are coordinate predictions,/>For predicted gate center point coordinates, x i,yi is the labeled gate center point, ω i、hi is the gate bbox width, height,/>To predict the width, height of the output gate bbox, C i represents the confidence of bbox containing the gate,/>Representing confidence in bbox predictions containing gates; the third term is to predict the confidence level of bbox that contains gates; the fourth term is to predict bbox that does not contain gates; lambda coordnoobj is a weight coefficient, B is the number of anchor frames, and s 2 is the total grid number on the feature map.
3. The video-based gate detection method of claim 1, wherein: in the step S3, the image is preprocessed by adopting a Gaussian filtering method, so that the image noise is reduced.
4. The video-based gate detection method of claim 1, wherein: in the step S4, the YOLO model detection gate detects the image obtained in the step S2, and a score corresponding to the detected gate position is obtained, where a score calculation formula is as follows:
where θ is a parameter vector, h θ (x) represents the probability that its corresponding class label belongs to the positive case, i.e., the score in the text, for a given input x.
5. The video-based gate detection method of claim 1, wherein: in step S4, the process of detecting the image obtained in step S3 includes: detecting the whole image through the trained YOLO model, and recording the position and score of the detected target; if the detected target is larger than 0.8, the detected target is considered as an effective target, and the moving direction of the effective target is judged; if the detection target is smaller than 0.8, the gate is in a closed state.
CN202010864191.1A 2020-08-25 2020-08-25 Gate detection method based on video Active CN112001320B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202010864191.1A CN112001320B (en) 2020-08-25 2020-08-25 Gate detection method based on video

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202010864191.1A CN112001320B (en) 2020-08-25 2020-08-25 Gate detection method based on video

Publications (2)

Publication Number Publication Date
CN112001320A CN112001320A (en) 2020-11-27
CN112001320B true CN112001320B (en) 2024-04-23

Family

ID=73471492

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202010864191.1A Active CN112001320B (en) 2020-08-25 2020-08-25 Gate detection method based on video

Country Status (1)

Country Link
CN (1) CN112001320B (en)

Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103973999A (en) * 2013-02-01 2014-08-06 佳能株式会社 Imaging apparatus and control method therefor
CN204010121U (en) * 2014-07-11 2014-12-10 山东新北洋信息技术股份有限公司 The position detecting mechanism of gate and paper money processing machine
CN108318581A (en) * 2017-12-08 2018-07-24 中国兵器科学研究院宁波分院 A kind of arc surface workpiece ultrasonic C-scanning automatic testing method without Set and Positioning
CN109405808A (en) * 2018-10-19 2019-03-01 天津英田视讯科技有限公司 A kind of hydrological monitoring spherical camera
CN110516615A (en) * 2019-08-29 2019-11-29 广西师范大学 Human and vehicle shunting control method based on convolutional neural networks
CN110999765A (en) * 2018-10-08 2020-04-14 台湾积体电路制造股份有限公司 Irrigation system for irrigating land
CN111209811A (en) * 2019-12-26 2020-05-29 的卢技术有限公司 Method and system for detecting eyeball attention position in real time

Family Cites Families (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US7415126B2 (en) * 1992-05-05 2008-08-19 Automotive Technologies International Inc. Occupant sensing system
US20200005468A1 (en) * 2019-09-09 2020-01-02 Intel Corporation Method and system of event-driven object segmentation for image processing

Patent Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103973999A (en) * 2013-02-01 2014-08-06 佳能株式会社 Imaging apparatus and control method therefor
CN204010121U (en) * 2014-07-11 2014-12-10 山东新北洋信息技术股份有限公司 The position detecting mechanism of gate and paper money processing machine
CN108318581A (en) * 2017-12-08 2018-07-24 中国兵器科学研究院宁波分院 A kind of arc surface workpiece ultrasonic C-scanning automatic testing method without Set and Positioning
CN110999765A (en) * 2018-10-08 2020-04-14 台湾积体电路制造股份有限公司 Irrigation system for irrigating land
CN109405808A (en) * 2018-10-19 2019-03-01 天津英田视讯科技有限公司 A kind of hydrological monitoring spherical camera
CN110516615A (en) * 2019-08-29 2019-11-29 广西师范大学 Human and vehicle shunting control method based on convolutional neural networks
CN111209811A (en) * 2019-12-26 2020-05-29 的卢技术有限公司 Method and system for detecting eyeball attention position in real time

Non-Patent Citations (3)

* Cited by examiner, † Cited by third party
Title
YOLOv3: An Incremental Improvement;Joseph Redmon等;《arXiv》;1-6 *
基于ARM的嵌入式闸门智能测控仪表的设计;涂从刚;《中国优秀硕士学位论文全文数据库 工程科技Ⅱ辑》(第07期);C030-5 *
水利枢纽闸门维护自动化-故障诊断技术研究;肖志怀;《中国优秀博硕士学位论文全文数据库 (博士) 工程科技Ⅱ辑》(第02期);C037-32 *

Also Published As

Publication number Publication date
CN112001320A (en) 2020-11-27

Similar Documents

Publication Publication Date Title
CN110059694B (en) Intelligent identification method for character data in complex scene of power industry
CN109644255B (en) Method and apparatus for annotating a video stream comprising a set of frames
CN112199993A (en) Method for identifying transformer substation insulator infrared image detection model in any direction based on artificial intelligence
CN113139521B (en) Pedestrian boundary crossing monitoring method for electric power monitoring
CN111652225A (en) Non-invasive camera reading method and system based on deep learning
CN106952293A (en) A kind of method for tracking target based on nonparametric on-line talking
CN112257527B (en) Mobile phone detection method based on multi-target fusion and space-time video sequence
CN111476160A (en) Loss function optimization method, model training method, target detection method, and medium
CN109583295A (en) A kind of notch of switch machine automatic testing method based on convolutional neural networks
CN112597928A (en) Event detection method and related device
CN112561885B (en) YOLOv 4-tiny-based gate valve opening detection method
CN115035328A (en) Converter image increment automatic machine learning system and establishment training method thereof
CN114879628A (en) Multi-mode industrial process fault diagnosis method based on local maximum mean difference resistance
CN112001320B (en) Gate detection method based on video
CN113657151A (en) Water traffic violation detection method based on YOLO target detection algorithm
CN111626104B (en) Cable hidden trouble point detection method and device based on unmanned aerial vehicle infrared thermal image
CN116205905B (en) Power distribution network construction safety and quality image detection method and system based on mobile terminal
CN116385485B (en) Video tracking method and system for long-strip-shaped tower crane object
CN115995056A (en) Automatic bridge disease identification method based on deep learning
CN116543327A (en) Method, device, computer equipment and storage medium for identifying work types of operators
CN114694090A (en) Campus abnormal behavior detection method based on improved PBAS algorithm and YOLOv5
CN114492657A (en) Plant disease classification method and device, electronic equipment and storage medium
CN110363109B (en) Computer vision-based coverage type oil leakage detection system and method
CN113673534A (en) RGB-D image fruit detection method based on fast RCNN
CN112906593A (en) Sluice image identification method based on fast RCNN

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant