CN109410496B - Intrusion early warning method and device and electronic equipment - Google Patents
Intrusion early warning method and device and electronic equipment Download PDFInfo
- Publication number
- CN109410496B CN109410496B CN201811256568.4A CN201811256568A CN109410496B CN 109410496 B CN109410496 B CN 109410496B CN 201811256568 A CN201811256568 A CN 201811256568A CN 109410496 B CN109410496 B CN 109410496B
- Authority
- CN
- China
- Prior art keywords
- target
- early warning
- abnormal
- model
- video frame
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G08—SIGNALLING
- G08B—SIGNALLING OR CALLING SYSTEMS; ORDER TELEGRAPHS; ALARM SYSTEMS
- G08B13/00—Burglar, theft or intruder alarms
- G08B13/18—Actuation by interference with heat, light, or radiation of shorter wavelength; Actuation by intruding sources of heat, light, or radiation of shorter wavelength
- G08B13/189—Actuation by interference with heat, light, or radiation of shorter wavelength; Actuation by intruding sources of heat, light, or radiation of shorter wavelength using passive radiation detection systems
- G08B13/194—Actuation by interference with heat, light, or radiation of shorter wavelength; Actuation by intruding sources of heat, light, or radiation of shorter wavelength using passive radiation detection systems using image scanning and comparing systems
- G08B13/196—Actuation by interference with heat, light, or radiation of shorter wavelength; Actuation by intruding sources of heat, light, or radiation of shorter wavelength using passive radiation detection systems using image scanning and comparing systems using television cameras
- G08B13/19602—Image analysis to detect motion of the intruder, e.g. by frame subtraction
- G08B13/19613—Recognition of a predetermined image pattern or behaviour pattern indicating theft or intrusion
Abstract
The invention provides an intrusion early warning method, an intrusion early warning device and electronic equipment, wherein the method comprises the following steps: judging whether an abnormal target exists in the designated area or not according to the video frame image; if yes, determining the target information of the abnormal target; the target information comprises one or more of target category, target position, target size, target speed and target track; generating an early warning level corresponding to target information through a hierarchical early warning model obtained through pre-training; wherein, the grading early warning model is a deep neural network model; and carrying out early warning according to the early warning level. The invention can better determine the information of the invader and carry out graded alarm based on the information of the invader, is beneficial to relevant personnel to take measures in a targeted manner, and further ensures the safety of railway operation.
Description
Technical Field
The invention relates to the technical field of safety, in particular to an intrusion early warning method and device and electronic equipment.
Background
With the continuous increase of the operating mileage of the high-speed railway and the gradual increase of the speed of the high-speed railway, the operating safety of the high-speed railway becomes a big thing of the national customs. However, in the road sections such as mountainous areas and tunnel portals, sudden falling rocks and debris flow are possible, and the condition that machines, personnel and construction materials invade existing line boundaries frequently occurs in the constructed road sections, and the conditions seriously affect the driving safety of the high-speed railway. Because the train has high running speed, foreign matter identification is carried out by driver vision and a traditional detection method, and the running safety is difficult to ensure. The limit invasion of foreign matters of the track line can not only lead to the destruction of existing buildings along the railway line and the interruption of railway running, but also lead to the derailment and the casualties of the train, thereby greatly harming the life and property safety of people and causing the loss of national property which can not be estimated. Therefore, the real-time detection and alarm of the intrusion of foreign matters in the railway have important significance for ensuring the safety of rail transit.
Therefore, various railway perimeter intrusion monitoring methods such as infrared, ultrasonic and laser are researched in various countries in the world, but in the existing method, only an intruder (also called as an abnormal target) in a designated area is detected and an alarm is issued, and related workers are difficult to acquire exact intrusion information, are inconvenient to take measures aiming at the intruder in a targeted manner, and are difficult to better guarantee the safety of railway operation.
Disclosure of Invention
In view of the above, an object of the present invention is to provide an intrusion early warning method, an intrusion early warning device, and an electronic device, which can better determine intruder information and perform a graded alarm based on the intruder information, thereby being helpful for relevant personnel to take measures in a targeted manner and further ensuring the safety of railway operation.
In order to achieve at least one of the above purposes, the technical solution adopted by the embodiment of the present invention is as follows:
in a first aspect, an embodiment of the present invention provides an intrusion early warning method, including: judging whether an abnormal target exists in the designated area or not according to the video frame image; wherein the video frame image is a shot image of the designated area; if yes, determining the target information of the abnormal target; the target information comprises one or more of target category, target position, target size, target speed and target track; generating an early warning level corresponding to the target information through a hierarchical early warning model obtained through pre-training; wherein, the grading early warning model is a deep neural network model; and carrying out early warning according to the early warning level.
Further, the step of judging whether an abnormal target exists in the designated area according to the monitoring image includes: judging whether a difference exists between the video frame image and a preset image or not; the monitoring image and the preset image are shot images of a designated area, and no abnormal object exists in the designated area in the preset image; if yes, determining that an abnormal target exists in the specified area.
Further, the step of determining the target information of the abnormal target includes: tracking and positioning the abnormal target by utilizing a plurality of continuously acquired video frame images, and determining one or more of the target position, the target speed and the target track of the abnormal target; wherein the target track comprises a target motion track and/or a target prediction track; performing target identification on the video frame image through a preset target identification model, and determining the target category of the abnormal target; wherein the target identification model is a deep convolutional neural network model.
Further, the target recognition model is a GoogLeNet network model.
Further, the training step of the grading early warning model comprises the following steps: acquiring a preset amount of training data; wherein the training data carries labels of different early warning levels; the training data is one or a combination of multiple information of target category, target position, target size, target speed and target track; and training the grading early warning model by adopting the training data, and stopping training until the loss function value of the grading early warning model converges to a preset threshold value.
Further, the grading early warning model comprises a feature extractor and a classifier; the feature extractor comprises a keyword extraction unit and an LSTM unit; the step of generating the early warning level corresponding to the target information through the pre-trained hierarchical early warning model comprises the following steps: inputting the target information into the keyword extraction unit, and extracting keywords of the target information through the keyword extraction unit to obtain a word vector; inputting the word vector into the LSTM unit, and obtaining a sentence vector through the LSTM unit based on the target information and the memorized historical target information; and inputting the sentence vectors into the classifier, and classifying the sentence vectors through the classifier to obtain an early warning level.
Further, the LSTM unit comprises an LSTM layer, an average pooling layer and a logistic regression layer which are sequentially connected.
In a second aspect, an embodiment of the present invention further provides an intrusion warning device, including: the abnormity judgment module is used for judging whether an abnormal target exists in the designated area according to the video frame image; wherein the video frame image is a shot image of the designated area; the target determining module is used for determining the target information of the abnormal target when the judgment result of the abnormal judging module is yes; the target information comprises one or more of target category, target position, target size, target speed and target track; the level generation module is used for generating an early warning level corresponding to the target information through a pre-trained hierarchical early warning model; wherein, the grading early warning model is a deep neural network model; and the early warning module is used for carrying out early warning according to the early warning level.
In a third aspect, an embodiment of the present invention provides an electronic device, including a memory and a processor, where the memory is used to store a program that supports the processor to execute the method in any one of the first aspect, and the processor is configured to execute the program stored in the memory.
In a fourth aspect, an embodiment of the present invention provides a machine-readable storage medium storing program code executable by a processor, the program code being configured to cause the processor to execute the method of any one of the first aspect.
The embodiment of the invention provides an intrusion early warning method, an intrusion early warning device and electronic equipment, wherein when an abnormal target exists in a designated area according to a video frame image (a shot image of the designated area), target information of the abnormal target can be determined, wherein the target information comprises one or more of target type, target position, target size, target speed and target track; and then generating an early warning level corresponding to the target information through a hierarchical early warning model obtained by pre-training, and carrying out early warning according to the early warning level. The method can better determine the information of the invader (namely the target information), and the neural network model is adopted to carry out graded alarm based on the invader information, thereby being beneficial to relevant personnel to take measures in a targeted manner and further ensuring the safety of railway operation.
Additional features and advantages of embodiments of the invention will be set forth in the description which follows, and in part will be obvious from the description, or may be learned by practice of embodiments of the invention as set forth above.
In order to make the aforementioned and other objects, features and advantages of the present invention comprehensible, preferred embodiments accompanied with figures are described in detail below.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, and it is obvious that the drawings in the following description are some embodiments of the present invention, and other drawings can be obtained by those skilled in the art without creative efforts.
Fig. 1 is a flowchart illustrating an intrusion warning method according to an embodiment of the present invention;
FIG. 2 is a schematic diagram of a hierarchical alarm system according to an embodiment of the present invention;
fig. 3 is a schematic structural diagram illustrating a hierarchical early warning model according to an embodiment of the present invention;
fig. 4 is a schematic diagram illustrating a hierarchical early warning model according to an embodiment of the present invention;
FIG. 5 illustrates a time-expanded view of an LSTM cell provided by an embodiment of the present invention;
fig. 6 is a block diagram illustrating an intrusion warning device according to an embodiment of the present invention;
fig. 7 shows a schematic structural diagram of an electronic device according to an embodiment of the present invention.
Detailed Description
To make the objects, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings, and it is apparent that the described embodiments are some, but not all embodiments of the present invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
In view of the fact that in the prior art, only an intruder is detected in a designated area and an alarm is issued, and related workers are difficult to acquire exact intrusion information and are inconvenient to take measures in a targeted manner for the intruder, in order to improve the problem, the intrusion early warning method, the intrusion early warning device and the electronic equipment provided by the embodiment of the invention can be applied to occasions such as a railway and the like requiring monitoring and early warning of the intruder, and the embodiment of the invention is described in detail below.
First, referring to a flowchart of an intrusion warning method shown in fig. 1, the method includes the following steps:
step S102, judging whether an abnormal target exists in the designated area according to the video frame image; the video frame image is a shot image of a designated area. The video frame image may be acquired by an image acquisition device such as a camera installed in a designated area when the designated area is monitored in real time, and may be one or more frames. The above-described designated area may also be referred to as an image monitoring area.
In one embodiment of determining whether an abnormal target exists, it may be determined whether a difference exists between a video frame image and a preset image; the monitoring image and the preset image are shot images of a designated area, and no abnormal object exists in the designated area in the preset image; if so, determining that an abnormal target exists in the specified area. The preset image may be a previously acquired image of a designated area where there is no abnormality. The image of the designated area where the abnormal object is not present may be referred to as a static background of the designated area. Of course, in other embodiments, the difference between a plurality of video frame images can be directly compared to determine whether there is an abnormal object in the designated area.
Step S104, if yes, determining the target information of the abnormal target; the target information comprises one or more of target category, target position, target size, target speed and target track. The target trajectory may include a target motion trajectory and a target predicted trajectory.
When determining the target information of the abnormal target, the abnormal target can be tracked and positioned by utilizing a plurality of continuously collected video frame images, and one or more of the target position, the target speed and the target track of the abnormal target are determined; then, carrying out target identification on the video frame image through a preset target identification model, and determining the target category of the abnormal target; the target recognition model is a deep convolutional neural network model.
The deep convolutional network model adopted in the embodiment can be implemented by network structures such as LeNet, ResNet, VGG, AlexNet, google LeNet and the like. In this embodiment, factors such as complexity and running time of the deep convolutional network model are comprehensively considered, and in an implementation, google lenet may be used to classify and identify the abnormal target. The deep convolution structure of the google lenet network model comprises 22 layers, and in order to enhance the algorithm real-time performance, the obtained target region (namely the region of interest) can be used as a model input on the basis of target tracking and positioning. The well-trained google lenet model is used for training again on the sample library used in the embodiment, so that the method can identify various common objects which easily enter a monitoring area, such as people, common animals, birds, insects, falling rocks, plastic bags, cartons and the like.
The embodiment can learn the object characteristics by training the target recognition model so as to improve the recognition accuracy. And the target recognition model can be used for reinforcement learning in the operation process, new object types can be continuously learned, and the recognition accuracy and pertinence are further improved.
Step S106, generating an early warning level corresponding to target information through a pre-trained hierarchical early warning model; wherein, the grading early warning model is a deep neural network model.
The deep neural network has strong fault tolerance, self-learning, self-organization and self-adaption capability and can simulate complex nonlinear mapping. In order to realize comprehensive judgment of the actual situation, the deep neural network is adopted to evaluate the possibility and the degree of harm of the target invading the dangerous area according to the target information of the abnormal target obtained in the previous steps.
For easy understanding, the embodiment further provides a training step of the hierarchical early warning model, including: acquiring a preset amount of training data; wherein the training data carries labels of different early warning levels; the training data is one or a combination of multiple information of target category, target position, target size, target speed and target track; and training the grading early warning model by adopting training data until the loss function value of the grading early warning model converges to a preset threshold value, and stopping training. Different target information has different safety threat degrees to the designated area, and different early warning levels can be generated based on the safety threat degree (or the emergency degree), wherein the early warning levels can be different levels such as general, attention, severity and emergency, or are directly graded according to modes such as primary early warning, secondary early warning and tertiary early warning.
And finally, the obtained grading early warning model can generate a corresponding early warning grade according to the input target information. As shown in fig. 2, a hierarchical alarm diagram is obtained by inputting information such as a classification recognition result, a target position, a target size, a target speed, and a target track into a hierarchical early warning model, and performing early warning (i.e., corresponding to a hierarchical alarm) based on an early warning level output by the hierarchical early warning model.
And S108, early warning according to the early warning level.
For example, the early warning modes corresponding to different early warning levels are different, if the early warning level is low, related personnel can be prompted to pay attention in a text mode on a designated display interface, and if the early warning level is high, modes of alarming, automatically initiating an alarm call to the related personnel or sending an alarm short message and the like can be adopted. The above is merely an example, and the actual application may be flexibly set according to requirements, and is not limited herein.
In the above method of the present embodiment, when it is determined from the video frame image (the captured image of the specified area) that there is an abnormal object in the specified area, object information of the abnormal object may be determined, the object information including one or more of an object type, an object position, an object size, an object speed, and an object trajectory; and then generating an early warning level corresponding to the target information through a hierarchical early warning model obtained by pre-training, and carrying out early warning according to the early warning level. The method can better determine the information of the invader (namely the target information), and the neural network model is adopted to carry out graded alarm based on the invader information, thereby being beneficial to relevant personnel to take measures in a targeted manner and further ensuring the safety of railway operation.
For ease of understanding, the following exemplifies the manner in which the target information is determined by the present embodiment: for example, the position, size and shape of the abnormal target can be calculated by the difference between adjacent frames in the video frame image, the actual position of the abnormal target can be deduced by using the camera information calibrated in advance, and the relative position between the abnormal target and the dangerous area can be judged by using the dangerous area position set in advance.
The movement speed of the abnormal object can be obtained through the front frame image and the rear frame image. Because the two frames of images have the same background, the two frames of images are respectively subtracted from the original background image without the abnormal target, and then the two moving targets are respectively obtained through denoising and binarization processing, and the distance between the central points of the two moving targets, namely the inter-frame moving speed of the abnormal target, is obtained. By recording the positions of the abnormal targets in different frame images, a target motion track can be generated.
And predicting the future motion track of the abnormal target according to the motion track of the abnormal target and the current motion speed and the motion direction of the abnormal target. For convenience of description, two frames of images can be captured from a video stream acquired by an image acquisition device for prediction, a straight line can be determined from a target central point in the two frames of images, and then the range can be reduced to a semicircular area with the target central point of the previous frame of image as a center and with a and b as inner and outer radii respectively. Where v is the moving distance of the abnormal object between frames, d is the maximum diameter of the abnormal object, and Δ d is the prediction interval error distance, the following formula (1) can be listed:
let A, B, C be the target center point of two consecutive images and the target center predicted point of the third image, respectively, and connect these three points to form a triangle, whichThe angle between middle AB and BC is alpha. The points obtained when the condition (2) is satisfied are all within a semicircular range, and AB2+BC2＜AC2；
By the method, the prediction region of the abnormal target in the next frame image can be obtained, then the prediction region is subtracted from the region corresponding to the original background image, and finally the position of the abnormal target can be obtained after denoising and binarization processing, so that the target prediction track can be generated.
The embodiment provides a specific implementation manner of a hierarchical early warning model, and refer to a schematic structural diagram of the hierarchical early warning model shown in fig. 3, where the hierarchical early warning model includes a feature extractor and a classifier; the feature extractor comprises a keyword extraction unit and an LSTM (Long Short-Term Memory) unit. Among them, LSTM is a time-recursive neural network, allowing information to persist, suitable for processing and predicting important events with relatively long intervals and delays in time series. With reference to the schematic diagram of the hierarchical early warning model shown in fig. 4 provided in this embodiment, the early warning level corresponding to the target information is generated by the hierarchical early warning model obtained through pre-training, and the following steps are performed:
(1) and inputting the target information into a keyword extraction unit, and extracting the keywords of the target information through the keyword extraction unit to obtain a word vector.
(2) And inputting the word vector into an LSTM unit, and obtaining a sentence vector through the LSTM unit based on the target information and the memorized historical target information.
(3) And inputting the sentence vectors into a classifier, and classifying the sentence vectors through the classifier to obtain the early warning level.
As shown in fig. 4, the initial input sentence may be a sentence composed of target information, the input sentence is extracted by the keywords to obtain a plurality of word vectors, and then the LSTM unit may integrate the extracted word vectors to generate a sentence vector recognizable by the classifier. The classifier may include an input node (which may also be referred to as an input layer), a hidden node (which may also be referred to as a hidden layer), and an output node (which may also be referred to as an output layer), and may perform classification processing on the sentence vector output by the LSTM, so as to obtain an early warning level corresponding to the initially input sentence.
The basic principle of the LSTM cell can be seen in FIG. 5, which is a time development of the LSTM cell, X (X)0、x1、…、xn) Is a vector representation of the input text with dimension d, wherein the dimension of the input text can be understood as the number of word information in the target information. A plurality of LSTM nodes, in particular an array of m LSTM nodes, is illustrated. Different LSTM nodes correspond to different times, and target information input to the LSTM nodes at different times is usually different. At each time unit t, the input of LSTM is xtOutput is ht. Then h for all time unitstAnd (3) carrying out average pooling (Mean pooling) to obtain a vector h, and finally carrying out Logistic regression classification (Logistic regression) on the vector h to obtain a final result. In this way, LSTM can also take into account stored past time accepted information when processing current information, such as in process h1Then, the h of the last time node is considered0And information is beneficial to improving the accuracy of results.
In specific implementation, the LSTM unit includes an LSTM layer, an average pooling layer, and a logistic regression layer, which are connected in sequence.
In summary, the intrusion early warning method provided by the embodiment can be applied to a video monitoring system, and has the following advantages:
(1) compared with the prior art that only the invasion object in the monitored area can be known, but the related information of the invasion object cannot be known exactly, the embodiment can more clearly and accurately determine the category of the invasion object (abnormal target) and the information such as the target classification recognition result, the target motion track, the target motion speed, the target motion direction, the target prediction track and the like.
(2) Compared with the prior art that an alarm is only initiated when an invader is monitored, the embodiment can lead relevant personnel to take corresponding measures in a targeted manner according to the early warning level by adopting a graded early warning mode. In the embodiment, the severity of the intrusion event can be determined by using the information of the intrusion object of the deep neural network, so that the early warning level corresponding to the information of the intrusion object can be generated for early warning. And because the invasion modes of the invaders are different, the types and sizes of the invaders, the positions of the invaders, the movement speeds and movement tracks of the invaders and other factors are combined together to be complex, the danger degree is not conveniently and objectively and reasonably judged manually based on the complex invasion condition, and the trained neural network model is adopted, so that the various information of the invaders can be comprehensively analyzed and judged by the model only by inputting the relevant information of the invaders into the model, and the corresponding early warning level is obtained, thereby achieving better early warning effect and further improving the safety and reliability of railway operation.
Furthermore, in the related art, the early warning level may be simply divided into several early warning levels by a manual operation based on a single threshold judgment (such as a single factor only according to the type of the invader or only according to the position of the invader), and then the level to which the field condition belongs is manually judged. Taking a railway as an example, the scene is very complex, the situations which can occur in the scene are variable, the types of invasive species are various, the invasive modes are various, and the emergency degree and the hazard degree of the situations cannot be simply determined manually. According to the embodiment, various information is comprehensively processed through an artificial intelligence method, reasonable and accurate grading early warning can be performed based on the invaders, and the safety of railway operation can be further guaranteed.
Corresponding to the intrusion early warning method provided in this embodiment, this embodiment further provides an intrusion early warning device, as shown in fig. 6, where the device includes:
an anomaly judgment module 62, configured to judge whether an anomaly target exists in the designated area according to the video frame image; the video frame image is a shot image of a designated area;
a target determining module 64, configured to determine target information of the abnormal target when the determination result of the abnormality determining module is yes; the target information comprises one or more of target category, target position, target size, target speed and target track;
the level generation module 66 is configured to generate an early warning level corresponding to the target information through a pre-trained hierarchical early warning model; wherein, the grading early warning model is a deep neural network model;
and the early warning module 68 is used for carrying out early warning according to the early warning level.
The device provided by the embodiment can better determine the invader information (namely, the target information), and performs graded alarm based on the invader information by adopting the neural network model, so that relevant personnel can take measures in a targeted manner, and the safety of railway operation is further guaranteed.
In one embodiment, the abnormality determining module is configured to: judging whether a difference exists between the video frame image and a preset image or not; the monitoring image and the preset image are shot images of a designated area, and no abnormal object exists in the designated area in the preset image; if so, determining that an abnormal target exists in the specified area.
In one embodiment, the target determination module is configured to: tracking and positioning an abnormal target by utilizing a plurality of continuously collected video frame images, and determining one or more of the target position, the target speed and the target track of the abnormal target; the target track comprises a target motion track and/or a target prediction track; performing target identification on the video frame image through a preset target identification model, and determining the target category of an abnormal target; the target recognition model is a deep convolutional neural network model.
In one embodiment, the target recognition model is a google lenet network model.
In one embodiment, the training step of the hierarchical early warning model includes: acquiring a preset amount of training data; wherein the training data carries labels of different early warning levels; the training data is one or a combination of multiple information of target category, target position, target size, target speed and target track; and training the grading early warning model by adopting training data until the loss function value of the grading early warning model converges to a preset threshold value, and stopping training.
In one embodiment, the hierarchical early warning model includes a feature extractor and a classifier; the feature extractor comprises a keyword extraction unit and an LSTM unit;
based on this, the level generation module is configured to: inputting the target information into a keyword extraction unit, and extracting keywords of the target information through the keyword extraction unit to obtain a word vector; inputting the word vector into an LSTM unit, and obtaining a sentence vector through the LSTM unit based on target information and memorized historical target information; and inputting the sentence vectors into a classifier, and classifying the sentence vectors through the classifier to obtain the early warning level.
In one embodiment, the LSTM unit includes an LSTM layer, an averaging pooling layer, and a logistic regression layer, which are connected in sequence.
The device provided by the embodiment has the same implementation principle and technical effect as the foregoing embodiment, and for the sake of brief description, reference may be made to the corresponding contents in the foregoing method embodiment for the portion of the embodiment of the device that is not mentioned.
Further, the present embodiment also provides a machine-readable storage medium storing program code executable by a processor, the program code being configured to cause the processor to execute the aforementioned intrusion alert method.
The embodiment provides an electronic device, referring to a schematic structural diagram of an electronic device shown in fig. 7, the electronic device includes: a processor 70, a memory 71, a bus 72 and a communication interface 73, wherein the processor 70, the communication interface 73 and the memory 71 are connected through the bus 72; the processor 70 is arranged to execute executable modules, such as computer programs, stored in the memory 71.
The Memory 71 may include a high-speed Random Access Memory (RAM) and may further include a non-volatile Memory (non-volatile Memory), such as at least one disk Memory. The communication connection between the network element of the system and at least one other network element is realized through at least one communication interface 73 (which may be wired or wireless), and the internet, a wide area network, a local network, a metropolitan area network, and the like can be used.
The bus 72 may be an ISA bus, PCI bus, EISA bus, or the like. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of illustration, only one double-headed arrow is shown in FIG. 7, but this does not indicate only one bus or one type of bus.
The memory 71 is configured to store a program, and the processor 70 executes the program after receiving an execution instruction, and the method executed by the apparatus defined by the flow program disclosed in any of the foregoing embodiments of the present invention may be applied to the processor 70, or implemented by the processor 70.
The processor 70 may be an integrated circuit chip having signal processing capabilities. In implementation, the steps of the above method may be performed by integrated logic circuits of hardware or instructions in the form of software in the processor 70. The Processor 70 may be a general-purpose Processor, and includes a Central Processing Unit (CPU), a Network Processor (NP), and the like; the device can also be a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field Programmable Gate Array (FPGA) or other Programmable logic device, a discrete Gate or transistor logic device, or a discrete hardware component. The various methods, steps, and logic blocks disclosed in this embodiment may be implemented or performed. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like. The steps of the method disclosed in connection with the present embodiment may be directly implemented by a hardware decoding processor, or implemented by a combination of hardware and software modules in the decoding processor. The software module may be located in ram, flash memory, rom, prom, or eprom, registers, etc. storage media as is well known in the art. The storage medium is located in a memory 71, and the processor 70 reads the information in the memory 71 and completes the steps of the method in combination with the hardware thereof.
The electronic device can be used for executing the grading early warning method provided by the embodiment, or provided with the grading early warning device provided by the embodiment.
Further, the present embodiment also provides a machine-readable storage medium storing program code executable by a processor, the program code being configured to cause the processor to execute the aforementioned intrusion alert method.
It can be clearly understood by those skilled in the art that, for convenience and brevity of description, the specific working process of the system described above may refer to the corresponding process in the foregoing embodiments, and is not described herein again.
The intrusion early warning method, the intrusion early warning device and the computer program product of the electronic device provided by the embodiments of the present invention include a computer-readable storage medium storing program codes, instructions included in the program codes may be used to execute the method described in the foregoing method embodiments, and specific implementation may refer to the method embodiments, and will not be described herein again.
The functions, if implemented in the form of software functional units and sold or used as a stand-alone product, may be stored in a computer readable storage medium. Based on such understanding, the technical solution of the present invention may be embodied in the form of a software product, which is stored in a storage medium and includes instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the method according to the embodiments of the present invention. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk or an optical disk, and other various media capable of storing program codes.
Finally, it should be noted that: the above-mentioned embodiments are only specific embodiments of the present invention, which are used for illustrating the technical solutions of the present invention and not for limiting the same, and the protection scope of the present invention is not limited thereto, although the present invention is described in detail with reference to the foregoing embodiments, those skilled in the art should understand that: any person skilled in the art can modify or easily conceive the technical solutions described in the foregoing embodiments or equivalent substitutes for some technical features within the technical scope of the present disclosure; such modifications, changes or substitutions do not depart from the spirit and scope of the embodiments of the present invention, and they should be construed as being included therein. Therefore, the protection scope of the present invention shall be subject to the protection scope of the appended claims.
Claims (6)
1. An intrusion early warning method, which is used for railway intrusion monitoring, is characterized by comprising the following steps:
judging whether an abnormal target exists in the designated area or not according to the video frame image; wherein the video frame image is a shot image of the designated area;
if yes, determining the target information of the abnormal target; the target information comprises a target type, a target position, a target size, a target speed and a target track;
generating an early warning level corresponding to the target information through a hierarchical early warning model obtained through pre-training; wherein, the grading early warning model is a deep neural network model;
carrying out early warning according to the early warning level;
the grading early warning model comprises a feature extractor and a classifier; the feature extractor comprises a keyword extraction unit and an LSTM unit;
the step of generating the early warning level corresponding to the target information through the pre-trained hierarchical early warning model comprises the following steps: inputting the target information into the keyword extraction unit, and extracting keywords of the target information through the keyword extraction unit to obtain a word vector; inputting the word vector into the LSTM unit, and obtaining a sentence vector through the LSTM unit based on the target information and the memorized historical target information; inputting the sentence vectors into the classifier, and classifying the sentence vectors through the classifier to obtain an early warning level;
the step of judging whether an abnormal target exists in the designated area according to the video frame image comprises the following steps: judging whether a difference exists between the video frame image and a preset image or not; the video frame image and the preset image are shot images of a designated area, and no abnormal object exists in the designated area in the preset image; if yes, determining that an abnormal target exists in the specified area; or comparing the difference between a plurality of video frame images to judge whether the specified area has an abnormal target;
the step of determining the target information of the abnormal target includes: tracking and positioning the abnormal target by utilizing a plurality of continuously acquired video frame images, and determining the target position, the target speed and the target track of the abnormal target; wherein the target track comprises a target motion track and/or a target prediction track; performing target identification on the video frame image through a preset target identification model, and determining the target category of the abnormal target; wherein the target identification model is a deep convolutional neural network model;
the training step of the grading early warning model comprises the following steps: acquiring a preset amount of training data; wherein the training data carries labels of different early warning levels; the training data comprises target types, target positions, target sizes, target speeds and target tracks; and training the grading early warning model by adopting the training data, and stopping training until the loss function value of the grading early warning model converges to a preset threshold value.
2. The method of claim 1, wherein the target recognition model is a google lenet network model.
3. The method of claim 1, wherein the LSTM unit comprises a LSTM layer, an average pooling layer, and a logistic regression layer connected in sequence.
4. An intrusion early warning device, the device is used for railway intrusion monitoring, its characterized in that includes:
the abnormity judgment module is used for judging whether an abnormal target exists in the designated area according to the video frame image; wherein the video frame image is a shot image of the designated area;
the target determining module is used for determining the target information of the abnormal target when the judgment result of the abnormal judging module is yes; the target information comprises a target type, a target position, a target size, a target speed and a target track;
the level generation module is used for generating an early warning level corresponding to the target information through a pre-trained hierarchical early warning model; wherein, the grading early warning model is a deep neural network model;
the early warning module is used for carrying out early warning according to the early warning level;
the grading early warning model comprises a feature extractor and a classifier; the feature extractor comprises a keyword extraction unit and an LSTM unit;
the level generation module is to: inputting the target information into a keyword extraction unit, and extracting keywords of the target information through the keyword extraction unit to obtain a word vector; inputting the word vector into an LSTM unit, and obtaining a sentence vector through the LSTM unit based on target information and memorized historical target information; inputting the sentence vectors into a classifier, and classifying the sentence vectors through the classifier to obtain an early warning level;
the abnormity judging module is used for: judging whether a difference exists between the video frame image and a preset image or not; the video frame image and the preset image are shot images of a designated area, and no abnormal object exists in the designated area in the preset image; if yes, determining that an abnormal target exists in the specified area; or comparing the difference between a plurality of video frame images to judge whether the specified area has an abnormal target;
the goal determination module is to: tracking and positioning an abnormal target by using a plurality of continuously collected video frame images, and determining the target position, the target speed and the target track of the abnormal target; the target track comprises a target motion track and/or a target prediction track; performing target identification on the video frame image through a preset target identification model, and determining the target category of an abnormal target; wherein, the target recognition model is a deep convolutional neural network model;
the device also comprises a grading early warning model training module used for acquiring training data with preset quantity; wherein the training data carries labels of different early warning levels; the training data comprises target types, target positions, target sizes, target speeds and target tracks; and training the grading early warning model by adopting the training data, and stopping training until the loss function value of the grading early warning model converges to a preset threshold value.
5. An electronic device comprising a memory for storing a program that enables a processor to perform the method of any of claims 1 to 3 and a processor configured to execute the program stored in the memory.
6. A machine-readable storage medium having stored thereon program code executable by a processor, the program code configured to cause the processor to perform the method of any of claims 1 to 3.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811256568.4A CN109410496B (en) | 2018-10-25 | 2018-10-25 | Intrusion early warning method and device and electronic equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811256568.4A CN109410496B (en) | 2018-10-25 | 2018-10-25 | Intrusion early warning method and device and electronic equipment |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN109410496A CN109410496A (en) | 2019-03-01 |
| CN109410496B true CN109410496B (en) | 2022-04-01 |
Family
ID=65469157
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201811256568.4A Active CN109410496B (en) | 2018-10-25 | 2018-10-25 | Intrusion early warning method and device and electronic equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109410496B (en) |
Families Citing this family (31)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109903499A (en) * | 2019-03-27 | 2019-06-18 | 河南九乾电子科技有限公司 | The intelligent control method and device of wireless self-organization network |
| EP3731203B1 (en) | 2019-04-24 | 2023-05-31 | Carrier Corporation | Alarm system |
| CN110119730A (en) * | 2019-06-03 | 2019-08-13 | 齐鲁工业大学 | A kind of monitor video processing method, system, terminal and storage medium |
| CN110060441A (en) * | 2019-06-14 | 2019-07-26 | 三星电子（中国）研发中心 | Method and apparatus for terminal anti-theft |
| CN110363098B (en) * | 2019-06-24 | 2021-08-13 | 深圳市中电数通智慧安全科技股份有限公司 | Violent behavior early warning method and device, readable storage medium and terminal equipment |
| CN110334167B (en) * | 2019-06-26 | 2023-06-09 | 浪潮软件集团有限公司 | Position deviation early warning method and device based on neural network track |
| CN110222675A (en) * | 2019-07-02 | 2019-09-10 | 同略科技有限公司 | A kind of police strength dispatching method, system, terminal and storage medium |
| CN110379125A (en) * | 2019-07-24 | 2019-10-25 | 广东电网有限责任公司 | Cross the border recognition methods, system and relevant apparatus for a kind of danger zone |
| CN110378300A (en) * | 2019-07-24 | 2019-10-25 | 广东电网有限责任公司 | A kind of hybrid recognition methods of material elevator people goods, system and relevant apparatus |
| CN110533889B (en) * | 2019-08-30 | 2021-08-13 | 中国电子科技网络信息安全有限公司 | Sensitive area electronic equipment monitoring and positioning device and method |
| CN110481601B (en) * | 2019-09-04 | 2022-03-08 | 深圳市镭神智能系统有限公司 | Track detection system |
| CN110674761B (en) * | 2019-09-27 | 2022-07-15 | 三星电子（中国）研发中心 | Regional behavior early warning method and system |
| CN110927731B (en) * | 2019-11-15 | 2021-12-17 | 深圳市镭神智能系统有限公司 | Three-dimensional protection method, three-dimensional detection device and computer readable storage medium |
| CN112989882A (en) * | 2019-12-16 | 2021-06-18 | 陈军 | Invasive species type identification system |
| CN111141412A (en) * | 2019-12-25 | 2020-05-12 | 深圳供电局有限公司 | Cable temperature and anti-theft dual-monitoring method and system and readable storage medium |
| CN113283273A (en) * | 2020-04-17 | 2021-08-20 | 上海锐明轨交设备有限公司 | Front obstacle real-time detection method and system based on vision technology |
| CN111523472A (en) * | 2020-04-23 | 2020-08-11 | 杭州海康威视系统技术有限公司 | Active target counting method and device based on machine vision |
| CN111541834B (en) * | 2020-04-30 | 2022-04-26 | 深圳市凯木金科技有限公司 | Outdoor camera, alarm method thereof and computer readable storage medium |
| CN111639765A (en) * | 2020-05-15 | 2020-09-08 | 视若飞信息科技(上海)有限公司 | Interaction method for using point track and detection domain |
| CN112329974B (en) * | 2020-09-03 | 2024-02-27 | 中国人民公安大学 | LSTM-RNN-based civil aviation security event behavior subject identification and prediction method and system |
| CN112288975A (en) * | 2020-11-13 | 2021-01-29 | 珠海大横琴科技发展有限公司 | Event early warning method and device |
| CN112528791A (en) * | 2020-12-03 | 2021-03-19 | 山东星宏电讯有限责任公司 | Anti-intrusion system and method for railway station yard |
| CN112863100B (en) * | 2020-12-31 | 2022-09-06 | 山东奥邦交通设施工程有限公司 | Intelligent construction safety monitoring system and method |
| CN112776856A (en) * | 2021-01-15 | 2021-05-11 | 中国神华能源股份有限公司神朔铁路分公司 | Track foreign matter intrusion monitoring method, device and system and monitoring host equipment |
| CN112686895B (en) * | 2021-03-11 | 2021-05-18 | 成都点泽智能科技有限公司 | Method and device for detecting infrared dim target and server |
| CN113033355B (en) * | 2021-03-11 | 2023-04-07 | 中北大学 | Abnormal target identification method and device based on intensive power transmission channel |
| CN115471972A (en) * | 2022-08-23 | 2022-12-13 | 深圳市中燃科技有限公司 | Pipe network abnormity early warning method, device, equipment and storage medium |
| CN115909215B (en) * | 2022-12-09 | 2023-07-14 | 厦门农芯数字科技有限公司 | Edge intrusion early warning method and system based on target detection |
| CN116499589B (en) * | 2023-06-16 | 2023-09-01 | 长春理工大学 | Multispectral polarization imaging position detection integrated device and imaging method |
| CN116631134A (en) * | 2023-07-26 | 2023-08-22 | 海南至元科技有限公司 | Intelligent monitoring anti-theft device |
| CN117830911A (en) * | 2024-03-06 | 2024-04-05 | 一脉通(深圳)智能科技有限公司 | Intelligent identification method and device for intelligent camera, electronic equipment and medium |
Family Cites Families (25)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN100585656C (en) * | 2007-03-14 | 2010-01-27 | 中国科学院自动化研究所 | An all-weather intelligent video analysis monitoring method based on a rule |
| CN101399672B (en) * | 2008-10-17 | 2011-03-02 | 章毅 | Intrusion detection method for fusion of multiple neutral networks |
| US8660368B2 (en) * | 2011-03-16 | 2014-02-25 | International Business Machines Corporation | Anomalous pattern discovery |
| CN104616003B (en) * | 2015-03-06 | 2016-05-04 | 国家电网公司 | Foreign matter identification system on power line |
| CN104787084B (en) * | 2015-04-16 | 2017-06-23 | 北京交通大学 | A kind of railway foreign body intrusion detecting system and detection method |
| CN105346565B (en) * | 2015-09-29 | 2017-12-01 | 北京京天威科技发展有限公司 | Invade limit intelligent monitor system and monitoring center database |
| CN206212165U (en) * | 2015-11-02 | 2017-05-31 | 克拉玛依市格恩赛电子科技有限公司 | A kind of over-the-horizon radar wide area joint defense system |
| CN105512633A (en) * | 2015-12-11 | 2016-04-20 | 谭焕玲 | Power system dangerous object identification method and apparatus |
| CN105809679B (en) * | 2016-03-04 | 2019-06-18 | 李云栋 | Mountain railway side slope rockfall detection method based on visual analysis |
| US10733231B2 (en) * | 2016-03-22 | 2020-08-04 | Sensormatic Electronics, LLC | Method and system for modeling image of interest to users |
| CN105915846B (en) * | 2016-04-26 | 2019-09-20 | 成都通甲优博科技有限责任公司 | A kind of the invader monitoring method and system of the multiplexing of list binocular |
| CN108073941A (en) * | 2016-11-17 | 2018-05-25 | 江南大学 | A kind of image, semantic generation method based on deep learning |
| KR20180081645A (en) * | 2017-01-06 | 2018-07-17 | 한양대학교 에리카산학협력단 | Embedded vision system for real-time detection of safety anomaly in subway screen door and method thereof |
| CN106971154A (en) * | 2017-03-16 | 2017-07-21 | 天津大学 | Pedestrian's attribute forecast method based on length memory-type recurrent neural network |
| CN106991410A (en) * | 2017-04-17 | 2017-07-28 | 合肥酷庆信息科技有限公司 | A kind of railway foreign body intrusion distributed intelligent monitoring system based on machine vision |
| CN107492113B (en) * | 2017-06-01 | 2019-11-05 | 南京行者易智能交通科技有限公司 | A kind of moving object in video sequences position prediction model training method, position predicting method and trajectory predictions method |
| CN107808139B (en) * | 2017-11-01 | 2021-08-06 | 电子科技大学 | Real-time monitoring threat analysis method and system based on deep learning |
| CN107862763B (en) * | 2017-11-06 | 2020-12-29 | 中国人民解放军国防科技大学 | Train safety early warning evaluation model training method, module and monitoring evaluation system |
| CN108055501A (en) * | 2017-11-22 | 2018-05-18 | 天津市亚安科技有限公司 | A kind of target detection and the video monitoring system and method for tracking |
| CN207502720U (en) * | 2017-11-29 | 2018-06-15 | 航科院（北京）科技发展有限公司 | A kind of airfield runway monitoring foreign bodies and removing guiding system |
| CN108124065A (en) * | 2017-12-05 | 2018-06-05 | 浙江鹏信信息科技股份有限公司 | A kind of method junk call content being identified with disposal |
| CN108171283B (en) * | 2017-12-31 | 2020-06-16 | 厦门大学 | Image content automatic description method based on structured semantic embedding |
| CN108399743B (en) * | 2018-02-07 | 2021-09-07 | 武汉理工大学 | Highway vehicle abnormal behavior detection method based on GPS data |
| CN108446630B (en) * | 2018-03-20 | 2019-12-31 | 平安科技（深圳）有限公司 | Intelligent monitoring method for airport runway, application server and computer storage medium |
| CN108600701B (en) * | 2018-05-02 | 2020-11-24 | 广州飞宇智能科技有限公司 | Monitoring system and method for judging video behaviors based on deep learning |
-
2018
- 2018-10-25 CN CN201811256568.4A patent/CN109410496B/en active Active
Also Published As
| Publication number | Publication date |
|---|---|
| CN109410496A (en) | 2019-03-01 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109410496B (en) | Intrusion early warning method and device and electronic equipment | |
| JP7272533B2 (en) | Systems and methods for evaluating perceptual systems | |
| CN111986228B (en) | Pedestrian tracking method, device and medium based on LSTM model escalator scene | |
| US20130170696A1 (en) | Clustering-based object classification | |
| JP6532106B2 (en) | Monitoring device, monitoring method and program for monitoring | |
| CN113484858A (en) | Intrusion detection method and system | |
| KR101900237B1 (en) | On-site judgment method using situation judgment data on acquired images | |
| CN112776856A (en) | Track foreign matter intrusion monitoring method, device and system and monitoring host equipment | |
| CN114218992B (en) | Abnormal object detection method and related device | |
| KR20220146670A (en) | Traffic anomaly detection methods, devices, devices, storage media and programs | |
| CN113936465A (en) | Traffic incident detection method and device | |
| Thiruppathiraj et al. | Automatic pothole classification and segmentation using android smartphone sensors and camera images with machine learning techniques | |
| Makhmutova et al. | Intelligent detection of object's anomalies for road surveilance cameras | |
| CN110458144A (en) | Object area intrusion detection method, system, device and readable storage medium storing program for executing | |
| Fratama et al. | Real-time multiple vehicle counter using background subtraction for traffic monitoring system | |
| Bravi et al. | Detection of stop sign violations from dashcam data | |
| Karim et al. | An attention-guided multistream feature fusion network for localization of risky objects in driving videos | |
| Tschürtz et al. | System of systems safety analysis and evaluation in ZalaZONE | |
| Yadav et al. | Post-Crash Detection and Traffic Analysis | |
| CN110333517B (en) | Obstacle sensing method, obstacle sensing device and storage medium | |
| Adewopo et al. | Smart city transportation: Deep learning ensemble approach for traffic accident detection | |
| Zhang et al. | Analyzing influence of robustness of neural networks on the safety of autonomous vehicles | |
| KR20210114169A (en) | Method for analyzing monitoring image using object verification, and apparatus for the same | |
| CN110827316A (en) | Crowd panic scatter detection method and system, readable storage medium and electronic equipment | |
| Yatbaz et al. | Run-time Introspection of 2D Object Detection in Automated Driving Systems Using Learning Representations |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |