CN112200101B - Video monitoring and analyzing method for maritime business based on artificial intelligence - Google Patents

Video monitoring and analyzing method for maritime business based on artificial intelligence Download PDF

Info

Publication number
CN112200101B
CN112200101B CN202011102923.XA CN202011102923A CN112200101B CN 112200101 B CN112200101 B CN 112200101B CN 202011102923 A CN202011102923 A CN 202011102923A CN 112200101 B CN112200101 B CN 112200101B
Authority
CN
China
Prior art keywords
frame
identification
tracking
target object
target
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202011102923.XA
Other languages
Chinese (zh)
Other versions
CN112200101A (en
Inventor
赵睿
杜红飞
万为东
李超
王华东
赵志明
许宁
路轩轩
徐顺
张曼霞
王鹏
崔敬涛
顾鹏飞
郎亚辉
王文才
柳小涛
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Henan Zhonggong Design and Research Institute Group Co.,Ltd.
Original Assignee
Henan Provincial Communication Planning and Design Institute Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Henan Provincial Communication Planning and Design Institute Co Ltd filed Critical Henan Provincial Communication Planning and Design Institute Co Ltd
Priority to CN202011102923.XA priority Critical patent/CN112200101B/en
Publication of CN112200101A publication Critical patent/CN112200101A/en
Application granted granted Critical
Publication of CN112200101B publication Critical patent/CN112200101B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N7/00—Television systems
    • H04N7/18—Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/40—Scenes; Scene-specific elements in video content
    • G06V20/41—Higher-level, semantic clustering, classification or understanding of video scenes, e.g. detection, labelling or Markovian modelling of sport events or news items

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • General Physics & Mathematics (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Artificial Intelligence (AREA)
  • Multimedia (AREA)
  • Software Systems (AREA)
  • Computational Linguistics (AREA)
  • Evolutionary Computation (AREA)
  • General Engineering & Computer Science (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Health & Medical Sciences (AREA)
  • Biomedical Technology (AREA)
  • Biophysics (AREA)
  • Evolutionary Biology (AREA)
  • General Health & Medical Sciences (AREA)
  • Molecular Biology (AREA)
  • Computing Systems (AREA)
  • Mathematical Physics (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Signal Processing (AREA)
  • Image Analysis (AREA)

Abstract

The invention discloses a video monitoring and analyzing method facing to maritime affairs based on artificial intelligence, which comprises the following steps that 1, a target object in each frame identification area of a video data source is identified by adopting an identification algorithm; 2, distinguishing and marking the target objects of the front frame and the rear frame of the video data source in the buffer area through a marking algorithm, finishing the non-repeated marking of the same target object and ensuring the uniqueness of the identified object; intercepting an identification area from each frame of the video, tracking the identification object in an internal area of the buffer area by using a tracking algorithm, tracking the track of the target object out of the buffer area, and ensuring that the target object is not disordered under the influence of overlapping, shielding and re-separating factors in the tracking process so as to obtain the result of the identification object; and 4, recording the position of the tracking target object out of the identification area, and performing behavior analysis and statistics on the tracking target object by using the position. The invention fully utilizes the video monitoring equipment of the existing inland waterway, thereby greatly saving the equipment replacement cost.

Description

Video monitoring and analyzing method for maritime business based on artificial intelligence
Technical Field
The invention relates to the field of inland waterway monitoring management, in particular to a video monitoring and analyzing method for maritime business based on artificial intelligence.
Background
In recent years, with the increasing number of touring ships, freight ships, ferrying ships and river resource development operation ships in inland waterways, hundreds of water traffic safety accidents are caused every year, hundreds of casualties and immeasurable property losses are caused, and great challenges are brought to supervision work of maritime departments.
In order to enhance the navigation control of inland river navigation sections, at present, the maritime department usually adopts the Automatic Identification of Ships (AIS) and the very high frequency shore ship data communication system (VHF) technology as main technologies at key docks and assists with the channel video technical means of a VTS radar and a closed circuit television monitoring system (CCTV) to carry out the safety supervision of various ships. The automatic ship identification system (AIS) is matched with a Global Positioning System (GPS) to broadcast ship static information such as ship position, ship speed, changed course rate and course and the like dynamically combined with ship names, call signs, draught, dangerous goods and the like to ships and shore stations in nearby water areas through Very High Frequency (VHF) channels, so that the adjacent ships and shore stations can timely master dynamic and static information of all ships on nearby water surfaces, mutual communication and coordination can be realized at once, necessary avoidance actions are taken, and great help is brought to the safety of the ships.
The inland river very high frequency shore vessel data communication system works in a Very High Frequency (VHF) wave band, is one of the most main communication means of inland river and offshore radio mobile services, can carry out ship distress, emergency, safe communication and daily service communication, and is also an important communication tool of search and rescue operation, coordination and avoidance among ships and a ship traffic service system. However, AIS and VTS radars also have exposed many drawbacks in the practical application of electronic cruise, and cannot meet the business requirements of intelligent maritime supervision. For example, AIS has signal blind areas, many ships are not opened or equipped with AIS for various reasons, and the information fusion of AIS and VTS radar is not perfect. The main performance lies in that AIS lacks visual observation and control capability of field conditions, and simultaneously, with continuous improvement of business requirements of a maritime department, the defects of an early-stage standard definition video monitoring system also appear in the application process, for example, when a ship is overspeed, overloaded, turns around randomly or overtakes, an original standard definition video camera cannot provide effective image details, particularly cannot see the name of the ship clearly, and great inconvenience is brought to maritime supervision and law enforcement personnel.
Disclosure of Invention
The invention aims to provide a video monitoring and analyzing method for marine business based on artificial intelligence, which mainly realizes ship tracking and monitoring by taking a video means as a main means and effectively makes up for the defects of the existing ship positioning equipment.
In order to achieve the purpose, the invention adopts the following technical scheme:
the invention relates to a maritime business-oriented video monitoring and analyzing method based on artificial intelligence, which comprises the following steps:
step 1, identifying a target object in each frame identification area of a video data source by adopting an identification algorithm so as to complete identification of the target object by applying an irregular identification area; namely: sequentially reading each frame of a video data source in sequence and setting an identification area on each frame;
step 2, distinguishing and marking the target objects of the front frame and the rear frame of the video data source in a buffer area through a marking algorithm, completing the non-repeated marking of the same target object, distinguishing new and old target objects and ensuring the uniqueness of an identified object; namely: setting a buffer area according to a set reduction ratio according to the set identification area of each frame, wherein the buffer area is superposed with the central point of the identification area;
step 3, tracking the identification object in the internal area of the buffer area by using a tracking algorithm, tracking the track of the target object out of the buffer area, and ensuring that the target object is not disordered under the influence of overlapping, shielding and re-separating factors in the tracking process; namely: intercepting an identification area from each frame of the video, and processing the identification area by using an identification algorithm to obtain a result of an identification object, specifically: intercepting a recognition area from a video picture by using OpenCV, recognizing objects in an input model video interval frame by adopting a trained YOLO recognition model, and obtaining a recognition object list;
YOLO (You Only Look one) is an existing object recognition and positioning algorithm based on a deep neural network, and has the biggest characteristic of high operation speed and can be used for a real-time system; the innovation point of YOLO is that a region suggestion frame type detection framework is improved, the two stages of a candidate region and an object recognition are combined into a whole, and a predefined candidate region is adopted; dividing the whole image, wherein each part is responsible for target detection centered on the part, and predicting candidate frames, positioning confidence degrees and probability vectors of all classes of targets contained in each part at one time; after removing the candidate region, the structure of YOLO includes convolution, pooling and the last two fully-connected layers; the maximum difference is that the final output layer uses a linear function as an activation function to predict the position of a candidate frame and the probability of an object, and the YOLO target detection step is as follows:
step 3.1, read
Figure 100002_DEST_PATH_IMAGE002
Frame image, call reserve function (function for adjusting picture size) to adjust image size, and divide image into parts
Figure DEST_PATH_IMAGE003
A grid;
step 3.2, performing feature extraction on the image by using a convolutional neural network;
step 3.3, predicting the position and the type of the target: if the center of a target object falls in a grid, the grid is responsible for predicting the target object; to predict per mesh
Figure 100002_DEST_PATH_IMAGE004
In a candidate frame
Figure DEST_PATH_IMAGE005
Confidence and
Figure 100002_DEST_PATH_IMAGE006
a category of (1); an output of magnitude
Figure DEST_PATH_IMAGE007
The tensor of (a);
Figure 328594DEST_PATH_IMAGE008
in order to divide the number of the meshes,
Figure DEST_PATH_IMAGE009
the number of frames responsible for each mesh,
Figure 711647DEST_PATH_IMAGE010
the number of categories; each mesh will correspond to
Figure 897909DEST_PATH_IMAGE004
A boundary frame with width and heightThe range is a full graph and represents the position of a bounding box of the object which is searched by taking the grid as the center; each bounding box corresponds to a score which represents whether an object exists at the position and the positioning accuracy:
Figure DEST_PATH_IMAGE011
each grid corresponds to
Figure 27539DEST_PATH_IMAGE012
Probability value, finding out the category corresponding to the maximum probability
Figure DEST_PATH_IMAGE013
And the object or a portion of the object is considered to be contained in the grid; each grid corresponding to
Figure 186119DEST_PATH_IMAGE014
The information contained in the dimensional vector is as follows:
1、
Figure DEST_PATH_IMAGE015
the probability of each object classification can be expressed as:
Figure 477423DEST_PATH_IMAGE016
；
Figure DEST_PATH_IMAGE017
indicating the existence of the grid
Figure 354243DEST_PATH_IMAGE018
The probability of (d);
2、
Figure DEST_PATH_IMAGE019
the position information of each candidate frame includes a center point
Figure 348217DEST_PATH_IMAGE020
The coordinates,
Figure DEST_PATH_IMAGE021
Coordinates of the objectCandidate frame widthwCandidate frame heighth(Center_x,Center_y,width,height)，
Figure 751517DEST_PATH_IMAGE022
A candidate frame is commonly required
Figure DEST_PATH_IMAGE023
A number value to indicate its position;
3、
Figure 89089DEST_PATH_IMAGE024
the confidence formula for the confidence candidates for each candidate box is:
Figure DEST_PATH_IMAGE025
；
Figure 984363DEST_PATH_IMAGE026
a confidence level expressed as a target object;
Figure DEST_PATH_IMAGE027
is the probability of an object existing within the candidate box, as distinguished from
Figure 334092DEST_PATH_IMAGE028
；
Figure DEST_PATH_IMAGE029
The method embodies the degree of closeness of the predicted candidate frame and the real target frame;
4. traversing the scores, excluding objects with lower scores and higher overlapping degrees, and outputting predicted objects;
and 4, recording the position of the tracking target object out of the identification area, and performing behavior analysis and statistics on the tracking target object by using the position.
The invention relates to an existing video monitoring device in inland river navigation waters, which comprises a shore-based video monitoring device mainly comprising a port and a wharf and a video monitoring device in a cabin, and realizes automatic video monitoring and statistical analysis for maritime affairs by means of technologies such as big data, cloud computing, artificial intelligence, machine learning and the like. The specific application fields comprise ship identification, ship tracking, ship running track monitoring, port ship entry and exit statistics, ship personnel behavior analysis and the like.
On one hand, the invention solves the problems that the prior video monitoring equipment only supports remote viewing and can not complete related maritime supervision services in an automatic mode, and the traditional manual means not only consumes time and labor, but also can not dispose emergent emergency events in time and the like; on the other hand, the problem of software and hardware binding is solved, the software and the hardware are separated, video monitoring equipment of the existing inland waterway can be fully utilized, and equipment replacement cost is greatly saved.
Drawings
FIG. 1 is a flow chart of the present invention.
FIG. 2 is a flow chart of the YOLO algorithm flow described in this invention.
FIG. 3 is a flow chart of the KCF filtering algorithm of the present invention.
Fig. 4 is a schematic diagram of setting an identification area on each frame according to the present invention.
Fig. 5 is a schematic diagram of the present invention setting a buffer area in the identification area.
Fig. 6 is a schematic diagram of determining new and old objects in step 3.5 according to the embodiment of the present invention.
FIG. 7 is a diagram illustrating the object crossing the buffer at step 3.6 according to the embodiment of the present invention.
FIG. 8 is a diagram of the trigger buffer for tracking the position of the object in step 3.7 according to the embodiment of the present invention.
FIG. 9 is a diagram illustrating the behavior of determining the tracking object in step 4 according to the embodiment of the present invention.
Detailed Description
The following describes embodiments of the present invention in detail with reference to the accompanying drawings, which are implemented on the premise of the technical solution of the present invention, and give detailed implementation manners and specific operation procedures, but the scope of the present invention is not limited to the following embodiments.
The invention relates to a video monitoring and analyzing method for marine business based on artificial intelligence, which realizes the automatic video monitoring and statistical analysis for the marine business by relying on technologies such as big data, cloud computing, artificial intelligence, machine learning and the like. As shown in fig. 1, the steps are as follows:
step 1, identifying a target object in each frame identification area of a video data source by adopting an identification algorithm so as to be suitable for an irregular identification area to complete the complete identification of the target object; namely: sequentially reading each frame of the video data source in sequence and setting an identification area 1 on each frame, as shown in fig. 4;
step 2, distinguishing and marking the target objects of the front frame and the rear frame of the video data source in the buffer area 2 through a marking algorithm, completing the non-repeated marking of the same target object, distinguishing new and old target objects and ensuring the uniqueness of the identified object; namely: setting a buffer area 2 according to a set reduction ratio (for example, 95%) according to the identification area 1 set for each frame, wherein the buffer area 2 is shown as a shaded area in fig. 5, and the buffer area 2 coincides with the center point of the identification area 1; the buffer 2 is set for: 1. eliminating the influence of the moving object under the condition of constant change of the identification area 1; 2. distinguishing new and old objects so as to mark the objects or update the objects;
step 3, intercepting the identification area 1 from each frame of the video, and processing the identification area by using an identification algorithm to obtain the result of the identification object, namely: intercepting a recognition area 1 from a video picture by using OpenCV, recognizing objects in an interval frame of an input model video by adopting a trained YOLO recognition model, and obtaining a recognition object list; the method comprises the following specific steps:
step 3.1, as shown in fig. 2, the step of YOLO target detection:
step 3.1.1, read
Figure 732843DEST_PATH_IMAGE030
Frame image, call up
Figure DEST_PATH_IMAGE031
Function operations resize images, divide images into
Figure 631529DEST_PATH_IMAGE032
A grid;
step 3.1.2, performing feature extraction on the image by using a convolutional neural network;
step 3.1.3, predicting the position and the type of the target: if the center of a target object falls in the grid, the grid is responsible for predicting the target object; to predict per mesh
Figure DEST_PATH_IMAGE033
In a candidate frame
Figure 482942DEST_PATH_IMAGE034
Confidence and
Figure DEST_PATH_IMAGE035
a category of (1); an output of magnitude
Figure 99343DEST_PATH_IMAGE036
The tensor of (a);
Figure DEST_PATH_IMAGE037
in order to divide the number of the meshes,
Figure 477235DEST_PATH_IMAGE038
the number of frames responsible for each mesh,
Figure DEST_PATH_IMAGE039
the number of the categories; each mesh will correspond to
Figure 687768DEST_PATH_IMAGE040
The wide-height range of the bounding box is a full graph and represents the position of the bounding box for finding an object by taking the grid as a center; each bounding box corresponds to a score which represents whether an object exists at the position and the positioning accuracy:
Figure DEST_PATH_IMAGE041
each grid corresponds to
Figure 698580DEST_PATH_IMAGE042
Probability value, finding out the category corresponding to the maximum probability
Figure DEST_PATH_IMAGE043
And the object or a portion of the object is considered to be contained in the grid; each grid corresponding to
Figure 643972DEST_PATH_IMAGE044
The information contained in the dimensional vector is as follows:
1，
Figure DEST_PATH_IMAGE045
the probability of each object classification can be expressed as:
Figure 610791DEST_PATH_IMAGE046
，
Figure DEST_PATH_IMAGE047
for which there is an object
Figure 461066DEST_PATH_IMAGE048
The probability of (d);
2，
Figure DEST_PATH_IMAGE049
the position information of the candidate frame includes a center point
Figure 146125DEST_PATH_IMAGE050
The coordinates,
Figure DEST_PATH_IMAGE051
Coordinates, frame candidate width w, frame candidate height h (Center _ x, center _ y, width, height),
Figure 107259DEST_PATH_IMAGE052
a candidate frame is needed
Figure DEST_PATH_IMAGE053
A number value to indicate its position;
3，
Figure 663006DEST_PATH_IMAGE054
confidence of each candidate box:
the confidence formula for the candidate box is:
Figure DEST_PATH_IMAGE055
；
Figure 212411DEST_PATH_IMAGE056
a confidence level expressed as a target object;
Figure DEST_PATH_IMAGE057
the representation target is
Figure 119187DEST_PATH_IMAGE058
The probability of (d);
Figure DEST_PATH_IMAGE059
the degree of closeness of the predicted candidate frame and the real candidate frame is embodied;
step 3.1.4, traversing the scores, excluding objects with lower scores and higher overlapping degrees, and outputting predicted objects;
step 3.2, loss function of yolo algorithm:
the YOLO algorithm considers target detection as a regression problem, adopts a mean square error loss function, and adopts different weight values for different parts; firstly distinguishing positioning error and classification error, adopting larger positioning error, namely boundary frame coordinate prediction error, then distinguishing confidence coefficient of boundary frame not containing target and boundary frame containing target, and adopting smaller weight value for former boundary frame
Figure 743067DEST_PATH_IMAGE060
All other weighted values are set to be 1; then, the mean square error is adopted, the mean square error equally treats the boundary boxes with different sizes, the prediction of the width and the height of the network boundary box is changed into the prediction of the square root, namely the predicted value is changed into the prediction of the square root
Figure DEST_PATH_IMAGE061
(ii) a For classification errors it means that there is a mesh of objects to account for the error; the error formula is as follows:
Figure 559844DEST_PATH_IMAGE062
coordinate prediction error:
Figure DEST_PATH_IMAGE063
；
Figure 142135DEST_PATH_IMAGE064
a weight value when a target object is present;
Figure DEST_PATH_IMAGE065
is an accumulation operation;
Figure 414503DEST_PATH_IMAGE066
as a candidate frame center point
Figure DEST_PATH_IMAGE067
The coordinates of the position of the object to be imaged,
Figure 842073DEST_PATH_IMAGE068
as the center point of the candidate frame
Figure DEST_PATH_IMAGE069
Fourier transform of the coordinates;
Figure DEST_PATH_IMAGE071
as the centre of a candidate frame
Figure 778937DEST_PATH_IMAGE072
The coordinates of the position of the object to be imaged,
Figure DEST_PATH_IMAGE073
as the center point of the candidate frame
Figure 407495DEST_PATH_IMAGE074
Squares of coordinate fourier transform values;
Figure DEST_PATH_IMAGE075
is the value of the square of the candidate frame width,
Figure 23284DEST_PATH_IMAGE076
a Fourier transform square value of the candidate frame width;
Figure DEST_PATH_IMAGE077
the square value of the candidate box height is,
Figure 254545DEST_PATH_IMAGE078
fourier transform square values of the candidate box heights;
Figure DEST_PATH_IMAGE079
accumulating each prediction frame and each grid in sequence;
Figure 777406DEST_PATH_IMAGE080
representation grid
Figure DEST_PATH_IMAGE081
To (1)
Figure 904762DEST_PATH_IMAGE082
The object exists in each prediction frame, when the object exists, the difference between the coordinate and the Fourier transform of the object is calculated, each prediction frame and each grid are accumulated in sequence, and the result and the weight of the result are obtainedMultiplication of the weight values gives the prediction error of the coordinates, i.e.
Figure DEST_PATH_IMAGE083
Width w, height h error of frame
Figure 273427DEST_PATH_IMAGE084
The same as above;
Figure DEST_PATH_IMAGE085
confidence error of candidate box containing target object:
Figure 42799DEST_PATH_IMAGE086
Figure DEST_PATH_IMAGE087
representation grid
Figure 423096DEST_PATH_IMAGE088
To (1)
Figure DEST_PATH_IMAGE089
There is an object in each of the prediction boxes,
Figure 252512DEST_PATH_IMAGE090
is as follows
Figure DEST_PATH_IMAGE091
An object;
Figure 857542DEST_PATH_IMAGE092
is as follows
Figure DEST_PATH_IMAGE093
The square of the individual subject fourier transform values;
Figure 305972DEST_PATH_IMAGE094
in order toAccumulating each prediction frame and each grid;
when the object exists
Figure DEST_PATH_IMAGE095
Accumulating the difference of the square of the Fourier transform of the target object and each prediction frame and each grid in sequence, and calculating the confidence error of a candidate frame containing the target object;
confidence error for candidate box without target object:
Figure 71934DEST_PATH_IMAGE096
Figure DEST_PATH_IMAGE097
representation grid
Figure 337830DEST_PATH_IMAGE098
To (1)
Figure DEST_PATH_IMAGE099
No object exists in each prediction box;
Figure 149928DEST_PATH_IMAGE100
a weight value when no object is present;
Figure DEST_PATH_IMAGE101
is a first
Figure 664699DEST_PATH_IMAGE102
The number of the objects is one,
Figure DEST_PATH_IMAGE103
is as follows
Figure 878642DEST_PATH_IMAGE104
Fourier transform values of the individual subjects;
Figure DEST_PATH_IMAGE105
accumulating each prediction frame and each grid in sequence;
Figure 190806DEST_PATH_IMAGE106
class prediction error:
Figure DEST_PATH_IMAGE107
Figure 490200DEST_PATH_IMAGE108
is a first
Figure DEST_PATH_IMAGE109
Objects exist in the grids;
Figure 670646DEST_PATH_IMAGE110
a probability that the object is of a certain class;
Figure DEST_PATH_IMAGE111
the object is the square of a certain class of probability value Fourier transform;
Figure 614462DEST_PATH_IMAGE112
in order to accumulate over all of the categories,
Figure DEST_PATH_IMAGE113
accumulating for each grid;
by passing
Figure 959512DEST_PATH_IMAGE114
Making a difference with the square of Fourier transform, and then performing accumulation operation to calculate a category prediction error;
step 3.3, training of the YOLO network:
before trainingFirstly, pre-training is carried out on ImageNet (image training set), the pre-trained classification model adopts the first 53 convolutional layers, and 5 pooling layers and full-connection layers are added; testing of the network, of each candidate frame
Figure DEST_PATH_IMAGE115
(associated confidence score for target frame class)
Figure 746202DEST_PATH_IMAGE116
(ii) a Calculate to obtain each
Figure DEST_PATH_IMAGE117
Is/are as follows
Figure 340126DEST_PATH_IMAGE118
(relevant confidence score of target frame class), setting a threshold value, filtering out candidate frames with low score, and performing NMS (non-maximum suppression) processing on the reserved candidate frames to obtain a final detection result;
step 3.4, the marking algorithm, process the above identified object list, determine if it is a new object, the determination method uses optimized IOU (intersection ratio,
Figure DEST_PATH_IMAGE119
) Calculating the overlapping ratio of the new object area, the buffer area 2 and the area of the existing object list, if the overlapping ratio is greater than or equal to the threshold value, the new object is not determined, and if the overlapping ratio is less than the threshold value, the new object is determined; the threshold value is obtained by intersecting and comparing the cache region 2 with the intersection of the new object and the old object, and the value of the common threshold value is more than 0.5; the specific algorithm steps are as follows:
suppose that the coordinates of the upper left vertex and the lower right vertex of the recognition box A are divided into
Figure 263082DEST_PATH_IMAGE120
，
Figure DEST_PATH_IMAGE121
Identification frame
Figure 41682DEST_PATH_IMAGE122
The coordinates of the upper left vertex and the lower right vertex of (1) are divided into
Figure DEST_PATH_IMAGE123
,
Figure 315669DEST_PATH_IMAGE124
(ii) a For ease of understanding, the recognition box is described in an algorithmic language: the coordinates are converted into a matrix and,
Figure DEST_PATH_IMAGE125
、
Figure 444774DEST_PATH_IMAGE126
calculate the matrix
Figure DEST_PATH_IMAGE127
(integer value) if
Figure 753396DEST_PATH_IMAGE128
If the numerical value of the integer value in the matrix is less than 0, the identification frames are not intersected; if it is
Figure DEST_PATH_IMAGE129
If the numerical value of the integer value in the matrix is greater than 0, carrying out transformation multiplication on the matrix;
Figure 578264DEST_PATH_IMAGE130
Figure DEST_PATH_IMAGE131
；
Figure 73967DEST_PATH_IMAGE132
；
setting the size of the identified object and the measured threshold value to be 0.5;
step 3.5, as shown in fig. 6, judging the new and old objects: if the object is a new object, marking, establishing a tracker, recording information such as an ID (identity) and an initial position; if the object is not a new object, updating the initial position in the existing object tracker;
step 3.6, tracking algorithm: as shown in fig. 7, when the object passes through the buffer 2, a tracking algorithm is used to record the position of the object 3 at any time, and a KCF (Kernel Correlation Filter) filtering algorithm is used to mainly solve the problem of multi-object tracking overlap;
the KCF is a discrimination tracking method, which generally trains a target detector in the tracking process, uses the target detector to detect whether the next frame prediction position is a target, and then uses a new detection result to update a training set so as to update the target detector; when the target detector is trained, a target area is generally selected as a positive sample, the area around the target is a negative sample, and the probability that the area closer to the target is the positive sample is higher; as shown in fig. 3, the steps are as follows:
step 3.6.1, in
Figure DEST_PATH_IMAGE133
In the frame, at the current position
Figure 134327DEST_PATH_IMAGE134
Nearby sampling, training a regressor, wherein the regressor can calculate the response of small-window sampling;
step 3.6.2 at
Figure DEST_PATH_IMAGE135
In a frame, at a previous frame position
Figure 31876DEST_PATH_IMAGE136
Nearby sampling, and judging the response of each sample by using the regressor;
step 3.6.3, responding to the strongest sample as the frame position
Figure DEST_PATH_IMAGE137
：
Matrix algorithm including circulation matrix Fourier space diagonalization, fourier diagonalization simplified ridge regression, kernel space ridge regression; circulant matrix fourier diagonalization equation:
Figure 753276DEST_PATH_IMAGE138
；
Figure DEST_PATH_IMAGE139
is a circulant matrix;
Figure 267434DEST_PATH_IMAGE140
is a primary vector
Figure DEST_PATH_IMAGE141
Fourier transform of (1);
Figure 865906DEST_PATH_IMAGE142
is a fourier transform matrix;
upper label
Figure DEST_PATH_IMAGE143
Represents the conjugate transpose:
Figure 493328DEST_PATH_IMAGE144
in other words, the first and second electrodes,
Figure DEST_PATH_IMAGE145
similar to a diagonal matrix;
the ridge regression formula is:
Figure 784632DEST_PATH_IMAGE146
；
Figure DEST_PATH_IMAGE147
a regression coefficient matrix;
Figure 254927DEST_PATH_IMAGE148
regularization strength;
Figure DEST_PATH_IMAGE149
is a feature matrix;
Figure 795105DEST_PATH_IMAGE150
a target variable matrix is obtained;
Figure DEST_PATH_IMAGE151
linear least squares with regularization.
Step 3.7, as shown in fig. 8, when the buffer area 2 is triggered by the position of the tracked object 4, the object 4 is monitored at all times, and whether the position of the object is away from the identification area 2 is compared;
step 3.8, after the tracked object 4 leaves the buffer area 2, destroying the corresponding ID tracker, further judging the behavior of the tracked object 4 according to the azimuth relation between the initial area and the leaving area, carrying out classification statistics, outputting the behavior to a screen, and storing the behavior to a database;
suppose that the coordinates of the upper left vertex and the lower right vertex of the recognition box A are divided into
Figure 667247DEST_PATH_IMAGE152
Identification frame
Figure DEST_PATH_IMAGE153
Is divided into the coordinates of the upper left vertex and the lower right vertex
Figure 270397DEST_PATH_IMAGE154
,
Figure DEST_PATH_IMAGE155
(ii) a For ease of understanding, the identification box is described in terms of algorithmic logic:
the coordinates are converted into a matrix and,
Figure 227989DEST_PATH_IMAGE156
；
according toRespectively calculating the position of mass center from the coordinates of the recognition frame
Figure DEST_PATH_IMAGE157
；
Figure 168263DEST_PATH_IMAGE158
(ii) a The position of the dotted line is the central line of the identification area, and the central line position can be calculated
Figure DEST_PATH_IMAGE159
(ii) a Ignore
Figure 35856DEST_PATH_IMAGE160
Influence of coordinates, i.e. determination
Figure DEST_PATH_IMAGE161
Position of
Figure 671893DEST_PATH_IMAGE162
；
Step 4, recording the position of the tracking target object out of the identification area 1, and performing behavior analysis and statistics on the tracking target object by using the position; as shown in fig. 9, the steps are as follows:
step 4.1, if the starting centroid and the leaving centroid are both on the left side of the dotted line 5, the target object 6 is turned back for departure;
step 4.2, the starting centroid is on the left side of the dotted line 5, and the departure centroid is on the right side of the dotted line 5, and the departure target object 7 is the departure target object;
step 4.3, if the starting centroid and the leaving centroid are both on the right side of the dotted line 5, turning back a target object 8 for entering;
step 4.4, the starting centroid to the right of the dashed line 5 and the departure centroid to the left of the dashed line 5 is the inbound target object 9.

Claims (2)

1. A video monitoring and analyzing method facing to maritime affairs based on artificial intelligence is characterized in that: the method comprises the following steps:
step 1, identifying a target object in each frame identification area of a video data source by adopting an identification algorithm so as to be suitable for an irregular identification area to complete the complete identification of the target object; namely: sequentially reading each frame of a video data source in sequence and setting an identification area on each frame;
step 2, distinguishing and marking the target objects of the front frame and the rear frame of the video data source in a buffer area through a marking algorithm, completing the non-repeated marking of the same target object, distinguishing new and old target objects and ensuring the uniqueness of an identified object; setting a buffer area according to the set reduction ratio according to the set identification area of each frame, wherein the buffer area is arranged in the identification area and is superposed with the central point of the identification area;
step 3, intercepting an identification area from each frame of the video, tracking the identification object in an internal area of the buffer area by using a tracking algorithm, tracking the track of the target object out of the buffer area, and ensuring that the target object is not disordered under the influence of overlapping, shielding and re-separating factors in the tracking process, thereby obtaining the result of the identification object;
and 4, recording the position of the tracking target object out of the identification area, and performing behavior analysis and statistics on the tracking target object by using the position.
2. The artificial intelligence based maritime service oriented video monitoring and analysis method according to claim 1, wherein: in step 3, tracking the identified object by using a tracking algorithm, comprising the following steps:
step 3.1, reading the P frame image, calling the Resize function to adjust the image size, and dividing the image into
Figure DEST_PATH_IMAGE002
A grid;
step 3.2, performing feature extraction on the image by using a convolutional neural network;
step 3.3, predicting the position and the type of the target: if the center of a target object falls in a certain grid, the grid is responsible for predicting the target object; to predict per mesh
Figure DEST_PATH_IMAGE004
In a candidate frame
Figure DEST_PATH_IMAGE006
Confidence, and category; an output of magnitude of
Figure DEST_PATH_IMAGE008
The tensor of (a);
Figure DEST_PATH_IMAGE010
in order to divide the number of the meshes,
Figure DEST_PATH_IMAGE012
the number of frames responsible for each mesh,
Figure DEST_PATH_IMAGE014
the number of categories; each grid corresponds to
Figure DEST_PATH_IMAGE016
The wide-height range of the bounding box is a full graph and represents the position of the bounding box for finding an object by taking the grid as a center; each bounding box corresponds to a score which represents whether an object exists at the position and the positioning accuracy;
each mesh corresponds to
Figure DEST_PATH_IMAGE018
Probability value, finding out the category corresponding to the maximum probability
Figure DEST_PATH_IMAGE020
Wherein
Figure DEST_PATH_IMAGE022
Is composed of
Figure DEST_PATH_IMAGE024
Subject is at
Figure DEST_PATH_IMAGE026
Probability of occurrence under the condition, and considering the grid as containing the probabilityAn object or a part of the object;
each grid corresponding to
Figure DEST_PATH_IMAGE028
The information contained in the dimensional vector is as follows:
1、
Figure DEST_PATH_IMAGE030
the probability of each object classification can be expressed as:
Figure DEST_PATH_IMAGE032
，
Figure DEST_PATH_IMAGE034
indicating the presence of the grid object
Figure DEST_PATH_IMAGE036
The probability of (d);
2、
Figure DEST_PATH_IMAGE038
the position information of each candidate frame includes a center point
Figure DEST_PATH_IMAGE040
The coordinates,
Figure DEST_PATH_IMAGE042
Coordinates, candidate box width w, candidate box height h,
Figure DEST_PATH_IMAGE044
a candidate frame is commonly required
Figure DEST_PATH_IMAGE046
A number value to indicate its position;
3、
Figure DEST_PATH_IMAGE048
confidence for individual candidate boxes the confidence formula for the candidate box is:
Figure DEST_PATH_IMAGE050
；
Figure DEST_PATH_IMAGE052
a confidence level expressed as a target object;
Figure DEST_PATH_IMAGE054
the representation target is
Figure DEST_PATH_IMAGE056
The probability of (d);
Figure DEST_PATH_IMAGE058
representing the cross ratio of the real position and the predicted position;
Figure DEST_PATH_IMAGE060
is the probability of an object existing within the candidate box, as distinguished from
Figure DEST_PATH_IMAGE062
，
Figure DEST_PATH_IMAGE064
The degree of proximity of the predicted candidate frame and the actual target frame is reflected;
4. and traversing all the scores, excluding the objects with lower scores and higher overlapping degrees, and outputting predicted objects.
CN202011102923.XA 2020-10-15 2020-10-15 Video monitoring and analyzing method for maritime business based on artificial intelligence Active CN112200101B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202011102923.XA CN112200101B (en) 2020-10-15 2020-10-15 Video monitoring and analyzing method for maritime business based on artificial intelligence

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202011102923.XA CN112200101B (en) 2020-10-15 2020-10-15 Video monitoring and analyzing method for maritime business based on artificial intelligence

Publications (2)

Publication Number Publication Date
CN112200101A CN112200101A (en) 2021-01-08
CN112200101B true CN112200101B (en) 2022-10-14

Family

ID=74009065

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202011102923.XA Active CN112200101B (en) 2020-10-15 2020-10-15 Video monitoring and analyzing method for maritime business based on artificial intelligence

Country Status (1)

Country Link
CN (1) CN112200101B (en)

Families Citing this family (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP7462113B2 (en) * 2021-05-17 2024-04-04 Eizo株式会社 Information processing device, information processing method, and computer program
CN113516093B (en) * 2021-07-27 2024-09-10 浙江大华技术股份有限公司 Labeling method and device of identification information, storage medium and electronic device

Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101515378A (en) * 2009-03-17 2009-08-26 上海普适导航技术有限公司 Informationization management method for vessel entering and leaving port
CN104394507A (en) * 2014-11-13 2015-03-04 厦门雅迅网络股份有限公司 Method and system for solving alarm regional report omission through buffer zone
CN104766064A (en) * 2015-04-13 2015-07-08 郑州天迈科技股份有限公司 Method for recognizing and positioning access station and access field through vehicle-mounted video DVR images
CN105761490A (en) * 2016-04-22 2016-07-13 北京国交信通科技发展有限公司 Method of carrying out early warning on hazardous chemical substance transport vehicle parking in service area
CN107067447A (en) * 2017-01-26 2017-08-18 安徽天盛智能科技有限公司 A kind of integration video frequency monitoring method in large space region
WO2018008893A1 (en) * 2016-07-06 2018-01-11 주식회사 파킹패스 Off-street parking management system using tracking of moving vehicle, and method therefor
CN109684996A (en) * 2018-12-22 2019-04-26 北京工业大学 Real-time vehicle based on video passes in and out recognition methods
CN109785664A (en) * 2019-03-05 2019-05-21 北京悦畅科技有限公司 A kind of statistical method and device of the remaining parking stall quantity in parking lot

Family Cites Families (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110991272B (en) * 2019-11-18 2023-07-18 东北大学 Multi-target vehicle track recognition method based on video tracking

Patent Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101515378A (en) * 2009-03-17 2009-08-26 上海普适导航技术有限公司 Informationization management method for vessel entering and leaving port
CN104394507A (en) * 2014-11-13 2015-03-04 厦门雅迅网络股份有限公司 Method and system for solving alarm regional report omission through buffer zone
CN104766064A (en) * 2015-04-13 2015-07-08 郑州天迈科技股份有限公司 Method for recognizing and positioning access station and access field through vehicle-mounted video DVR images
CN105761490A (en) * 2016-04-22 2016-07-13 北京国交信通科技发展有限公司 Method of carrying out early warning on hazardous chemical substance transport vehicle parking in service area
WO2018008893A1 (en) * 2016-07-06 2018-01-11 주식회사 파킹패스 Off-street parking management system using tracking of moving vehicle, and method therefor
CN107067447A (en) * 2017-01-26 2017-08-18 安徽天盛智能科技有限公司 A kind of integration video frequency monitoring method in large space region
CN109684996A (en) * 2018-12-22 2019-04-26 北京工业大学 Real-time vehicle based on video passes in and out recognition methods
CN109785664A (en) * 2019-03-05 2019-05-21 北京悦畅科技有限公司 A kind of statistical method and device of the remaining parking stall quantity in parking lot

Non-Patent Citations (3)

* Cited by examiner, † Cited by third party
Title
Vessel Detection and Tracking Method Based on Video Surveillance;Natalia Wawrzyniak 等;《MDPI》;20191128;全文 *
基于深度学习的多船舶目标跟踪与流量统计;冼允廷等;《微型电脑应用》;20200320(第03期);全文 *
面向普速车站的智能到发作业系统设计与实现;吴兴华;《铁路计算机应用》;20180725;全文 *

Also Published As

Publication number Publication date
CN112200101A (en) 2021-01-08

Similar Documents

Publication Publication Date Title
JP6759474B2 (en) Vessel automatic tracking methods and systems based on deep learning networks and average shifts
CN105389567B (en) Group abnormality detection method based on dense optical flow histogram
CN109725310B (en) Ship positioning supervision system based on YOLO algorithm and shore-based radar system
CN110223302A (en) A kind of naval vessel multi-target detection method extracted based on rotary area
CN110097568A (en) A kind of the video object detection and dividing method based on the double branching networks of space-time
CN110060508B (en) Automatic ship detection method for inland river bridge area
CN112200101B (en) Video monitoring and analyzing method for maritime business based on artificial intelligence
CN110348304A (en) A kind of maritime affairs distress personnel search system being equipped on unmanned plane and target identification method
CN105184271A (en) Automatic vehicle detection method based on deep learning
CN109977897A (en) A kind of ship's particulars based on deep learning recognition methods, application method and system again
CN110378308A (en) The improved harbour SAR image offshore Ship Detection based on Faster R-CNN
CN104318258A (en) Time domain fuzzy and kalman filter-based lane detection method
CN110427981A (en) SAR ship detecting system and method based on deep neural network
CN101145200A (en) Inner river ship automatic identification system of multiple vision sensor information fusion
CN106778540B (en) Parking detection is accurately based on the parking event detecting method of background double layer
CN110458160A (en) A kind of unmanned boat waterborne target recognizer based on depth-compression neural network
CN113808282A (en) Multi-navigation-factor data fusion method
Bloisi et al. Camera based target recognition for maritime awareness
CN110334703B (en) Ship detection and identification method in day and night image
CN111091095A (en) Method for detecting ship target in remote sensing image
Ozcelik et al. A vision based traffic light detection and recognition approach for intelligent vehicles
CN113743260B (en) Pedestrian tracking method under condition of dense pedestrian flow of subway platform
CN116434159A (en) Traffic flow statistics method based on improved YOLO V7 and Deep-Sort
Zhang et al. A warning framework for avoiding vessel‐bridge and vessel‐vessel collisions based on generative adversarial and dual‐task networks
Zhu et al. Arbitrary-oriented ship detection based on retinanet for remote sensing images

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant
CP03 Change of name, title or address
CP03 Change of name, title or address

Address after: 450046 No.9 Zeyu street, Zhengdong New District, Zhengzhou City, Henan Province

Patentee after: Henan Zhonggong Design and Research Institute Group Co.,Ltd.

Country or region after: China

Address before: 450046 No.9 Zeyu street, Zhengdong New District, Zhengzhou City, Henan Province

Patentee before: HENAN PROVINCIAL COMMUNICATIONS PLANNING & DESIGN INSTITUTE Co.,Ltd.

Country or region before: China